feat: add application log ingestion with OpenSearch storage
Agents can now send application log entries in batches via POST /api/v1/data/logs.
Logs are indexed directly into OpenSearch daily indices (logs-{yyyy-MM-dd}) using
the bulk API. Index template defines explicit mappings for full-text search readiness.
New DTOs (LogEntry, LogBatch) added to cameleer3-common in the agent repo.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
18
HOWTO.md
18
HOWTO.md
@@ -100,7 +100,7 @@ JWTs carry a `roles` claim. Endpoints are restricted by role:
|
||||
|
||||
| Role | Access |
|
||||
|------|--------|
|
||||
| `AGENT` | Data ingestion (`/data/**`), heartbeat, SSE events, command ack |
|
||||
| `AGENT` | Data ingestion (`/data/**` — executions, diagrams, metrics, logs), heartbeat, SSE events, command ack |
|
||||
| `VIEWER` | Search, execution detail, diagrams, agent list |
|
||||
| `OPERATOR` | VIEWER + send commands to agents |
|
||||
| `ADMIN` | OPERATOR + user management (`/admin/**`) |
|
||||
@@ -220,6 +220,20 @@ curl -s -X POST http://localhost:8081/api/v1/data/metrics \
|
||||
-H "X-Protocol-Version: 1" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '[{"agentId":"agent-1","metricName":"cpu","value":42.0,"timestamp":"2026-03-11T00:00:00Z","tags":{}}]'
|
||||
|
||||
# Post application log entries (batch)
|
||||
curl -s -X POST http://localhost:8081/api/v1/data/logs \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
-d '{
|
||||
"entries": [{
|
||||
"timestamp": "2026-03-25T10:00:00Z",
|
||||
"level": "INFO",
|
||||
"loggerName": "com.acme.MyService",
|
||||
"message": "Processing order #12345",
|
||||
"threadName": "main"
|
||||
}]
|
||||
}'
|
||||
```
|
||||
|
||||
**Note:** The `X-Protocol-Version: 1` header is required on all `/api/v1/data/**` endpoints. Missing or wrong version returns 400.
|
||||
@@ -361,6 +375,8 @@ Key settings in `cameleer3-server-app/src/main/resources/application.yml`:
|
||||
| `security.oidc.client-secret` | | OAuth2 client secret (`CAMELEER_OIDC_CLIENT_SECRET`) |
|
||||
| `security.oidc.roles-claim` | `realm_access.roles` | JSONPath to roles in OIDC id_token (`CAMELEER_OIDC_ROLES_CLAIM`) |
|
||||
| `security.oidc.default-roles` | `VIEWER` | Default roles for new OIDC users (`CAMELEER_OIDC_DEFAULT_ROLES`) |
|
||||
| `opensearch.log-index-prefix` | `logs-` | OpenSearch index prefix for application logs (`CAMELEER_LOG_INDEX_PREFIX`) |
|
||||
| `opensearch.log-retention-days` | `7` | Days before log indices are deleted (`CAMELEER_LOG_RETENTION_DAYS`) |
|
||||
|
||||
## Web UI Development
|
||||
|
||||
|
||||
Reference in New Issue
Block a user