d40833b96a
- LogQueryController: note response shape, sort param, and that the cursor tiebreak is the insert_id UUID column (not exchange/instance) - AgentEventsController: cursor now carries insert_id UUID (was instanceId); order is (timestamp DESC, insert_id DESC) - core-classes: add AgentEventPage record; note that the non-paginated AgentEventRepository.query(...) path has been removed - core-classes: note LogSearchRequest.sources/levels are now List<String> with multi-value OR semantics Keeps the rule files in sync with the cursor-pagination + multi-select filter work on main. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
161 lines
13 KiB
Markdown
161 lines
13 KiB
Markdown
---
|
|
paths:
|
|
- "cameleer-server-app/**"
|
|
---
|
|
|
|
# App Module Key Classes
|
|
|
|
`cameleer-server-app/src/main/java/com/cameleer/server/app/`
|
|
|
|
## URL taxonomy
|
|
|
|
User-facing data and config endpoints live under `/api/v1/environments/{envSlug}/...`. Env is a path segment, never a query param. The `envSlug` is resolved to an `Environment` bean via the `@EnvPath` argument resolver (`web/EnvironmentPathResolver.java`) — 404 on unknown slug.
|
|
|
|
**Slugs are immutable after creation** for both environments and apps. Slug regex: `^[a-z0-9][a-z0-9-]{0,63}$`. Validated in `EnvironmentService.create` and `AppService.createApp`. Update endpoints (`PUT`) do not accept a slug field; Jackson drops it as an unknown property.
|
|
|
|
### Flat-endpoint allow-list
|
|
|
|
These paths intentionally stay flat (no `/environments/{envSlug}` prefix). Every new endpoint should be env-scoped unless it appears here and the reason is documented.
|
|
|
|
| Path prefix | Why flat |
|
|
|---|---|
|
|
| `/api/v1/data/**` | Agent ingestion. JWT `env` claim is authoritative; URL-embedded env would invite spoofing. |
|
|
| `/api/v1/agents/register`, `/refresh`, `/{id}/heartbeat`, `/{id}/events` (SSE), `/{id}/deregister`, `/{id}/commands`, `/{id}/commands/{id}/ack`, `/{id}/replay` | Agent self-service; JWT-bound. |
|
|
| `/api/v1/agents/commands`, `/api/v1/agents/groups/{group}/commands` | Operator fan-out; target scope is explicit in query params. |
|
|
| `/api/v1/agents/config` | Agent-authoritative config read; JWT → registry → (app, env). |
|
|
| `/api/v1/admin/{users,roles,groups,oidc,license,audit,rbac/stats,claim-mappings,thresholds,sensitive-keys,usage,clickhouse,database,environments}` | Truly cross-env admin. Env CRUD URLs use `{envSlug}`, not UUID. |
|
|
| `/api/v1/catalog`, `/api/v1/catalog/{applicationId}` | Cross-env discovery is the purpose. Env is an optional filter via `?environment=`. |
|
|
| `/api/v1/executions/{execId}`, `/processors/**` | Exchange IDs are globally unique; permalinks. |
|
|
| `/api/v1/diagrams/{contentHash}/render`, `POST /api/v1/diagrams/render` | Content-addressed or stateless. |
|
|
| `/api/v1/auth/**` | Pre-auth; no env context exists. |
|
|
| `/api/v1/health`, `/prometheus`, `/api-docs/**`, `/swagger-ui/**` | Server metadata. |
|
|
|
|
## Tenant isolation invariant
|
|
|
|
ClickHouse is shared across tenants. Every ClickHouse query must filter by `tenant_id` (from `CAMELEER_SERVER_TENANT_ID` env var, resolved via `TenantContext`/config) in addition to `environment`. New controllers added under `/environments/{envSlug}/...` must preserve this — the env filter from the path does not replace the tenant filter.
|
|
|
|
## controller/ — REST endpoints
|
|
|
|
### Env-scoped (user-facing data & config)
|
|
|
|
- `AppController` — `/api/v1/environments/{envSlug}/apps`. GET list / POST create / GET `{appSlug}` / DELETE `{appSlug}` / GET `{appSlug}/versions` / POST `{appSlug}/versions` (JAR upload) / PUT `{appSlug}/container-config`. App slug uniqueness is per-env (`(env, app_slug)` is the natural key). `CreateAppRequest` body has no env (path), validates slug regex.
|
|
- `DeploymentController` — `/api/v1/environments/{envSlug}/apps/{appSlug}/deployments`. GET list / POST create (body `{ appVersionId }`) / POST `{id}/stop` / POST `{id}/promote` (body `{ targetEnvironment: slug }` — target app slug must exist in target env) / GET `{id}/logs`.
|
|
- `ApplicationConfigController` — `/api/v1/environments/{envSlug}`. GET `/config` (list), GET/PUT `/apps/{appSlug}/config`, GET `/apps/{appSlug}/processor-routes`, POST `/apps/{appSlug}/config/test-expression`. PUT also pushes `CONFIG_UPDATE` to LIVE agents in this env.
|
|
- `AppSettingsController` — `/api/v1/environments/{envSlug}`. GET `/app-settings` (list), GET/PUT/DELETE `/apps/{appSlug}/settings`. ADMIN/OPERATOR only.
|
|
- `SearchController` — `/api/v1/environments/{envSlug}`. GET `/executions`, POST `/executions/search`, GET `/stats`, `/stats/timeseries`, `/stats/timeseries/by-app`, `/stats/timeseries/by-route`, `/stats/punchcard`, `/attributes/keys`, `/errors/top`.
|
|
- `LogQueryController` — GET `/api/v1/environments/{envSlug}/logs` (filters: source (multi, comma-split, OR-joined), level (multi, comma-split, OR-joined), application, agentId, exchangeId, logger, q, time range; sort asc/desc). Cursor-paginated, returns `{ data, nextCursor, hasMore, levelCounts }`; cursor is base64url of `"{timestampIso}|{insert_id_uuid}"` — same-millisecond tiebreak via the `insert_id` UUID column on `logs`.
|
|
- `RouteCatalogController` — GET `/api/v1/environments/{envSlug}/routes` (merged route catalog from registry + ClickHouse; env filter unconditional).
|
|
- `RouteMetricsController` — GET `/api/v1/environments/{envSlug}/routes/metrics`, GET `/api/v1/environments/{envSlug}/routes/metrics/processors`.
|
|
- `AgentListController` — GET `/api/v1/environments/{envSlug}/agents` (registered agents with runtime metrics, filtered to env).
|
|
- `AgentEventsController` — GET `/api/v1/environments/{envSlug}/agents/events` (lifecycle events; cursor-paginated, returns `{ data, nextCursor, hasMore }`; order `(timestamp DESC, insert_id DESC)`; cursor is base64url of `"{timestampIso}|{insert_id_uuid}"` — `insert_id` is a stable UUID column used as a same-millisecond tiebreak).
|
|
- `AgentMetricsController` — GET `/api/v1/environments/{envSlug}/agents/{agentId}/metrics` (JVM/Camel metrics). Rejects cross-env agents (404) as defence-in-depth.
|
|
- `DiagramRenderController` — GET `/api/v1/environments/{envSlug}/apps/{appSlug}/routes/{routeId}/diagram` (env-scoped lookup). Also GET `/api/v1/diagrams/{contentHash}/render` (flat — content hashes are globally unique).
|
|
|
|
### Env admin (env-slug-parameterized, not env-scoped data)
|
|
|
|
- `EnvironmentAdminController` — `/api/v1/admin/environments`. GET list / POST create / GET `{envSlug}` / PUT `{envSlug}` / DELETE `{envSlug}` / PUT `{envSlug}/default-container-config` / PUT `{envSlug}/jar-retention`. Slug immutable — PUT body has no slug field; any slug supplied is dropped by Jackson. Slug validated on POST.
|
|
|
|
### Agent-only (JWT-authoritative, intentionally flat)
|
|
|
|
- `AgentRegistrationController` — POST `/register` (requires `environmentId` in body; 400 if missing), POST `/{id}/refresh` (rejects tokens with no `env` claim), POST `/{id}/heartbeat` (env from body preferred, JWT fallback; 400 if neither), POST `/{id}/deregister`.
|
|
- `AgentSseController` — GET `/{id}/events` (SSE connection).
|
|
- `AgentCommandController` — POST `/{agentId}/commands`, POST `/groups/{group}/commands`, POST `/commands` (broadcast), POST `/{agentId}/commands/{commandId}/ack`, POST `/{agentId}/replay`.
|
|
- `AgentConfigController` — GET `/api/v1/agents/config`. Agent-authoritative config read: resolves (app, env) from JWT subject → registry (registry miss falls back to JWT env claim; no registry entry → 404 since application can't be derived).
|
|
|
|
### Ingestion (agent-only, JWT-authoritative)
|
|
|
|
- `LogIngestionController` — POST `/api/v1/data/logs` (accepts `List<LogEntry>`; WARNs on missing identity, unregistered agents, empty payloads, buffer-full drops).
|
|
- `EventIngestionController` — POST `/api/v1/data/events`.
|
|
- `ChunkIngestionController` — POST `/api/v1/ingestion/chunk/{executions|metrics|diagrams}`.
|
|
- `ExecutionController` — POST `/api/v1/data/executions` (legacy ingestion path when ClickHouse disabled).
|
|
- `MetricsController` — POST `/api/v1/data/metrics`.
|
|
- `DiagramController` — POST `/api/v1/data/diagrams` (resolves applicationId + environment from the agent registry keyed on JWT subject; stamps both on the stored `TaggedDiagram`).
|
|
|
|
### Cross-env discovery (flat)
|
|
|
|
- `CatalogController` — GET `/api/v1/catalog` (merges managed apps + in-memory agents + CH stats; optional `?environment=` filter). DELETE `/api/v1/catalog/{applicationId}` (ADMIN: dismiss app, purge all CH data + PG record).
|
|
|
|
### Admin (cross-env, flat)
|
|
|
|
- `UserAdminController` — CRUD `/api/v1/admin/users`, POST `/{id}/roles`, POST `/{id}/set-password`.
|
|
- `RoleAdminController` — CRUD `/api/v1/admin/roles`.
|
|
- `GroupAdminController` — CRUD `/api/v1/admin/groups`.
|
|
- `OidcConfigAdminController` — GET/POST `/api/v1/admin/oidc`, POST `/test`.
|
|
- `SensitiveKeysAdminController` — GET/PUT `/api/v1/admin/sensitive-keys`. GET returns 200 or 204 if not configured. PUT accepts `{ keys: [...] }` with optional `?pushToAgents=true`. Fan-out iterates every distinct `(application, environment)` slice — intentional global baseline + per-env overrides.
|
|
- `ClaimMappingAdminController` — CRUD `/api/v1/admin/claim-mappings`, POST `/test`.
|
|
- `LicenseAdminController` — GET/POST `/api/v1/admin/license`.
|
|
- `ThresholdAdminController` — CRUD `/api/v1/admin/thresholds`.
|
|
- `AuditLogController` — GET `/api/v1/admin/audit`.
|
|
- `RbacStatsController` — GET `/api/v1/admin/rbac/stats`.
|
|
- `UsageAnalyticsController` — GET `/api/v1/admin/usage` (ClickHouse `usage_events`).
|
|
- `ClickHouseAdminController` — GET `/api/v1/admin/clickhouse/**` (conditional on `infrastructureendpoints` flag).
|
|
- `DatabaseAdminController` — GET `/api/v1/admin/database/**` (conditional on `infrastructureendpoints` flag).
|
|
|
|
### Other (flat)
|
|
|
|
- `DetailController` — GET `/api/v1/executions/{executionId}` + processor snapshot endpoints.
|
|
- `MetricsController` — exposes `/api/v1/metrics` and `/api/v1/prometheus` (server-side Prometheus scrape endpoint).
|
|
|
|
## runtime/ — Docker orchestration
|
|
|
|
- `DockerRuntimeOrchestrator` — implements RuntimeOrchestrator; Docker Java client (zerodep transport), container lifecycle
|
|
- `DeploymentExecutor` — @Async staged deploy: PRE_FLIGHT -> PULL_IMAGE -> CREATE_NETWORK -> START_REPLICAS -> HEALTH_CHECK -> SWAP_TRAFFIC -> COMPLETE. Container names are `{tenantId}-{envSlug}-{appSlug}-{replicaIndex}` (globally unique on Docker daemon). Sets per-replica `CAMELEER_AGENT_INSTANCEID` env var to `{envSlug}-{appSlug}-{replicaIndex}`.
|
|
- `DockerNetworkManager` — ensures bridge networks (cameleer-traefik, cameleer-env-{slug}), connects containers
|
|
- `DockerEventMonitor` — persistent Docker event stream listener (die, oom, start, stop), updates deployment status
|
|
- `TraefikLabelBuilder` — generates Traefik Docker labels for path-based or subdomain routing. Also emits `cameleer.replica` and `cameleer.instance-id` labels per container for labels-first identity.
|
|
- `PrometheusLabelBuilder` — generates Prometheus Docker labels (`prometheus.scrape/path/port`) per runtime type for `docker_sd_configs` auto-discovery
|
|
- `ContainerLogForwarder` — streams Docker container stdout/stderr to ClickHouse with `source='container'`. One follow-stream thread per container, batches lines every 2s/50 lines via `ClickHouseLogStore.insertBufferedBatch()`. 60-second max capture timeout.
|
|
- `DisabledRuntimeOrchestrator` — no-op when runtime not enabled
|
|
|
|
## metrics/ — Prometheus observability
|
|
|
|
- `ServerMetrics` — centralized business metrics: gauges (agents by state, SSE connections, buffer depths), counters (ingestion drops, agent transitions, deployment outcomes, auth failures), timers (flush duration, deployment duration). Exposed via `/api/v1/prometheus`.
|
|
|
|
## storage/ — PostgreSQL repositories (JdbcTemplate)
|
|
|
|
- `PostgresAppRepository`, `PostgresAppVersionRepository`, `PostgresEnvironmentRepository`
|
|
- `PostgresDeploymentRepository` — includes JSONB replica_states, deploy_stage, findByContainerId
|
|
- `PostgresUserRepository`, `PostgresRoleRepository`, `PostgresGroupRepository`
|
|
- `PostgresAuditRepository`, `PostgresOidcConfigRepository`, `PostgresClaimMappingRepository`, `PostgresSensitiveKeysRepository`
|
|
- `PostgresAppSettingsRepository`, `PostgresApplicationConfigRepository`, `PostgresThresholdRepository`. Both `app_settings` and `application_config` are env-scoped (PK `(app_id, environment)` / `(application, environment)`); finders take `(app, env)` — no env-agnostic variants.
|
|
|
|
## storage/ — ClickHouse stores
|
|
|
|
- `ClickHouseExecutionStore`, `ClickHouseMetricsStore`, `ClickHouseMetricsQueryStore`
|
|
- `ClickHouseStatsStore` — pre-aggregated stats, punchcard
|
|
- `ClickHouseDiagramStore`, `ClickHouseAgentEventRepository`
|
|
- `ClickHouseUsageTracker` — usage_events for billing
|
|
- `ClickHouseRouteCatalogStore` — persistent route catalog with first_seen cache, warm-loaded on startup
|
|
|
|
## search/ — ClickHouse search and log stores
|
|
|
|
- `ClickHouseLogStore` — log storage and query, MDC-based exchange/processor correlation
|
|
- `ClickHouseSearchIndex` — full-text search
|
|
|
|
## security/ — Spring Security
|
|
|
|
- `SecurityConfig` — WebSecurityFilterChain, JWT filter, CORS, OIDC conditional
|
|
- `JwtAuthenticationFilter` — OncePerRequestFilter, validates Bearer tokens
|
|
- `JwtServiceImpl` — HMAC-SHA256 JWT (Nimbus JOSE)
|
|
- `OidcAuthController` — /api/v1/auth/oidc (login-uri, token-exchange, logout)
|
|
- `OidcTokenExchanger` — code -> tokens, role extraction from access_token then id_token
|
|
- `OidcProviderHelper` — OIDC discovery, JWK source cache
|
|
|
|
## agent/ — Agent lifecycle
|
|
|
|
- `SseConnectionManager` — manages per-agent SSE connections, delivers commands
|
|
- `AgentLifecycleMonitor` — @Scheduled 10s, LIVE->STALE->DEAD transitions
|
|
- `SsePayloadSigner` — Ed25519 signs SSE payloads for agent verification
|
|
|
|
## retention/ — JAR cleanup
|
|
|
|
- `JarRetentionJob` — @Scheduled 03:00 daily, per-environment retention, skips deployed versions
|
|
|
|
## config/ — Spring beans
|
|
|
|
- `RuntimeOrchestratorAutoConfig` — conditional Docker/Disabled orchestrator + NetworkManager + EventMonitor
|
|
- `RuntimeBeanConfig` — DeploymentExecutor, AppService, EnvironmentService
|
|
- `SecurityBeanConfig` — JwtService, Ed25519, BootstrapTokenValidator
|
|
- `StorageBeanConfig` — all repositories
|
|
- `ClickHouseConfig` — ClickHouse JdbcTemplate, schema initializer
|