Documents the three Flyway migrations added by the alerting feature branch
so future sessions have an accurate migration map.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rule creation now goes through POST /alerts/rules (exercises saveTargets on the
write path). Clock is replaced with @MockBean(name="alertingClock") and re-stubbed
in @BeforeEach to survive Mockito's inter-test reset. Six ordered steps:
1. seed log → tick evaluator → assert FIRING instance with non-empty targets (B-1)
2. tick dispatcher → assert DELIVERED notification + lastNotifiedAt stamped (B-2)
3. ack via REST → assert ACKNOWLEDGED state
4. create silence → inject PENDING notification → tick dispatcher → assert silenced (FAILED)
5. delete rule → assert rule_id nullified, rule_snapshot preserved (ON DELETE SET NULL)
6. new rule with reNotifyMinutes=1 → first dispatch → advance clock 61s →
evaluator sweep → second dispatch → verify 2 WireMock POSTs (B-2 cadence)
Background scheduler races addressed by resetting claimed_by/claimed_until before
each manual tick. Simulated clock set AFTER log insert to guarantee log timestamp
falls within the evaluator window. Re-notify notifications backdated in Postgres
to work around the simulated vs real clock gap in claimDueNotifications.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
newInstance() now maps rule.targets() into targetUserIds/targetGroupIds/targetRoleNames
so newly created AlertInstance rows carry the correct target arrays.
Previously these were always empty List.of(), making the inbox query return nothing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
V13 migration creates alert_instances_open_rule_uq — a partial unique index on
(rule_id) WHERE state IN ('PENDING','FIRING','ACKNOWLEDGED'), preventing
duplicate open instances per rule. PostgresAlertInstanceRepository.save() catches
DuplicateKeyException and returns the existing open instance instead of failing.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AlertNotificationRepository gains resetForRetry(UUID, Instant) which sets
attempts=0, status=PENDING, next_attempt_at=now, and clears claim/response
fields. AlertNotificationController calls resetForRetry instead of
scheduleRetry so a manual retry always starts from a clean slate.
AlertNotificationControllerIT adds retryResetsAttemptsToZero to verify
attempts==0 and status==PENDING after three prior markFailed calls.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AlertInstanceRepository gains listFiringDueForReNotify(Instant) — only returns
instances where last_notified_at IS NOT NULL and cadence has elapsed (IS NULL
branch excluded: sweep only re-notifies, initial notify is the dispatcher's job).
AlertEvaluatorJob.sweepReNotify() runs at the end of each tick, enqueues fresh
notifications for eligible instances and stamps last_notified_at.
NotificationDispatchJob stamps last_notified_at on the alert_instance when a
notification is DELIVERED, providing the anchor timestamp for cadence checks.
PostgresAlertInstanceRepositoryIT adds listFiringDueForReNotify test covering
the three-rule eligibility matrix (never-notified, long-ago, recent).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
saveTargets() is called unconditionally at the end of save() — it deletes
existing targets and re-inserts from the current targets list. findById()
and listByEnvironment() already call withTargets() so reads are consistent.
PostgresAlertRuleRepositoryIT adds saveTargets_roundtrip and
saveTargets_updateReplacesExistingTargets to cover the new write path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Investigated three approaches for CH 24.12:
- Inline SETTINGS on ADD PROJECTION: rejected (UNKNOWN_SETTING — not a query-level setting).
- ALTER TABLE MODIFY SETTING deduplicate_merge_projection_mode='rebuild': works; persists in
table metadata across connection restarts; runs before ADD PROJECTION in the SQL script.
- Session-level JDBC URL param: not pursued (MODIFY SETTING is strictly better).
alerting_projections.sql now runs MODIFY SETTING before the two executions ADD PROJECTIONs.
AlertingProjectionsIT strengthened to assert all four projections (including alerting_app_status
and alerting_route_status on executions) exist after schema init.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The evaluator mapped P95_LATENCY_MS to ExecutionStats.avgDurationMs because
stats_1m_route has no p95 column. Exposing the old name implied p95 semantics
operators did not get. Rename to AVG_DURATION_MS makes the contract honest.
Updated RouteMetric enum (with javadoc), evaluator switch, and admin guide.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
AbstractPostgresIT gained clickHouseSearchIndex and agentRegistryService mocks in Phase 9.
All 14 alerting IT subclasses that re-declared the same @MockBean fields now fail with
"Duplicate mock definition". Removed the redundant declarations; per-class clickHouseLogStore
mock kept where needed. 120 alerting tests now pass (0 failures).
Also adds docs/alerting-02-verification.md (Task 43).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds alerting stanza to application.yml with all AlertingProperties
fields backed by env-var overrides. Creates docs/alerting.md covering
six condition kinds (with example JSON), template variables, webhook
setup (Slack/PagerDuty examples), silence patterns, circuit-breaker
and retention troubleshooting, and Prometheus metrics reference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Nightly @Scheduled(03:00) job deletes RESOLVED alert_instances older
than eventRetentionDays and DELIVERED/FAILED alert_notifications older
than notificationRetentionDays. Uses injected Clock for testability.
IT covers: old-resolved deleted, fresh-resolved kept, FIRING kept
regardless of age, PENDING notification never deleted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /environments/{envSlug}/alerts/{alertId}/notifications — list notifications for instance (VIEWER+)
- POST /alerts/notifications/{id}/retry — manual retry of failed notification (OPERATOR+)
Flat path because notification IDs are globally unique (no env routing needed)
- scheduleRetry resets attempts to 0 and sets nextAttemptAt = now
- Added 11 alerting path matchers to SecurityConfig before outbound-connections block
- Fixed context loading failure in 6 pre-existing alerting storage/migration ITs by adding
@MockBean(clickHouseSearchIndex/clickHouseLogStore): ExchangeMatchEvaluator and
LogPatternEvaluator inject the concrete classes directly (not interface beans), so the
full Spring context fails without these mocks in tests that don't use the real CH container
- 5 IT tests: list, viewer-can-list, retry, viewer-cannot-retry, unknown-404
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- POST/GET/DELETE /environments/{envSlug}/alerts/silences
- 422 when endsAt <= startsAt ("endsAt must be after startsAt")
- OPERATOR+ for create/delete, VIEWER+ for list
- Audit: ALERT_SILENCE_CREATE/DELETE with AuditCategory.ALERT_SILENCE_CHANGE
- 6 IT tests: create, viewer-list, viewer-cannot-create, bad time-range, delete, audit event
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- GET /environments/{envSlug}/alerts — inbox filtered by userId/groupIds/roleNames via InAppInboxQuery
- GET /unread-count — memoized unread count (5s TTL)
- GET /{id}, POST /{id}/ack, POST /{id}/read, POST /bulk-read
- bulkRead filters instanceIds to env before delegating to AlertReadRepository
- VIEWER+ for all endpoints; env isolation enforced by requireInstance
- 7 IT tests: list, env isolation, unread-count, ack flow, read, bulk-read, viewer access
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- POST/GET/PUT/DELETE /environments/{envSlug}/alerts/rules CRUD
- POST /{id}/enable, /{id}/disable, /{id}/render-preview, /{id}/test-evaluate
- Attribute-key validation: rejects keys not matching ^[a-zA-Z0-9._-]+$ at rule-save time
(CRITICAL: ExchangeMatchCondition attribute keys are inlined into ClickHouse SQL)
- Webhook validation: verifies outboundConnectionId exists and is allowed in env
- Null-safe notification template defaults to "" for NOT NULL DB constraint
- Fixed misleading comment in ClickHouseSearchIndex to document validation contract
- OPERATOR+ for mutations, VIEWER+ for reads
- Audit: ALERT_RULE_CREATE/UPDATE/DELETE/ENABLE/DISABLE with AuditCategory.ALERT_RULE_CHANGE
- 11 IT tests covering RBAC, SQL-injection prevention, enable/disable, audit, render-preview
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
listInbox resolves user groups+roles via RbacService.getEffectiveGroupsForUser
/ getEffectiveRolesForUser then delegates to AlertInstanceRepository.
countUnread memoized per (envId, userId) with 5s TTL via ConcurrentHashMap
using a controllable Clock. 6 unit tests covering delegation, cache hit,
TTL expiry, and isolation between users/envs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Renders URL/headers/body with Mustache, optionally HMAC-signs the body
(X-Cameleer-Signature), supports POST/PUT/PATCH, classifies 2xx/4xx/5xx
into DELIVERED/FAILED/retry. 8 WireMock-backed IT tests including HTTPS
TRUST_ALL against WireMock self-signed cert.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
HmacSHA256 signer returning sha256=<lowercase-hex>. 5 unit tests covering
known vector, prefix, hex casing, and different secrets/bodies.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add withRuleSnapshot(Map) wither to AlertInstance (same pattern as other withers)
- Call snapshotRule(rule) + withRuleSnapshot in both applyResult (single-firing) and
applyBatchFiring paths so every persisted instance carries a non-empty JSONB snapshot
- Strip null values from the Jackson-serialized map before wrapping in the immutable
snapshot so Map.copyOf in the compact ctor does not throw NPE on nullable rule fields
- Add ruleSnapshotIsPersistedOnInstanceCreation IT: asserts name/severity/conditionKind
appear in the rule_snapshot column after a tick fires an instance
- Add historySurvivesRuleDelete IT: fires an instance, deletes the rule, asserts
rule_id IS NULL and rule_snapshot still contains the rule name (spec §5 guarantee)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- AlertEvaluatorJob implements SchedulingConfigurer; fixed-delay tick from
AlertingProperties.effectiveEvaluatorTickIntervalMs (5 s floor)
- Claim-polling via AlertRuleRepository.claimDueRules (FOR UPDATE SKIP LOCKED)
- Per-kind circuit breaker guards each evaluator; failures recorded, open kinds
skipped and rescheduled without evaluation
- Single-Firing path delegates to AlertStateTransitions; new FIRING instances
enqueue AlertNotification rows per rule.webhooks()
- Batch (PER_EXCHANGE) path creates one FIRING AlertInstance per Firing entry
- PENDING→FIRING promotion handled in applyResult via state machine
- Title/message rendered via MustacheRenderer + NotificationContextBuilder;
environment resolved from EnvironmentRepository.findById per tick
- AlertEvaluatorJobIT (4 tests): uses named @MockBean replacements for
ClickHouseSearchIndex + ClickHouseLogStore; @MockBean AgentRegistryService
drives Clear/Firing/resolve cycle without timing sensitivity
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
PER_EXCHANGE returns EvalResult.Batch(List<Firing>); last Firing carries
_nextCursor (Instant) in its context map for the job to persist as
evalState.lastExchangeTs.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
SilenceMatcherService.matches() evaluates AND semantics across ruleId,
severity, appSlug, routeId, agentId constraints. Null fields are wildcards.
Scope-based constraints (appSlug/routeId/agentId) return false when rule is
null (deleted rule — scope cannot be verified). 17 unit tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sentinel-substitution approach: unresolved {{x.y.z}} tokens are replaced
with a unique NUL-delimited sentinel before Mustache compilation, rendered
as opaque text, then post-replaced with the original {{x.y.z}} literal.
Malformed templates (unclosed {{) are caught and return the raw template.
Never throws. 9 unit tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Declared in cameleer-server-core pom (canonical location for unit-testable
rendering without Spring) and mirrored in cameleer-server-app pom so the
app module compiles standalone without a full reactor install.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds alerting_projections.sql with four projections (alerting_app_status,
alerting_route_status on executions; alerting_app_level on logs;
alerting_instance_metric on agent_metrics). ClickHouseSchemaInitializer now
runs both init.sql and alerting_projections.sql, with ADD PROJECTION and
MATERIALIZE treated as non-fatal — executions (ReplacingMergeTree) requires
deduplicate_merge_projection_mode=rebuild which is unavailable via JDBC pool.
MergeTree projections (logs, agent_metrics) always succeed and are asserted in IT.
Column names confirmed from init.sql: logs uses 'application' (not application_id),
agent_metrics uses 'collected_at' (not timestamp). All column names match the plan.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds AlertMatchSpec record (core) and ClickHouseSearchIndex.countExecutionsForAlerting —
no FINAL, no text subqueries. Filters by tenant, env, app, route, status, time window,
and optional after-cursor. Attributes (JSON string column) use inlined JSONExtractString
key literals since ClickHouse JDBC does not bind ? placeholders inside JSON functions.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds countLogs(LogSearchRequest) — no FINAL, no cursor/sort/limit —
reusing the same WHERE-clause logic as search() for tenant, env, app,
level, q, logger, source, exchangeId, and time-range filters.
Also extends ClickHouseTestHelper with executeInitSqlWithProjections()
and makes the script runner non-fatal for ADD/MATERIALIZE PROJECTION.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements AlertInstanceRepository: save (upsert), findById, findOpenForRule,
listForInbox (3-way OR: user/group/role via && array-overlap + ANY), countUnreadForUser
(LEFT JOIN alert_reads), ack, resolve, markSilenced, deleteResolvedBefore.
Integration test covers all 9 scenarios including inbox fan-out across all
three target types. Also adds @JsonIgnoreProperties(ignoreUnknown=true) to
SilenceMatcher to suppress Jackson serializing isWildcard() as a round-trip field.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the Plan 01 stub that returned [] with a real call through
AlertRuleRepository.findRuleIdsByOutboundConnectionId. Adds AlertingBeanConfig
exposing the AlertRuleRepository bean; widens OutboundBeanConfig constructor
to inject it. Delete and narrow-envs guards now correctly block when rules
reference a connection.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements AlertRuleRepository with JSONB condition/webhooks/eval_state
serialization via ObjectMapper, UPSERT on conflict, JSONB containment
query for findRuleIdsByOutboundConnectionId, and FOR UPDATE SKIP LOCKED
claim-polling for horizontal scale.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Replace hard-coded 'u1' user_id with per-test UUID to prevent PK collision on re-runs
- Add @AfterEach null-safe cleanup for environments and users rows
- Use containsExactlyInAnyOrder for enum assertions to catch misspelled names
- Slug suffix on environment insert avoids slug uniqueness conflicts on re-runs
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- OutboundConnectionRequest compact ctor: avoid NPE if tlsTrustMode is null
(defense-in-depth alongside @NotNull Bean Validation).
- Add operatorCannotTest IT case to lock the ADMIN-only contract on
POST /{id}/test — was previously untested.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Shared Spring test context meant seeded test-admin/test-operator/test-viewer/test-alice
users persisted across IT classes, breaking FlywayMigrationIT's "users is empty" assertion.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds OutboundConnectionsPage (list view with delete), lazy route at
/admin/outbound-connections, and Outbound Connections nav node in the
admin sidebar tree. No test file created — UI codebase has no existing
test infrastructure to build on.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Types are hand-authored (matching codebase admin-query convention);
schema.d.ts regeneration deferred until backend dev server is available.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
POST /{id}/test issues a synthetic probe against the connection URL.
TLS protocol/cipher/peer-cert details stubbed for now (Plan 02 follow-up).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New audit categories: OUTBOUND_CONNECTION_CHANGE, OUTBOUND_HTTP_TRUST_CHANGE.
Controller-level @PreAuthorize defaults to ADMIN; GETs relaxed to ADMIN|OPERATOR.
SecurityConfig permits OPERATOR GETs on /api/v1/admin/outbound-connections/**.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
V11 migration referenced users(id) as uuid, but V1 users table has
user_id as TEXT primary key. Amending V11 and the OutboundConnection
record before Task 7's integration tests catch this at Flyway startup.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds ApacheOutboundHttpClientFactory (Apache HttpClient 5) that memoizes
CloseableHttpClient instances keyed on effective TLS + timeout config, and
OutboundHttpConfig (@ConfigurationProperties) that validates trusted CA paths
at startup and exposes OutboundHttpClientFactory as a Spring bean.
TRUST_ALL mode disables both cert validation (TrustAllManager in SslContextBuilder)
and hostname verification (NoopHostnameVerifier on SSLConnectionSocketFactoryBuilder).
WireMock HTTPS integration test covers trust-all bypass, system-default PKIX rejection,
and client memoization.
OIDC audit: OidcProviderHelper and OidcTokenExchanger use Nimbus SDK's own HTTP layer
(DefaultResourceRetriever for JWKS, HTTPRequest.send() for token exchange) plus the
bespoke InsecureTlsHelper for TLS skip-verify; neither uses OutboundHttpClientFactory.
Retrofit deferred to a separate follow-up per plan §20.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Defense-in-depth per code review. DTO layer already validates HTTPS at save
time; this DB-level check guards against future code paths that might bypass
the DTO validator. Mustache template variables in the URL (e.g., {{env.slug}})
remain valid since only the scheme prefix is constrained.
First of three sequenced plans for the alerting feature. Covers:
- Cross-cutting http/ module (OutboundHttpClientFactory, SslContextBuilder,
TLS trust composition, startup validation)
- Admin-managed OutboundConnection with PG persistence, AES-GCM-encrypted
HMAC secret (resolves spec §20 item 2)
- Admin CRUD REST + test endpoint + RBAC + audit
- Admin UI page with TLS config, allowed-envs multi-select, test action
- OIDC retrofit deliberately deferred (documented in Task 4 audit)
Plan 02 (alerting backend) and Plan 03 (alerting UI) written after Plan 01
executes — lets reality inform their details, especially the secret-cipher
interface and the rules-referencing integration point.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BL-002 / gitea#138 tracks deferred native provider types (Slack Block Kit,
PagerDuty Events v2, Teams connector) with shipped templates as a post-v1
fast-follow once usage data informs which providers matter.
Spec §13 folds in context-aware variable auto-complete for the shared
<MustacheEditor /> component used in rule editor, webhook overrides, and
outbound-connection admin. Available variables filter by condition kind.
Completion engine choice added to §20 as a planning-phase decision.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comprehensive design spec for a confined, env-scoped alerting feature:
6 signal sources, shared env-scoped rules with RBAC-targeted notifications,
in-app inbox + webhook delivery via admin-managed outbound connections,
claim-based polling for horizontal scalability, 4 CH projections for hot-path
reads. Backlog entry BL-001 / gitea#137 tracks deferred managed-CA investigation
(reuse SaaS-layer CA handling first before building in-server storage).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
JSX attribute strings don't process JS escape sequences — "Search logs\u2026"
rendered the literal "\u2026" in the placeholder. Replaced with the actual
ellipsis character.
Also aligned .logSearchInput (Application Log search) with EventFeed's
internal search input: --bg-surface background, --border border,
mono font family, 28px height. Previously used --bg-body + --border-subtle
+ body font, which looked visibly different next to the Timeline panel.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- LogQueryController: note response shape, sort param, and that the
cursor tiebreak is the insert_id UUID column (not exchange/instance)
- AgentEventsController: cursor now carries insert_id UUID (was instanceId);
order is (timestamp DESC, insert_id DESC)
- core-classes: add AgentEventPage record; note that the non-paginated
AgentEventRepository.query(...) path has been removed
- core-classes: note LogSearchRequest.sources/levels are now List<String>
with multi-value OR semantics
Keeps the rule files in sync with the cursor-pagination + multi-select
filter work on main.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Both panels now use the same card wrapper (logStyles.logCard), header
container (logStyles.logHeader, 12px 16px padding), and DS SectionHeader
for the title. Previously Timeline rendered a custom 13px span while
Application Log used SectionHeader's uppercase style, so the two panels
side-by-side looked inconsistent.
Removes the now-orphaned .eventCard/.eventCardHeader/.sectionTitle and
.timelineCard/.timelineHeader CSS rules.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EventFeed has its own search + filter toolbar inside the component.
Wrapping it in InfiniteScrollArea made the toolbar scroll out of
sight. Drop InfiniteScrollArea for the Timeline, give EventFeed a
bounded-height flex container (it scrolls its own .list internally),
and add an explicit 'Load older events' button for cursor
pagination. Polling always on for events (low volume).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Page 1 refetches were using the captured timeRange.end, so rows
arriving after the initial render were outside the query window and
never surfaced. When timeRange.preset is set (e.g. 'last 1h'), each
fetch now advances 'to' to Date.now() so the poll picks up new rows.
Absolute ranges are unchanged.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
EventFeed's overflow-y:auto lives on its inner .list, not the root
where className lands. Extending .flatScroll to .flatScroll * covers
nested scroll containers, and relaxing the root's height:100% (which
EventFeed sets) lets content size naturally so the outer
InfiniteScrollArea owns the single scrollbar.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LogViewer and EventFeed each apply overflow-y:auto to their root
container, which produced a nested scrollbar inside the
InfiniteScrollArea that also scrolls. A flatScroll override class
flattens the DS component so the outer InfiniteScrollArea owns the
single scrollbar — matching the IntersectionObserver sentinels that
drive infinite-load.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same-millisecond rows were silently skipped between pages because the
log cursor had no tiebreak and the events cursor tied by instance_id
(which also collides when one instance emits multiple events within a
millisecond). Add an insert_id UUID (DEFAULT generateUUIDv4()) column
to both logs and agent_events, order by (timestamp, insert_id)
consistently, and encode the cursor as 'timestamp|insert_id'. Existing
data is materialized via ALTER TABLE MATERIALIZE COLUMN (one-time
background mutation).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reversing logStream.items client-side breaks across infinite-scroll
pages. Passing sort='asc'/'desc' into the query key and URL triggers
a fresh first-page fetch in the selected order.
AgentEventService.queryEvents, AgentEventRepository.query, and the
ClickHouse implementation have had no callers since /agents/events
became cursor-paginated. Remove them along with their dedicated IT
tests. queryPage and its tests remain as the single query path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends the existing ApiExceptionHandler @RestControllerAdvice to map
DateTimeParseException and IllegalArgumentException to 400 Bad Request.
Logs and agent-events endpoints both parse ISO-8601 query params and
previously leaked parse failures as internal server errors. All
IllegalArgumentException throw sites in production code are
input-validation usages (slug validation, containerConfig validation,
cursor decoding), so mapping to 400 is correct across the board.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
JDBC Timestamp binding shifted timestamps by the JVM local timezone
offset on both insert and query, producing asymmetric UTC offsets that
broke time-range filtering and cursor pagination. Switching inserts
(indexBatch, insertBufferedBatch) and all WHERE predicates to ISO-8601
strings via parseDateTime64BestEffort, and reading timestamps back as
epoch-millis via toUnixTimestamp64Milli, pins everything to UTC and
fixes the time-range filter test plus cursor pagination.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Reflects LogQueryController's multi-value source/level filters,
AgentEventsController's cursor pagination shape, and the new
useInfiniteStream/InfiniteScrollArea UI primitives used by streaming
views.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Same pattern as AgentHealth, scoped to a single agent instance
(passes agentId to both log and timeline streams).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- useInfiniteStream: wrap fetchNextPage and refresh in useCallback so
InfiniteScrollArea's IntersectionObserver does not re-subscribe on
every parent render.
- InfiniteScrollArea: do not render 'End of stream' until at least one
item has loaded and the initial query has settled (was flashing on
mount before first fetch).
- AgentHealth: pass isLoading + hasItems to both InfiniteScrollArea
wrappers.
Application Log: source + level filters move server-side; text search
stays client-side. Timeline: cursor-paginated via useInfiniteAgentEvents.
Both wrapped in InfiniteScrollArea with top-gated auto-refetch.
Server-side filters on source/level/time-range, client-side text
search on top of flattened items. Leaves useApplicationLogs and
useStartupLogs untouched for bounded consumers (LogTab, StartupLogPanel).
Scrollable container with top/bottom IntersectionObserver sentinels.
Fires onTopVisibilityChange when the top is fully in view and
onEndReached when the bottom is within 100px. Used by infinite log
and event streams.
Captures the cursor-paginated /agents/events response shape
(AgentEventPageResponse with data/nextCursor/hasMore and a new ?cursor
param). Also folds in pre-existing drift from 62dd71b (environment
field on agent event rows). Consumer UI hooks are updated in
Tasks 9-11.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wraps DateTimeParseException from Instant.parse in IllegalArgumentException
so the controller maps it to 400. Also rejects cursors with empty
instance_id (trailing '|') which would otherwise produce a vacuous
keyset predicate.
Orders by (timestamp DESC, instance_id ASC). Cursor is
base64url('timestampIso|instanceId') with a tuple keyset predicate
for stable paging across ties.
Introduces cursor-paginated query on AgentEventRepository. The cursor
format is owned by the implementation. The existing non-paginated
query(...) is kept for internal consumers.
Replaces LogSearchRequest.source (String) with sources (List<String>)
and emits 'source IN (...)' when non-empty. LogQueryController parses
?source=a,b,c the same way it parses ?level=a,b,c.
The agent_events table has an `environment` column and AgentEventsController
filters on it, but the INSERT never populated it — every row got the
column default ('default'). Result: Timeline on the Application Runtime
page was empty whenever the user's selected env was anything other than
'default'.
Thread env through the write path:
- AgentEventRepository.insert + AgentEventService.recordEvent gain an
`environment` param; delete the no-env query overload (unused).
- ClickHouseAgentEventRepository.insert writes the column (falls back to
'default' on null to match column DEFAULT).
- All 5 callers source env from the agent registry (AgentInfo.environmentId)
or the registration request body; AgentLifecycleMonitor, deregister,
command ack, event ingestion, register/re-register.
- Integration test updated for the new signatures.
Pre-existing rows in deployed CH will still report environment='default'.
New events from this build forward will carry the correct env. Backfill
(UPDATE ... FROM apps) is left as a manual DB step if historical timeline
is needed for non-default envs.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LayoutShell's <main> sets overflow:hidden + min-height:0; pages must
handle their own scroll. SwaggerPage's root div had no height constraint,
so the Swagger UI rendered below the viewport with no scrollbar.
Match the pattern used by AppsTab/DashboardTab/etc.: flex:1 + min-height:0
+ overflow-y:auto on the page root. Inline style since this is a leaf
page with no existing module.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The deployed instance at 192.168.50.86:30090 is the canonical source of
truth for the API schema during development — regen against it instead
of requiring a local backend boot with Postgres + ClickHouse.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Fetched from http://192.168.50.86:30090/api/v1/api-docs (running origin/main
through b7a107d — full P3B/P3C env-scoping migration live there).
SPA TS types now match the env-scoped URL shape used at runtime:
- /environments/{envSlug}/... for data, config, search, logs, routes, agents
- /agents/config (agent-authoritative)
- /admin/environments/{envSlug}/... (env CRUD)
Note: ExecutionDetail.environment isn't in the regenerated schema yet —
commit d02fa73 (local, not yet pushed/deployed) adds that backend field.
The local type extension in ui/src/components/ExecutionDiagram/types.ts
covers the gap until the next redeploy + regen.
UI typecheck (tsc -p tsconfig.app.json --noEmit) passes.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Correlated exchanges always share the env of the one being viewed —
using the globally-selected env from the picker was wrong if the user
switched envs after opening a detail view (or arrived via permalink).
Thread `environment` through:
- `ExecutionStore.ExecutionRecord` gains `environment` field; the
ClickHouse `executions` table already stores this, just not read back.
- `ClickHouseExecutionStore.findById` SELECT adds the column; mapper
populates it.
- `ExecutionDetail` gains `environment`; `DetailService` passes through.
- `IngestionService.toExecutionRecord` passes null — this legacy PG
ingestion path isn't active when ClickHouse is enabled, and the
read-side is what drives the correlation UI.
- UI `ExchangeHeader` reads `detail.environment ?? storeEnv` and
extends the TS type locally (schema.d.ts catches up on next regen).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The env-scoping migration (P3A) changed useCorrelationChain to require an
environment arg and gate on `enabled: !!correlationId && !!environment`,
but ExchangeHeader was still calling it with one arg. Result: the query
never fired, so the header always rendered "no correlated exchanges
found" even when 4+ exchanges shared a correlationId.
Fix: read the selected env from the Zustand environment store and pass
it through.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up the URL moves from P2/P3A/P3B/P3C. Also fixes a latent bug in
AppControllerIT.uploadJar_asOperator_returns201 / DeploymentControllerIT
setUp: the tests were passing the app's UUID as the {appSlug} path
variable (via `path("id").asText()`); the old AppController looked up
apps via getBySlug(), so the legacy URL call would 404 when the slug
literal was a UUID. Now the test tracks the known slug string and uses
it for every /apps/{appSlug}/... path.
Test URL updates:
- SearchControllerIT: /api/v1/search/executions →
/api/v1/environments/default/executions (GET) and
/api/v1/environments/default/executions/search (POST).
- AppControllerIT: /api/v1/apps → /api/v1/environments/default/apps.
Request bodies drop environmentId (it's in the path).
- DeploymentControllerIT: /api/v1/apps/{appId}/deployments →
/api/v1/environments/default/apps/{appSlug}/deployments. DeployRequest
body drops environmentId.
- JwtRefreshIT + RegistrationSecurityIT: smoke-test protected endpoint
call updated to the new /environments/default/executions shape.
All tests compile clean. Runtime behavior requires a full stack
(Postgres + ClickHouse + Docker); validating integration tests is a
pre-merge step before merging the feature branch.
Remaining pre-merge items (not blocked by code):
1. Regenerate ui/src/api/schema.d.ts + openapi.json by running
`cd ui && npm run generate-api:live` against a running backend.
SearchController, DeploymentController, etc. DTO signatures have
changed; schema.d.ts is frozen at the pre-migration shape.
Raw-fetch call sites introduced in P3A/P3C work at runtime without
the schema; the regen only sharpens TypeScript coverage.
2. Smoke test locally: boot server, verify EnvironmentsPage,
AppsTab, Exchanges, Dashboard, Runtime pages all function.
3. Run `mvn verify` end-to-end (Testcontainers + Docker required).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Locks the new conventions into rule files so future agents and humans
don't drift back into old patterns.
- .claude/rules/app-classes.md: replaces the flat endpoint catalog
with a taxonomy-aware reorganization (env-scoped / env-admin /
agent-only / ingestion / cross-env discovery / admin / other).
Adds the flat-endpoint allow-list with rationale per prefix and
documents the tenant-filter invariant for ClickHouse queries.
- CLAUDE.md: adds four convention bullets in Key Conventions —
URL taxonomy with allow-list pointer, slug immutability rule,
app uniqueness as (env, app_slug), env-required on env-scoped
endpoints.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P3C — the last data/query wave of the taxonomy migration. Every user-
facing read endpoint that was keyed on env-as-query-param is now under
the env-scoped URL, making env impossible to omit and unambiguous in
server-side tenant+env filtering.
Server:
- SearchController: /api/v1/search/** → /api/v1/environments/{envSlug}/...
Endpoints: /executions (GET), /executions/search (POST), /stats,
/stats/timeseries, /stats/timeseries/by-app, /stats/timeseries/by-route,
/stats/punchcard, /attributes/keys, /errors/top. Env comes from path.
- LogQueryController: /api/v1/logs → /api/v1/environments/{envSlug}/logs.
- RouteCatalogController: /api/v1/routes/catalog → /api/v1/environments/
{envSlug}/routes. Env filter unconditional (path).
- RouteMetricsController: /api/v1/routes/metrics →
/api/v1/environments/{envSlug}/routes/metrics (and /metrics/processors).
- DiagramRenderController: /{contentHash}/render stays flat (hashes are
globally unique). Find-by-route moved to /api/v1/environments/{envSlug}/
apps/{appSlug}/routes/{routeId}/diagram — the old GET /diagrams?...
handler is removed.
- Agent views split cleanly:
- AgentListController (new): /api/v1/environments/{envSlug}/agents
- AgentEventsController: /api/v1/environments/{envSlug}/agents/events
- AgentMetricsController: /api/v1/environments/{envSlug}/agents/
{agentId}/metrics — now also rejects cross-env agents (404) as a
defense-in-depth check, fulfilling B3.
Agent self-service endpoints (register/refresh/heartbeat/deregister)
remain flat at /api/v1/agents/** — JWT-authoritative.
SPA:
- queries/agents.ts, agent-metrics.ts, logs.ts, catalog.ts (route
metrics only; /catalog stays flat), processor-metrics.ts,
executions.ts (attributes/keys, stats, timeseries, search),
dashboard.ts (all stats/errors/punchcard), correlation.ts,
diagrams.ts (by-route) — all rewritten to env-scoped URLs.
- Hooks now either read env from useEnvironmentStore internally or
require it as an argument. Query keys include env so switching env
invalidates caches.
- useAgents/useAgentEvents signature simplified — env is no longer a
parameter; it's read from the store. Callers (LayoutShell,
AgentHealth, AgentInstance) updated accordingly.
- LogTab and useStartupLogs thread env through to useLogs.
- envFetch helper introduced in executions.ts for env-prefixed raw
fetch until schema.d.ts is regenerated against the new backend.
BREAKING CHANGE: All these flat paths are removed:
/api/v1/search/**, /api/v1/logs, /api/v1/routes/catalog,
/api/v1/routes/metrics (and /processors), /api/v1/diagrams
(lookup), /api/v1/agents (list), /api/v1/agents/events-log,
/api/v1/agents/{id}/metrics, /api/v1/agent-events.
Clients must use the /api/v1/environments/{envSlug}/... equivalents.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P3B of the taxonomy migration. App and deployment routes are now
env-scoped in the URL itself, making the (env, app_slug) uniqueness
key explicit. Previously /api/v1/apps/{appSlug} was ambiguous: with
the same app deployed to multiple environments (dev/staging/prod),
the handler called AppService.getBySlug(slug) which returns the
first row matching slug regardless of env.
Server:
- AppController: @RequestMapping("/api/v1/environments/{envSlug}/
apps"). Every handler now calls
appService.getByEnvironmentAndSlug(env.id(), appSlug) — 404 if the
app doesn't exist in *this* env. CreateAppRequest body drops
environmentId (it's in the path).
- DeploymentController: @RequestMapping("/api/v1/environments/
{envSlug}/apps/{appSlug}/deployments"). DeployRequest body drops
environmentId. PromoteRequest body switches from
targetEnvironmentId (UUID) to targetEnvironment (slug);
promote handler resolves the target env by slug and looks up the
app with the same slug in the target env (fails with 404 if the
target app doesn't exist yet — apps must exist in both source
and target before promote).
- AppService: added getByEnvironmentAndSlug helper; createApp now
validates slug against ^[a-z0-9][a-z0-9-]{0,63}$ (400 on
invalid).
SPA:
- queries/admin/apps.ts: rewritten. Hooks take envSlug where
env-scoped. Removed useAllApps (no flat endpoint). Renamed path
param naming: appId → appSlug throughout. Added
usePromoteDeployment. Query keys include envSlug so cache is
env-scoped.
- AppsTab.tsx: call sites updated. When no environment is selected,
the managed-app list is empty — cross-env discovery lives in the
Runtime tab (catalog). handleDeploy/handleStop/etc. pass envSlug
to the new hook signatures.
BREAKING CHANGE: /api/v1/apps/** paths removed. Clients must use
/api/v1/environments/{envSlug}/apps/{appSlug}/**. Request bodies
for POST /apps and POST /apps/{slug}/deployments no longer accept
environmentId (use the URL path instead). Promote body uses slug
not UUID.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
P3A of the taxonomy migration. Env-scoped config and settings endpoints
now live under the env-prefixed URL shape, making env a first-class
path segment instead of a query param. Agent-authoritative config is
split off into a dedicated endpoint so agent env comes from the JWT
only — never spoofable via URL.
Server:
- ApplicationConfigController: @RequestMapping("/api/v1/environments/
{envSlug}"). Handlers use @EnvPath Environment env, appSlug as
@PathVariable. Removed the dual-mode resolveEnvironmentForRead —
user flow only; agent flow moved to AgentConfigController.
- AgentConfigController (new): GET /api/v1/agents/config. Reads
instanceId from JWT subject, resolves (app, env) from registry,
returns AppConfigResponse. Registry miss → falls back to JWT env
claim for environment, but 404s if application cannot be derived
(no other source without registry).
- AppSettingsController: @RequestMapping("/api/v1/environments/
{envSlug}"). List at /app-settings, per-app at /apps/{appSlug}/
settings. Access class-wide PreAuthorize preserved (ADMIN/OPERATOR).
SPA:
- commands.ts: useAllApplicationConfigs, useApplicationConfig,
useUpdateApplicationConfig, useProcessorRouteMapping,
useTestExpression — rewritten URLs to /environments/{env}/apps/
{app}/... shape. environment now required on every call. Query
keys include environment so cache is env-scoped.
- dashboard.ts: useAppSettings, useAllAppSettings, useUpdateAppSettings
rewritten.
- TapConfigModal: new required environment prop; callers updated.
- RouteDetail, ExchangesPage: thread selectedEnv into test-expression
and modal.
Config changes in SecurityConfig for the new shape landed earlier in
P0.2; no security rule changes needed in this commit.
BREAKING CHANGE: /api/v1/config/** and /api/v1/admin/app-settings/**
paths removed. Agents must use /api/v1/agents/config instead of
GET /api/v1/config/{app}; users must use /api/v1/environments/{env}/
apps/{app}/config and /api/v1/environments/{env}/apps/{app}/settings.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
UUID-based admin paths were the only remaining UUID-in-URL pattern in
the API. Migrates /api/v1/admin/environments/{id} to /{envSlug} so
slugs are the single environment identifier in every URL. UUIDs stay
internal to the database.
- Controller: @PathVariable UUID id → @PathVariable String envSlug on
get/update/delete and the two nested endpoints (default-container-
config, jar-retention). Handlers resolve slug → Environment via
EnvironmentService.getBySlug, then delegate to existing UUID-based
service methods.
- Service: create() now validates slug against ^[a-z0-9][a-z0-9-]{0,63}$
and returns 400 on invalid slugs. Rationale documented in the class:
slugs are immutable after creation because they appear in URLs,
Docker network names, container names, and ClickHouse partition keys.
- UpdateEnvironmentRequest has no slug field and Jackson's default
ignore-unknown behavior drops any slug supplied in a PUT body;
regression test (updateEnvironment_withSlugInBody_ignoresSlug)
documents this invariant.
- SPA: mutation args change from { id } to { slug }. EnvironmentsPage
still uses env.id for local selection state (UUID from DB) but
passes env.slug to every mutation.
BREAKING CHANGE: /api/v1/admin/environments/{id:UUID}/... paths removed.
Clients must use /{envSlug}/... (slug from the environments list).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes two cross-env data leakage paths. Both endpoints previously
returned data aggregated across all environments, so a diagram or
attribute key from dev could appear in a prod UI query (and vice versa).
B1: GET /api/v1/diagrams?application=&routeId= now requires
?environment= and resolves agents via
registryService.findByApplicationAndEnvironment instead of
findByApplication. Prevents serving a dev diagram for a prod route.
B2: GET /api/v1/search/attributes/keys now requires ?environment=.
SearchIndex.distinctAttributeKeys gains an environment parameter and
the ClickHouse query adds the env filter alongside the existing
tenant_id filter. Prevents prod attribute names leaking into dev
autocompletion (and vice versa).
SPA hooks updated to thread environment through from
useEnvironmentStore; query keys include environment so React Query
re-fetches on env switch. No call-site changes needed — hook
signatures unchanged.
B3 (AgentMetricsController env scope) deferred to P3C: agent-env is
effectively 1:1 today via the instance_id naming
({envSlug}-{appSlug}-{replicaIndex}), and the URL migration in P3C
to /api/v1/environments/{env}/agents/{agentId}/metrics naturally
introduces env from path. A minimal P1 fix would regress the "view
metrics of a killed agent" case.
BREAKING CHANGE: Both endpoints now require ?environment= (slug).
Clients omitting the parameter receive 400.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Groundwork for the REST API taxonomy migration. Introduces the
infrastructure that future waves use to move data/query endpoints under
/api/v1/environments/{envSlug}/... without per-handler boilerplate.
- Add @EnvPath annotation + EnvironmentPathResolver: injects the
Environment identified by the {envSlug} path variable, 404 on unknown
slug, registered via WebConfig.addArgumentResolvers.
- Add env-scoped URL matchers to SecurityConfig (config, settings,
executions/stats/logs/routes/agents/apps/deployments under
/environments/*/**). Legacy flat matchers kept in place and will be
removed per-wave as controllers migrate. New agent-authoritative
/api/v1/agents/config matcher prepared for the agent/user split.
- Document OpenAPI schema regen workflow in CLAUDE.md so future API
changes cover schema.d.ts regeneration as part of the change.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
BREAKING: wipe dev PostgreSQL before deploying — V1 checksum changes.
Agents must now send environmentId on registration (400 if missing).
Two tables previously keyed on app name alone caused cross-environment
data bleed: writing config for (app=X, env=dev) would overwrite the row
used by (app=X, env=prod) agents, and agent startup fetches ignored env
entirely.
- V1 schema: application_config and app_settings are now PK (app, env).
- Repositories: env-keyed finders/saves; env is the authoritative column,
stamped on the stored JSON so the row agrees with itself.
- ApplicationConfigController.getConfig is dual-mode — AGENT role uses
JWT env claim (agents cannot spoof env); non-agent callers provide env
via ?environment= query param.
- AppSettingsController endpoints now require ?environment=.
- SensitiveKeysAdminController fan-out iterates (app, env) slices so each
env gets its own merged keys.
- DiagramController ingestion stamps env on TaggedDiagram; ClickHouse
route_diagrams INSERT + findProcessorRouteMapping are env-scoped.
- AgentRegistrationController: environmentId is required on register;
removed all "default" fallbacks from register/refresh/heartbeat auto-heal.
- UI hooks (useApplicationConfig, useProcessorRouteMapping, useAppSettings,
useAllAppSettings, useUpdateAppSettings) take env, wired to
useEnvironmentStore at all call sites.
- New ConfigEnvIsolationIT covers env-isolation for both repositories.
Plan in docs/superpowers/plans/2026-04-16-environment-scoping.md.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Picks up the Sidebar fix that aligns Sidebar.Section header and
Sidebar.FooterLink icon/label columns. Bottom-positioned admin section
and footer links (e.g. API Docs) now share the same 9px left offset and
16px icon wrapper width.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The exchange search silently filtered by the in-memory agent registry's
current instance IDs on top of application_id. Historical exchanges written
by previous agent instances (or any instance not currently registered, e.g.
after a server restart before agents heartbeat back) were hidden from
results even though they matched the application filter.
Fix: drop the applicationId -> instanceIds resolution in SearchController.
Rely on application_id = ? in ClickHouseSearchIndex; keep explicit
instanceIds filtering only when a client passes them.
Related cleanup: the agentIds parameter on StatsStore.statsForRoute /
timeseriesForRoute was silently discarded inside ClickHouseStatsStore, so
per-route stats aggregated across any apps sharing a routeId. Replace with
String applicationId and add application_id to the stats_1m_route filters
so per-route stats are correctly scoped.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The design system doesn't define --text-inverse yet, so SVG icons
on colored badge backgrounds (trace, tap, status) had no fill color.
Define it in index.css as #fff until the DS adds it natively.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
IngestionService.hasAnyTraceData() and ChunkAccumulator only checked
for inputBody/outputBody when setting has_trace_data on executions.
Headers and properties captured via deep tracing were not considered,
causing the trace data indicator to be missing in the exchange list.
DetailService already checked all six fields — this aligns the
ingestion path to match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Merge the route state update and route catalog upsert blocks to share
one registryService.findById() call instead of two, reducing overhead
on the high-frequency heartbeat path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add RouteCatalogStore as a third data source in RouteCatalogController so that
/api/v1/routes/catalog surfaces routes with zero executions and routes from
previous app versions that fall within the requested time window.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wires RouteCatalogStore into CatalogController as a third data source:
routes with zero executions and routes from previous app versions
(within the queried time window) now appear in the unified catalog.
Also clears route_catalog on app dismiss via deleteByApplication().
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Wire RouteCatalogStore into AgentRegistrationController and call upsert
after registration and heartbeat so routes survive server restarts.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Routes with zero executions (sub-routes) vanish from the sidebar after
server restart because the catalog is purely in-memory with a ClickHouse
stats fallback that only covers executed routes. This spec describes a
persistent route_catalog table in ClickHouse with lifecycle tracking
(first_seen/last_seen) to reconstruct the sidebar without agent
reconnection and support historical time-window queries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use Activity, Cpu, and HeartPulse icons instead of "tps", "cpu", and
"ago" text in compact and expanded app cards. Bump design-system to
v0.1.55 for sidebar footer alignment fix.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add search icon, translucent background, and same padding/sizing
as the sidebar's built-in filter input. Placeholder changed to
"Filter..." to match sidebar convention.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Text input next to view toggle filters apps by name (case-insensitive
substring match). KPI stat strip uses unfiltered counts so totals
stay accurate. Clear button on non-empty input.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move toolbar above the grid conditional so it renders in both
view modes. Hidden only on app detail pages (isFullWidth).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Show per-instance CPU usage percentage instead of error rate in the
DataTable. Highlights >80% CPU in error color.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The toggle only renders inside the compact branch, so viewMode is
always 'compact' there. Use static class assignment instead of a
comparison TypeScript correctly flags as unreachable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add max CPU percentage to the meta row of both the full expanded
view and the overlay expanded card, consistent with compact cards.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backend:
- Add cpuUsage field to AgentInstanceResponse (-1 if unavailable)
- Add queryAgentCpuUsage() to AgentRegistrationController — queries
avg CPU per instance from agent_metrics over last 2 minutes
- Wire CPU into agent list response via withCpuUsage()
Frontend:
- Add cpuUsage to schema.d.ts
- Compute maxCpu per AppGroup (max across all instances)
- Show "X% cpu" on compact cards when available (hidden when -1)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add invisible backdrop (z-index 99) behind expanded overlay to
dismiss on outside click
- Remove background/padding from overlay wrapper so GroupCard
renders without visible extra border
- Use drop-shadow filter instead of box-shadow for natural card
shadow
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move view toggle into compact grid conditional so it only renders
on the overview page (not app detail /runtime/{slug})
- Left-align the toolbar buttons
- Change TPS format from "x.y/s" to "x.y tps"
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Bump overlay z-index to 100 so it renders above the sidebar
- App name in compact card navigates to /runtime/{slug} on click
- Add TPS (msg/s) as third metric on compact cards between live
count and heartbeat
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Move expand/collapse toggle from stat strip to dedicated toolbar
below KPIs
- Sort app groups alphabetically by name
- Expanded card overlays from clicked card position instead of
pushing other cards down
- Viewport constraint: overlay flips right-alignment and limits
height when near edges
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two sidebar bugs fixed:
1. Route entries never highlighted on navigation because sidebar-utils
generated /apps/ paths for route children while effectiveSelectedPath
normalizes to /exchanges/. The design system does exact string matching.
2. Routes disappeared from sidebar when agents had no recent exchange
data. Heartbeat carried routeStates (with route IDs as keys) but
AgentRegistryService.heartbeat() never updated AgentInfo.routeIds.
After server restart, auto-heal registered agents with empty routes,
leaving ClickHouse (24h window) as the only discovery source.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Un-ignore .claude/rules/ so path-scoped rule files are shared via git.
Add instruction in CLAUDE.md to update rule files when modifying classes,
controllers, endpoints, or metrics — keeps rules current as part of
normal workflow rather than requiring separate maintenance.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The agent shaded JAR now includes the log appender classes. Remove
PropertiesLauncher, -Dloader.path, and separate appender JAR references.
All JVM types now use: java -javaagent:/app/agent.jar -jar app.jar
Plain Java uses -cp with explicit main class. Native runs binary directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LogTab now checks mdc['cameleer.processorId'] first when filtering logs
for a selected processor node, falling back to fuzzy message/loggerName
matching for older agents without the new MDC key.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RuntimeDetector now derives the correct PropertiesLauncher FQN from
the JAR manifest Main-Class package. Spring Boot 3.2+ uses
org.springframework.boot.loader.launch.PropertiesLauncher, pre-3.2
uses org.springframework.boot.loader.PropertiesLauncher.
DockerRuntimeOrchestrator uses the detected class instead of a
hardcoded 3.2+ reference, falling back to 3.2+ when not auto-detected.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add cameleer-clickhouse-external Service (NodePort 30123) matching the
pattern used by cameleer-postgres-external (NodePort 30432).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The agent now sets cameleer.exchangeId in MDC (persists across processor
executions, unlike Camel's camel.exchangeId which is scoped to MDCUnitOfWork).
For ON_COMPLETION exchange copies, the agent uses the parent's exchange ID.
Server changes:
- ClickHouseLogStore ingestion: extract exchange_id preferring
cameleer.exchangeId, falling back to camel.exchangeId
- ClickHouseLogStore search: match exchangeId filter against exchange_id
column OR cameleer.exchangeId OR camel.exchangeId in MDC
- Update CLAUDE.md with log exchange correlation documentation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Applications section: maxHeight 50vh with scroll overflow
- Starred section: maxHeight 30vh with scroll overflow
- Admin section: pinned to bottom of sidebar via position="bottom"
- Update design-system to 0.1.54 (sidebar section maxHeight, position props)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
canSubmit no longer requires a JAR file when "Create only" is selected.
JAR upload and deploy steps are skipped when no file is provided.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Sidebar: make +App button more subtle (lower opacity, brightens on hover)
- Sidebar: add filter chips to hide empty routes and offline/stale apps
- Sidebar: hide filter chips and +App button when sidebar is collapsed
- Exchange table: reorder columns to Status, Attributes, App, Route, Started, Duration; remove ExchangeId and Agent columns
- Exchange detail log tab: query by exchangeId only (no applicationId required), filter by processorId when processor selected
- KPI tooltips: styled tooltips with current/previous values, time period labels, percentage change, themed with DS variables
- KPI tooltips: fix overflow by left-aligning first two and right-aligning last two
- Exchange detail: show full datetime (YYYY-MM-DD HH:mm:ss.SSS) for start/end times
- Status labels: unify to title-case (Completed, Failed, Running) across all views
- Status filter buttons: match title-case labels (Completed, Warning, Failed, Running)
- Create app: show full external URL using routingDomain from env config or window.location.origin fallback
- Create app: add Runtime Type selector and Custom Arguments to Resources tab
- Create app: add Sensitive Keys tab with agent defaults, global keys, and app-specific keys (matching admin page design)
- Create app: add placeholder text to all Input fields for consistency
- Update design-system to 0.1.52 (sidebar collapse toggle fix)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rename Java packages from com.cameleer3 to com.cameleer, module
directories from cameleer3-* to cameleer-*, and all references
throughout workflows, Dockerfiles, docs, migrations, and pom.xml.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The PULL_IMAGE deploy stage was a no-op — Docker only pulls on create
if the image is missing entirely, not when a newer version exists.
DeploymentExecutor now calls orchestrator.pullImage() to fetch the
latest base image from the registry before creating containers.
Also fixes the default base image from 'cameleer-runtime-base:latest'
(local-only name) to the fully qualified registry path
'gitea.siegeln.net/cameleer/cameleer-runtime-base:latest'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Four logging pipeline fixes:
1. Multi-replica startup logs: remove stopLogCaptureByApp from
SseConnectionManager — container log capture now expires naturally
after 60s instead of being killed when the first agent connects SSE.
This ensures all replicas' bootstrap output is captured.
2. Unified instance_id: container logs and agent logs now share the same
instance identity ({envSlug}-{appSlug}-{replicaIndex}). DeploymentExecutor
sets CAMELEER_AGENT_INSTANCEID per replica so the agent uses the same
ID as ContainerLogForwarder. Instance-level log views now show both
container and agent logs.
3. Labels-first container identity: TraefikLabelBuilder emits cameleer.replica
and cameleer.instance-id labels. Container names are tenant-prefixed
({tenantId}-{envSlug}-{appSlug}-{idx}) for global Docker daemon uniqueness.
4. Environment filter on log queries: useApplicationLogs now passes the
selected environment to the API, preventing log leakage across environments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract DockerClient creation into a standalone bean so both
runtimeOrchestrator and containerLogForwarder depend on it directly
instead of on each other. DockerRuntimeOrchestrator now receives
DockerClient via constructor instead of creating it in @PostConstruct.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detect when an agent instance already exists in the registry and record
a RE_REGISTERED event with route count and capabilities instead of a
generic REGISTERED event. UI shows a refresh icon for re-registrations.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The DS LogViewer expects level as a string union, but the API response
type uses plain string. Cast at the call site to fix the TS build error.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DatabaseAdminController's active-queries and kill-query endpoints could
expose SQL text from other tenants sharing the same PostgreSQL instance.
Added ApplicationName=tenant_{id} to the JDBC URL and filter
pg_stat_activity by application_name so each tenant only sees its own
connections.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add ContainerLogForwarder, StartupLogPanel, useStartupLogs to key classes
and UI files. Document log capture lifecycle and source badge rendering.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers streaming Docker logs to ClickHouse until agent SSE connect,
deployment log panel UI, and source badge in general log views.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Show agent built-in defaults as reference Badge pills, separate editable keys
section with count badge, amber-highlighted push toggle, right-aligned save
button. Fix info text: keys add to defaults, not replace. Add ClaimMapping
controller to CLAUDE.md.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The test endpoint now accepts inline rules from the client instead of reading
from the database, so unsaved rules can be tested. Matched rows show the
checkmark alongside action buttons instead of replacing them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds sensitiveKeys/globalSensitiveKeys/mergedSensitiveKeys fields to
ApplicationConfig, unwraps the new AppConfigResponse envelope in
useApplicationConfig, and renders an editable Sensitive Keys section
with read-only global pills and add/remove app-specific key tags.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- GET /config/{app} now returns AppConfigResponse with globalSensitiveKeys and mergedSensitiveKeys alongside the config
- PUT /config/{app} merges global + per-app sensitive keys before pushing CONFIG_UPDATE to agents via SSE
- extractSensitiveKeys() uses JsonNode reflection to avoid compile-time dependency on cameleer3-common getSensitiveKeys()
- SensitiveKeysRepository injected as new constructor parameter
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
GET/PUT /api/v1/admin/sensitive-keys (ADMIN only). PUT accepts optional
pushToAgents param — when true, fans out merged global+per-app sensitive
keys to all live agents via CONFIG_UPDATE SSE commands with 10-second
shared deadline. Per-app keys extracted via JsonNode to avoid depending
on ApplicationConfig.getSensitiveKeys() not yet in the published
cameleer3-common jar. Includes audit logging on every PUT.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The roles-claim and default-roles fallback paths in applyClaimMappings
were using assignRoleToUser (origin='direct'), causing OIDC-derived
roles to accumulate across logins and never be cleared. Changed both
to assignManagedRole (origin='managed') so all OIDC-assigned roles
are cleared and re-evaluated on every login, same as claim mapping
rules. Only roles assigned directly via the admin UI are preserved.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All edits (add, edit, delete, reorder) now modify local state only.
Cancel discards changes, Apply diffs local vs server and issues the
necessary create/update/delete API calls. Target selects now include
a placeholder option. Footer shows Cancel and Apply buttons.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Populate target field from existing roles (assign role) or groups
(add to group) instead of free-text input, preventing typos.
Switching action resets the target selection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace non-existent --surface-1/--surface-2 with --bg-raised (modal)
and --bg-hover (subtle backgrounds) from the design system.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Bump all font sizes from 11px/10px to 12px (project minimum)
- Fix handleMove race condition: use mutateAsync + Promise.all
- Clear stale test results after rule create/edit/delete/reorder
- Replace inline styles with CSS module classes in OidcConfigPage
- Remove dead .editRow CSS class
- Replace inline chevron with Lucide icon
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Now that cameleer3-common has getInputProperties/getOutputProperties on
ProcessorExecution, add the check to the processors_json deserialization
path as well.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The hasTrace flag on ProcessorNode now also checks inputProperties and
outputProperties on the flat-record code paths (buildTreeBySeq and
buildTreeByProcessorId). The ProcessorExecution path (processors_json)
will be updated once cameleer3-common publishes the new snapshot.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add support for exchange properties sent by the agent alongside headers.
Properties flow through the same pipeline as headers: ClickHouse columns
(input_properties, output_properties) on both executions and
processor_executions tables, MergedExecution record, ChunkAccumulator
extraction, DetailService snapshot, and REST API response.
UI adds a Properties tab next to Headers in the process diagram detail
panel, with the same input/output split table layout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds a subtle "+ App" button in the sidebar section header for quick
app creation without navigating to the Deployments tab first. Only
visible to OPERATOR and ADMIN roles.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch Vite base back to './' (relative paths) and always inject
<base href="${BASE_PATH}"> in the entrypoint, even when BASE_PATH=/.
This fixes asset loading for both deployment modes:
- Single-instance: <base href="/"> resolves ./assets/x.js to /assets/x.js
- SaaS tenant: <base href="/t/slug/"> resolves to /t/slug/assets/x.js
Previously base:'/' produced absolute /assets/ paths that the <base>
tag couldn't redirect, breaking SaaS tenants. And base:'./' without
<base> broke deep URLs in single-instance mode. Always injecting the
tag makes relative paths work universally.
The patched server-ui-entrypoint.sh in cameleer-saas (which rewrote
absolute href/src attributes via sed) is no longer needed and can be
removed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Revert base: './' back to '/' — relative asset paths break on deep
URLs like /dashboard/app/route where the browser resolves assets to
/dashboard/app/assets/ instead of /assets/.
Also fix processor metrics table clipping: remove flex:1/min-height:0
from .processorSection so the table takes its natural content height
and the page scrolls to show all rows (was clipping at ~12 of 18).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the server-ui is deployed under a subpath (/t/{slug}/), absolute
asset paths (/assets/...) resolve to the domain root instead of the
subpath, causing 404s. Using './' makes asset URLs relative to the
HTML page, so they resolve correctly regardless of mount path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse does not have lag() as a window function. Use lagInFrame()
with explicit ROWS BETWEEN frame instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Not needed yet -- all deployments are under our control and can be
reset manually if the old schema is encountered.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite init.sql as a pure CREATE IF NOT EXISTS file with no DROP or
INSERT statements. Safe for repeated runs on every startup without
corrupting aggregated stats data.
Old deployments with count()-based stats tables are migrated
automatically: ClickHouseSchemaInitializer checks system.columns for
the old AggregateFunction(count) type and drops those tables before
init.sql recreates them with the correct uniq() schema. This runs
once per table and is a no-op on fresh installs or already-migrated
deployments.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Change vite base from './' to '/' so asset paths are absolute. With
relative paths, direct navigation to multi-segment URLs like
/runtime/app/instance resolved assets to /runtime/assets/ which 404'd.
Also fix sidebar navigation: clicking a route while on the runtime tab
no longer navigates to /runtime/{appId}/{routeId} (which the runtime
page interprets as an instanceId). It stays at /runtime/{appId}.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Counter metrics like chunks.exported.count are monotonically increasing.
Add mode=delta query parameter to the agent metrics API that computes
per-bucket deltas server-side using ClickHouse lag() window function:
max(value) per bucket, then greatest(0, current - previous) to get the
increase per period with counter-reset handling.
The chunks exported/dropped charts now show throughput per bucket
instead of the ever-increasing cumulative total.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extend uniq-based dedup from processor tables to all stats tables
(stats_1m_all, stats_1m_app, stats_1m_route). Execution-level tables
use uniq(execution_id). Processor-level tables now use
uniq(concat(execution_id, toString(seq))) so loop iterations (same
exchange, different seq) are counted while chunk retry duplicates
(same exchange+seq) are collapsed.
All stats tables are dropped, recreated, and backfilled from raw
data on startup. All Java queries updated: countMerge -> uniqMerge,
countIfMerge -> uniqIfMerge.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The ClickHouseSchemaInitializer splits on semicolons before filtering
comments, so semicolons inside comment text created invalid statements.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Processor execution counts were inflated by duplicate inserts into the
plain MergeTree processor_executions table (chunk retries, reconnects).
Replace count()/countIf() with uniq(execution_id)/uniqIf() in both
stats_1m_processor and stats_1m_processor_detail MVs so each exchange
is counted once per processor regardless of duplicates.
Tables are dropped and rebuilt from raw data on startup. MV created
after backfill to avoid double-counting.
Also adds stats_1m_processor_detail to the catalog purge list (was
missing).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove redundant "X/X LIVE" badge from runtime page, breadcrumb trail
and routes section from agent detail page (pills moved into Process
Information card). Fix session expiry: guard against concurrent 401
refresh races and skip re-entrant triggers on auth endpoints.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The wheel event listener was attached in a useEffect with empty deps,
but the SVG element doesn't exist during the loading state. Switch
svgRef from a plain ref to a callback ref that triggers re-attachment
when the SVG element becomes available after data loads.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Renders an HTML tooltip below hovered diagram nodes with processor
metrics (avg, p99, % time, invocations, error rate). Styled inline
with the existing NodeToolbar pattern — positioned via screen-space
coordinates, uses DS tokens for background/border/shadow/typography.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dashboard L3 now shows a single Processor Metrics card with
Diagram/Table toggle buttons. The diagram shows native tooltips on
hover with full processor metrics (avg, p99, invocations, error rate,
% time).
Also fixes:
- Chart x-axis uses actual timestamps instead of bucket indices
- formatDurationShort uses locale formatting with max 3 decimals
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After server restart, agents send logs before re-registering. Instead
of dropping these logs, fall back to application and environment from
the JWT token claims. Only drops logs when neither registry nor JWT
provide an applicationId.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Avoids Date round-trip that crashes with toISOString() on invalid
timestamps from the timeseries API.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace custom LineChart/AreaChart/BarChart usage with ThemedChart
wrapper. Data format changed from ChartSeries[] to Recharts-native
flat objects. Uses DS v0.1.47.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
All chart series now use Date objects from the API response instead
of integer indices. This gives proper date/time on x-axes and in
tooltips (leveraging DS v0.1.46 responsive charts + timestamp
tooltips). GC chart switched from BarChart to AreaChart for
consistency with Date x-values.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Agent team migrated from JMX to Micrometer metrics. Update the 5
hardcoded metric names in AgentInstance.tsx JVM charts:
- jvm.cpu.process → process.cpu.usage.value
- jvm.memory.heap.used → jvm.memory.used.value
- jvm.memory.heap.max → jvm.memory.max.value
- jvm.threads.count → jvm.threads.live.value
- jvm.gc.time → jvm.gc.pause.total_time
Server backend is unaffected (generic MetricsSnapshot storage).
CLAUDE.md updated with full agent metric name reference.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Lists all business metrics (gauges, counters, timers) with their
tags and source classes, plus agent container label mapping table.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Labels prometheus.scrape, prometheus.path, and prometheus.port are now
set on every deployed container based on the resolved runtime type,
enabling automatic Prometheus service discovery via docker_sd_configs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously showed an infinite spinner because unmanaged apps have no
PostgreSQL record. Now shows an "Unmanaged Application" message with
a link to create a managed app.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Server defaultConfig() and UI fallbacks returned "NONE" for payload
capture, but the agent defaults to "BOTH". This caused unwanted
reconfiguration when users saved other settings — payload capture
would silently change from the agent's default BOTH to NONE.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sets query cache immediately on dismiss success so the sidebar updates
without waiting for the catalog refetch to complete.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Apps can now join additional Docker networks (e.g., monitoring,
prometheus) configured via containerConfig.extraNetworks. Flows through
the 3-layer config merge. Networks are created if absent and containers
are connected during deployment. UI adds a pill-list field on the
Resources tab (both create and edit views).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace window.confirm with design system ConfirmDialog for the dismiss
action. Move the "No agents connected" section to the top of the Runtime
page using Alert component with warning variant.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Spring's default handler silently returns 400 for malformed payloads
with no server-side log. Added @ExceptionHandler to catch and WARN with
the agent instance ID and root cause message.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Previously the endpoint silently returned 202 for all failures: missing
agent identity, unregistered agents, empty payloads, and buffer-full
drops. Now logs WARN for each failure case with context (instanceId,
entry count, reason). Normal ingestion logged at INFO with accepted
count. Buffer-full drops tracked individually with accepted/dropped
counts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After versionRepo.create(), detect the runtime type from the saved JAR
via RuntimeDetector and persist the result via updateDetectedRuntime().
Log messages now include the detected runtime type (or 'unknown').
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Flyway V10 migration adds the two nullable columns. AppVersion record,
AppVersionRepository interface, and PostgresAppVersionRepository are
updated to carry and persist detected runtime information.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LogEntry.getSource() exists but source is not a constructor parameter
in cameleer3-common — it uses a default value.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All constructor calls updated to include the new source field added
in the log forwarding v2 changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
HOWTO.md: log ingestion example updated from LogBatch wrapper to raw
JSON array with source field. CLAUDE.md: added LogIngestionController,
updated LogQueryController with new filters. SERVER-CAPABILITIES.md:
updated log ingestion and query descriptions, ClickHouse table note.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace LogBatch wrapper with raw List<LogEntry> on the ingestion endpoint.
Add source column to ClickHouse logs table and propagate it through the
storage, search, and HTTP layers (LogSearchRequest, LogEntryResult,
LogEntryResponse, LogQueryController).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds a useServerCapabilities hook that fetches /api/v1/health once per
session (staleTime: Infinity) and extracts the infrastructureEndpoints
flag. buildAdminTreeNodes now accepts an opts parameter so ClickHouse
and Database tabs are hidden when the server reports infra endpoints as
disabled. LayoutShell wires the hook result into the admin tree memo.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add cameleer.server.security.infrastructureendpoints property (default true) and
@ConditionalOnProperty to DatabaseAdminController and ClickHouseAdminController so
the SaaS provisioner can set CAMELEER_SERVER_SECURITY_INFRASTRUCTUREENDPOINTS=false
to suppress these endpoints (404) on tenant server containers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
14-task plan covering server-side @ConditionalOnProperty flag,
health endpoint capability exposure, UI sidebar filtering,
SaaS provisioner env var, and vendor infrastructure dashboard
with per-tenant PostgreSQL and ClickHouse visibility.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Covers restricting DB/ClickHouse admin endpoints in SaaS-managed
server instances via @ConditionalOnProperty flag, and building a
vendor-facing infrastructure dashboard in the SaaS platform with
per-tenant PostgreSQL and ClickHouse visibility.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
HOWTO.md configuration table rewritten with correct cameleer.server.*
property names, grouped by functional area. Removed stale CAMELEER_OIDC_*
env var references. SERVER-CAPABILITIES.md updated with correct env var
names for ingestion and agent registry tuning.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move container resource defaults into their own sub-namespace for
future extensibility:
cameleer.server.runtime.container.memorylimit → CAMELEER_SERVER_RUNTIME_CONTAINER_MEMORYLIMIT
cameleer.server.runtime.container.cpushares → CAMELEER_SERVER_RUNTIME_CONTAINER_CPUSHARES
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move OIDC properties into a nested Oidc class within SecurityProperties
for clearer grouping. Env vars gain an extra separator:
cameleer.server.security.oidc.issueruri → CAMELEER_SERVER_SECURITY_OIDC_ISSUERURI
cameleer.server.security.oidc.jwkseturi → CAMELEER_SERVER_SECURITY_OIDC_JWKSETURI
cameleer.server.security.oidc.audience → CAMELEER_SERVER_SECURITY_OIDC_AUDIENCE
cameleer.server.security.oidc.tlsskipverify → CAMELEER_SERVER_SECURITY_OIDC_TLSSKIPVERIFY
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse is the only storage backend — there is no alternative.
The enabled flag created a false sense of optionality: setting it to
false would crash on startup because most beans unconditionally depend
on the ClickHouse JdbcTemplate.
Remove all @ConditionalOnProperty annotations gating ClickHouse beans,
the enabled property from application.yml, and the K8s manifest entry.
Also fix old property names in AbstractPostgresIT test config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move all configuration properties under the cameleer.server.* namespace
with all-lowercase dot-separated names and mechanical env var mapping
(dots→underscores, uppercase). This aligns with the agent's convention
(cameleer.agent.*) and establishes a predictable pattern across all
components.
Changes:
- Move 6 config prefixes under cameleer.server.*: agent-registry,
ingestion, security, license, clickhouse, and cameleer.tenant/runtime/indexer
- Rename all kebab-case properties to concatenated lowercase
(e.g., bootstrap-token → bootstraptoken, jar-storage-path → jarstoragepath)
- Update all env vars to CAMELEER_SERVER_* mechanical mapping
- Fix container-cpu-request/container-cpu-shares mismatch bug
- Remove displayName from AgentRegistrationRequest (redundant with instanceId)
- Update agent container env vars to CAMELEER_AGENT_* convention
- Update K8s manifests and CI workflow for new env var names
- Update CLAUDE.md, HOWTO.md, SERVER-CAPABILITIES.md documentation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Admins can now disable route control and replay per environment via the
Default Resource Limits section. Both default to enabled. Apps in the
environment inherit these defaults unless overridden per-app.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Added routeControlEnabled and replayEnabled to ResolvedContainerConfig,
flowing through the three-layer config merge (global -> env -> app).
Both default to true. Admins can disable them per environment (e.g.
prod) via the defaultContainerConfig JSONB, or per app via the app's
containerConfig JSONB.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
buildEnvVars was missing CAMELEER_ROUTE_CONTROL_ENABLED and
CAMELEER_REPLAY_ENABLED, so deployed app containers defaulted to false
and agents didn't announce these capabilities.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The .section base class sets flex-direction: column, which caused the
config bar items (App Log Level, Agent Log Level, etc.) to stack
vertically instead of displaying in a horizontal row.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The server container mounts the platform's certs volume at /certs but
the CA bundle was never imported into the JVM truststore. OIDC discovery
failed with PKIX path building errors when a self-signed or custom CA
was in use.
The new entrypoint script splits the PEM bundle and imports each cert
via keytool before starting the app. This makes the conditional
CAMELEER_OIDC_TLS_SKIP_VERIFY logic in the SaaS provisioner work
correctly: when ca.pem exists, the JVM now actually trusts it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The password reset endpoint was fully blocked under OIDC mode. Now
M2M callers (identified by oidc: principal prefix) can reset local
user passwords, enabling the SaaS platform to manage the server's
built-in admin credentials.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allows the SaaS platform to identify and clean up all containers
belonging to a tenant on delete (cameleer/cameleer-saas#55).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace simple key-value rows with EnvEditor component that supports
editing variables as Table, Properties, YAML, or .env format.
Switching views converts data seamlessly. Includes file import
(drag-and-drop .properties/.yaml/.env) with auto-detect and merge.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cast the Set<string> from ButtonGroup.onChange to Set<ExchangeStatus>
before iterating, fixing TS2345 from DS TopBar decomposition.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update to @cameleer/design-system@0.1.40 which decomposes TopBar into a
composable shell. Move status filters, time range, search trigger, and
auto-refresh toggle from the DS TopBar into LayoutShell as composed
children. Fixescameleer/cameleer-saas#53.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Always include urn:logto:scope:organizations and
urn:logto:scope:organization_roles in OIDC auth requests. These are
required for role mapping in multi-tenant setups and harmless for
non-Logto providers (unknown scopes ignored per OIDC spec).
Filter them from the OIDC admin config page so they don't confuse
standalone server admins or SaaS tenants.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The OIDC callback extracted roles from the token's Custom JWT claim
(e.g. roles: [server:admin]) but never used them. The
applyClaimMappings fallback only assigned defaultRoles (VIEWER).
Now the fallback priority is: claim mapping rules > OIDC token
roles > defaultRoles. This ensures users get their org-mapped
roles (owner → server:admin) without requiring manual claim
mapping rule configuration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Environment networks now include the tenant ID to prevent cross-tenant
collisions: cameleer-env-{tenantId}-{envSlug} instead of cameleer-env-
{envSlug}. Without this, two tenants with a "dev" environment would
share the same Docker network.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Instead of hardcoding cameleer-traefik as the primary network for
deployed app containers, use CAMELEER_DOCKER_NETWORK (env var). In
SaaS mode this is the tenant-isolated network (cameleer-tenant-{slug}).
Apps still connect to cameleer-traefik (for routing) and cameleer-env-
{slug} (for intra-environment discovery) as additional networks.
This enables per-tenant network isolation: apps deployed by tenant A
cannot reach apps deployed by tenant B since they share no network.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The tableSection card wrapper broke the flex height chain — DataTable's
fillHeight couldn't constrain to viewport. Added .tableWrap with
flex: 1, min-height: 0, display: flex to re-establish the chain.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The relative `to="apps"` didn't resolve correctly. All other legacy
redirects use absolute paths (`to="/apps"`, `to="/runtime"`).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- 7.1: Add deployment status badge (StatusDot + Badge) to AppsTab app
list, sourced from catalog.deployment.status via slug lookup
- 7.3: Add X close button to top-right of exchange detail right panel
in ExchangesPage (position:absolute, triggers handleClearSelection)
- 7.5: PunchcardHeatmap shows "Requires at least 2 days of data"
when timeRangeMs < 2 days; DashboardL1 passes the range down
- 7.6: Command palette exchange results truncate IDs to ...{last8}
matching the exchanges table display
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds amber edit-mode banners to AppConfigDetailPage and both
DefaultResourcesSection/JarRetentionSection in EnvironmentsPage,
matching the existing ConfigSubTab pattern.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- RolesTab: wrap \u00b7 in JS expression {'\u00b7'} so JSX renders the middle dot correctly instead of literal backslash-u sequence
- UsersTab: add confirm password field with mismatch validation, hint text for password policy, and reset on cancel/success
- UserManagement.module.css: add .hintText style for password policy hint
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Throughput chart: divide totalCount by bucket duration (seconds) so Y-axis shows true msg/s instead of raw bucket counts; fixes flat-line appearance when TPS is low but totalCount is large
- Error Rate chart: convert failedCount/totalCount to percentage; change yLabel from "err/h" to "%" to match KPI stat card unit
- Memory chart: add threshold line at jvm.memory.heap.max so chart Y-axis extends to max heap and shows the reference line (spec 5.3)
- Agent state: suppress containerStatus badge when value is "UNKNOWN"; only render it with "Container: <state>" label when a non-UNKNOWN secondary state is present (spec 5.4)
- DashboardTab chartGrid: add pointer-events:none with pointer-events:auto on children so the chart grid overlay does not intercept clicks on the Application Health table rows below (spec 5.5)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Attributes column is now hidden when no exchanges in the current view
have attributes; shown conditionally via hasAttributes check on rows
- Status labels already standardized via statusLabel() in ExchangeHeader
- Agent names truncated to last two hyphen-separated segments via
shortAgentName(); full name preserved as tooltip title
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- formatDuration and formatDurationShort now show Xm Ys for durations >= 60s (e.g. "5m 21s" instead of "321s") and 1 decimal for 1-60s range ("6.7s" instead of "6.70s")
- Exchange ID column shows last 8 chars with ellipsis prefix; full ID on hover, copies to clipboard on click
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Override design system tokens in app root CSS: --text-muted raised to 4.5:1
contrast in both light (#766A5E) and dark (#9A9088) modes; --text-faint dark
mode raised from catastrophic 1.4:1 to 3:1 (#6A6058). Migrate --text-faint
usages on readable text (empty states, italic notes, buttons) to --text-muted.
Raise all 10px and 11px font-size declarations to 12px floor.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
adminFetch called logout() directly on 401/403 responses, which cleared
roles and caused RequireAdmin to redirect to /exchanges while users were
editing forms. Now adminFetch attempts a token refresh before failing,
and RequireAdmin tolerates a transient empty-roles state during refresh.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Return a JSON error body from UserAdminController instead of an empty 400,
and extract API error messages in adminFetch so toasts display the reason.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Detailed step-by-step plan covering critical bug fixes, layout/interaction
consistency, WCAG contrast compliance, data formatting, chart fixes, and
admin polish. Each task includes exact file paths, code snippets, and
verification steps.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Playwright-driven audit of the live UI (build 69dcce2, 60+ screenshots)
covering all pages, CRUD lifecycles, design consistency, and interaction
patterns. Spec defines 8 batches of work: critical bugs, layout
consistency, interaction consistency, contrast/readability, data
formatting, chart fixes, admin polish, and nice-to-have items.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backend: AgentRegistryService gains findByApplicationAndEnvironment()
and environment-aware addGroupCommandWithReplies() overload.
AgentCommandController and ApplicationConfigController accept optional
environment query parameter. When set, commands only target agents in
that environment. Backward compatible — null means all environments.
Frontend: All command mutations (config update, route control, traced
processors, tap config, route recording) now pass selectedEnv to the
backend via query parameter.
Prevents cross-environment command leakage — e.g., updating config for
prod no longer pushes to dev agents.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add environment parameter to AgentEventsController, AgentEventService,
and ClickHouseAgentEventRepository (filters agent_events by environment)
- Wire selectedEnv to useAgents and useAgentEvents in both AgentHealth
and AgentInstance pages
- Wire selectedEnv to useStatsTimeseries in AgentInstance
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Part A: Environment creation slug is now auto-derived from display name
and shown read-only (matching app creation pattern). Removes manual slug
input.
Part B: All data queries now pass the selected environment to backend:
- Exchanges search, Dashboard L1/L2/L3 stats, Routes metrics, Route
detail, correlation chains, and processor metrics all filter by
selected environment.
- Backend RouteMetricsController now accepts environment parameter for
both route and processor metrics endpoints.
Closes #XYZ
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The DockerEventMonitor only reacted to Docker events. If an event was
missed (e.g., during reconnect or startup race), a DEGRADED deployment
with all replicas healthy would never promote back to RUNNING.
Add a @Scheduled reconciliation (every 30s) that inspects actual
container state and corrects deployment status mismatches.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full audit of design system adoption, color consistency, inline styles,
layout patterns, and CSS module duplication across the server UI.
Includes 6-phase fix plan.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Prevent removal of last ADMIN role via role unassign, user delete,
or group role removal (returns 409 Conflict)
- Add password policy: min 12 chars, 3/4 character classes, no username
- Add brute-force protection: 5 attempts then 15min lockout, IP rate limit
- Add token revocation on password change via token_revoked_before column
- V9 migration adds failed_login_attempts, locked_until, token_revoked_before
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes pre-existing TS2322 where Record<string, string> was not
assignable to the StatusDotVariant union type.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consolidate 20+ duplicate function definitions across UI components into
three shared util files (format-utils, agent-utils, config-draft-utils).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Part 1 — Config snapshot:
- V8 migration adds resolved_config JSONB to deployments table
- DeploymentExecutor saves the full resolved config at deploy time
- Deployment record includes resolvedConfig for auditability
Part 2 — Composite health StatusDot:
- CatalogController computes composite health from deployment status +
agent health (green only when RUNNING AND agent live)
- CatalogApp includes healthTooltip (e.g. "Deployment: RUNNING,
Agents: live (1 connected)")
- StatusDot added to app detail header with deployment status Badge
- StatusDot added to deployment table rows
- Sidebar passes composite health + tooltip through to tree nodes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Compare app.updatedAt with deployment.deployedAt — if config was
modified after the deployment started, show a primary "Redeploy" button
in the Actions column. Also show a toast hint after saving config:
"Redeploy to apply changes to running deployments."
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Rename cpuShares to cpuRequest (millicores), cpuLimit from cores to
millicores. ResolvedContainerConfig translates to Docker-native units
via dockerCpuShares() and dockerCpuQuota() helpers. Future K8s
orchestrator can pass millicores through directly.
- Fix waitForAnyHealthy to wait for ALL replicas instead of returning
on first healthy one. Prevents false DEGRADED status with 2+ replicas.
- Default app detail to Configuration tab (was Overview)
- Reorder config sub-tabs: Monitoring, Resources, Variables, Traces &
Taps, Route Recording
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Normalize the sidebar selectedPath so the app highlight persists across
tab switches (Dashboard, Runtime, Deployments). Also make sidebar clicks
tab-aware: clicking an app navigates to the current tab's path instead
of always going to /exchanges/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Consolidate route catalog (agent-driven) and apps table (deployment-
driven) into a single GET /api/v1/catalog?environment={slug} endpoint.
Apps table is authoritative; agent data enriches with live health,
routes, and metrics. Unmanaged apps (agents without App record) appear
with managed=false.
- Add CatalogController merging App records + agent registry + ClickHouse
- Add CatalogApp DTO with deployment summary, managed flag, health
- Change AppController and DeploymentController to accept slugs (not UUIDs)
- Add AppRepository.findBySlug() and AppService.getBySlug()
- Replace useRouteCatalog() with useCatalog() across all UI components
- Navigate to /apps/{slug} instead of /apps/{UUID}
- Update sidebar, search, and all catalog lookups to use slug
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Unify route catalog (agent-driven) and apps table (deployment-driven)
into a single catalog endpoint. Apps table becomes authoritative,
agent data enriches with live health/routes. Slug-based URLs replace
UUIDs for navigation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Redesign DeploymentProgress component: track-based layout with amber
brand color, checkmarks for completed steps, user-friendly labels
(Prepare, Image, Network, Launch, Verify, Activate, Live)
- Delete terminal (STOPPED/FAILED) deployments before creating new ones
for the same app+environment, preventing duplicate rows in the UI
- Update CLAUDE.md with comprehensive key class locations, correct deploy
stages, database migration reference, and REST endpoint summary
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The runtime-base image and all agent Dockerfiles now read
CAMELEER_SERVER_URL instead of CAMELEER_EXPORT_ENDPOINT.
Updated the volume-mode entrypoint override to match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When CAMELEER_JAR_DOCKER_VOLUME is set, the orchestrator mounts the
named volume at the jar storage path instead of using a host bind mount.
This solves the path translation issue in Docker-in-Docker setups where
the server runs inside a container and manages sibling containers.
The entrypoint is overridden to use the volume-mounted JAR path via
the CAMELEER_APP_JAR env var.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Docker's connectToNetworkCmd needs the network ID (not name) and the
container's network sandbox must be ready. Moving network connection
to DeploymentExecutor where DockerNetworkManager handles ID resolution
and the container is already started.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Nginx defaults to 1MB body size, causing 413 on JAR uploads through
the UI proxy. Matches the Spring Boot multipart limit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The opacity:0 approach caused the native "Choose File" button to
appear in the accessibility tree and compete for clicks. The clip
pattern properly hides the input while keeping it functional for
programmatic .click().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Some browsers block programmatic .click() on display:none inputs.
Using position:absolute + opacity:0 keeps the input in the render tree.
Also added type="button" to prevent any form-submission interference.
Applied to both create page and detail view file inputs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Listens to Docker daemon events (die, oom, start, stop) for containers
labeled managed-by=cameleer3-server, updates replica states in Postgres,
and recomputes aggregate deployment status (RUNNING/DEGRADED/FAILED).
Bean is wired in RuntimeOrchestratorAutoConfig via instanceof guard so it
only activates when Docker is available.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Extends Deployment with targetState, deploymentStrategy, replicaStates
(List<Map<String,Object>>), and deployStage. Updates withStatus() to
carry the new fields through.
Adds DEGRADED and STOPPING to DeploymentStatus (reordered for lifecycle
clarity). Introduces DeployStage enum for tracking orchestration progress
through PRE_FLIGHT → COMPLETE.
The cameleer-traefik network disables inter-container communication
so app containers cannot reach each other directly — only through
Traefik. Environment networks keep ICC enabled for intra-env comms.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Per-environment "keep last N versions" setting (default 5, null for
unlimited). Nightly scheduled job at 03:00 deletes old versions from
both database and disk, skipping any version that is currently deployed.
Full stack:
- V6 migration: adds jar_retention_count column to environments
- Environment record, repository, service, admin controller endpoint
- JarRetentionJob: @Scheduled nightly, iterates environments and apps
- UI: retention policy editor on admin Environments page with
toggle between limited/unlimited and version count input
- AppVersionRepository.delete() for version cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New "Default Resource Limits" section in environment detail view with
memory limit/reserve, CPU shares/limit. These defaults apply to new
apps unless overridden per-app.
Added useUpdateDefaultContainerConfig hook for the PUT endpoint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Environment Variables moved from Resources into a dedicated "Variables"
tab, placed first in the tab order since it's the most commonly needed
config when creating new apps.
Tab order:
- Create page: Variables | Monitoring | Resources
- Detail page: Variables | Monitoring | Traces & Taps | Route Recording | Resources
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
As the user types the app name, the URL builds in real-time:
/{envSlug}/{appSlug}/
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Config sub-tabs are now: Monitoring | Traces & Taps | Route Recording | Resources
(renamed from Agent/Infrastructure, with traces and recording as their own tabs).
Also increase Spring multipart max-file-size and max-request-size to 200MB
to fix HTTP 413 on JAR uploads.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ConfigSubTab now uses inner tabs (Agent / Infrastructure):
- Agent: observability settings, compress success, traces & taps table,
route recording toggles
- Infrastructure: container resources, exposed ports, environment variables
This completes the Config tab consolidation — all features from the
standalone Config page now live in the Deployments tab.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Logs functionality already exists in Runtime tab (AgentHealth/AgentInstance).
Config functionality moved to Deployments tab ConfigSubTab.
Old routes redirect to /runtime and /apps respectively.
Navigation links updated throughout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Prevents accidental app deletion by requiring the user to type the app
slug before confirming.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace inline create form with a modal that handles the full flow:
- Name → auto-computed slug (editable if needed)
- Environment picker
- JAR file upload
- "Deploy immediately" toggle (on by default)
- Single "Create & Deploy" button runs all three API calls sequentially
with step indicator
After creation, navigates directly to the new app's detail view.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Overview sub-tab:
- Deployments table with env badge, version, status, URL, deployed time
- Actions (Start/Stop) scoped to selected environment; other envs show
"switch env to manage" hint with muted rows
- Versions list with per-env deploy target picker
Configuration sub-tab:
- Read-only by default with Edit mode gate (Cancel/Save banner)
- Agent observability: engine level, payload capture with size unit
selector, log levels, metrics toggle, sampling, replay and route
control (default enabled)
- Container resources: memory/CPU limits, exposed ports as deletable
pills with inline add input
- Environment variables: key-value editor with add/remove
- Reuses existing ApplicationConfig API for agent config push via SSE
Tab renamed from "Apps" to "Deployments" in the tab bar.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The list endpoint on EnvironmentAdminController now overrides the
class-level ADMIN guard with isAuthenticated(), so VIEWERs can see
the environment selector. The LayoutShell merges environments from
both the table and agent heartbeats, so the selector always shows
configured environments even when no agents are connected.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
apps.updated_at already exists from V3. The duplicate ALTER caused
Flyway to fail on startup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Apps tab visible to OPERATOR+ (hidden for VIEWER), scoped by
sidebar app selection and environment filter
- List view: DataTable with name, environment, updated, created columns
- Detail view: deployments across all envs, version upload with
per-env deploy target, container config form (resources, ports,
custom env vars) with explicit Save
- Memory reserve field disabled for non-production environments
with info hint
- Admin sidebar sorted alphabetically, Applications entry removed
- Old admin AppsPage deleted
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- V5 migration: container_config JSONB + updated_at on apps,
default_container_config JSONB on environments
- App/Environment records updated with new fields
- PUT /apps/{id}/container-config endpoint for per-app config
- PUT /admin/environments/{id}/default-container-config for env defaults
- GET /apps now supports optional environmentId (lists all when omitted)
- AppRepository.findAll() for cross-environment app listing
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- SplitPane layout with environment selector, app list, and detail pane
- Create/delete apps with slug uniqueness validation
- Upload JAR versions with file size display
- Deploy versions and stop running deployments with status badges
- Deployment list auto-refreshes every 5s for live status updates
- Registered at /admin/apps with sidebar entry
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add GET /api/v1/auth/me endpoint returning current user's UserDetail
- Add AboutMeDialog component with role badges and group memberships
- Add userMenuItems prop to TopBar via design-system update
- Wire "About Me" menu item into user dropdown above Logout
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
getDirectRolesForUser filtered on origin='direct', which excluded
roles assigned via claim mapping (origin='managed'). This caused
OIDC users to appear roleless even when claim mappings matched.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When no claim mapping rules are configured or none match the JWT
claims, fall back to assigning the OidcConfig.defaultRoles (e.g.
VIEWER). This restores the behavior that was lost when syncOidcRoles
was replaced with claim mapping.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
SplitPane with create/edit/delete, production flag toggle,
enabled/disabled toggle. Follows existing admin page patterns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Environments now have:
- production (bool): prod vs non-prod resource allocation
- enabled (bool): disabled blocks new deployments
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Local login was blocked when OIDC env vars were present, causing
bootstrap to fail (chicken-and-egg: bootstrap needs local auth to
configure OIDC). The backend now accepts both auth paths; the
frontend/UI decides which login flow to present.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- JAR storage path, base image, Docker network
- Container memory/CPU limits, health check timeout
- Routing mode and domain for Traefik integration
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- EnvironmentAdminController: CRUD under /api/v1/admin/environments (ADMIN)
- AppController: CRUD + JAR upload under /api/v1/apps (OPERATOR+)
- DeploymentController: deploy, stop, promote, logs under /api/v1/apps/{appId}/deployments
- Security rule for /api/v1/apps/** requiring OPERATOR or ADMIN role
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Async container deployment with health check polling
- Stops previous deployment before starting new one
- Configurable memory, CPU, health timeout via application properties
- @EnableAsync on application class for Spring async proxy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- EnvironmentService: CRUD with slug uniqueness, default env protection
- AppService: CRUD, JAR upload with SHA-256 checksumming
- DeploymentService: create, promote, status transitions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- EnvironmentRepository, AppRepository, AppVersionRepository, DeploymentRepository
- RuntimeOrchestrator interface with ContainerRequest and ContainerStatus
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- SecurityBeanConfig uses Ed25519SigningServiceImpl.ephemeral() when no jwt-secret
- Fixes pre-existing application context failure in integration tests
- Reverts test jwt-secret from application-test.yml (no longer needed)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ClaimMappingAdminControllerIT with create+list and delete tests
- Add adminHeaders() convenience method to TestSecurityHelper
- Add jwt-secret to test profile (fixes pre-existing Ed25519 init failure)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ADMIN-only REST endpoints at /api/v1/admin/claim-mappings
- Full CRUD: list, get by ID, create, update, delete
- OpenAPI annotations for Swagger documentation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- UiAuthController.login returns 404 when OIDC issuer is configured
- JwtAuthenticationFilter skips internal user tokens in OIDC mode (agents still work)
- UserAdminController.createUser and resetPassword return 400 in OIDC mode
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- OidcUserInfo now includes allClaims map from id_token + access_token
- OidcAuthController.callback() calls applyClaimMappings instead of syncOidcRoles
- applyClaimMappings evaluates rules, clears managed assignments, applies new ones
- Supports both assignRole and addToGroup actions
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add clearManagedAssignments, assignManagedRole, addUserToManagedGroup to interface
- Update assignRoleToUser and addUserToGroup to explicitly set origin='direct'
- Update getDirectRolesForUser to filter by origin='direct'
- Implement managed assignment methods with ON CONFLICT upsert
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GET /api/v1/admin/license returns current license info
- POST /api/v1/admin/license validates and loads new license token
- Requires ADMIN role, validates Ed25519 signature before applying
- OpenAPI annotations for Swagger documentation
- LicenseBeanConfig wires LicenseGate bean with startup validation
- Supports token from CAMELEER_LICENSE_TOKEN env var or CAMELEER_LICENSE_FILE path
- Falls back to open mode when no license or no public key configured
- Add license config properties to application.yml
- JdbcTemplate-based CRUD for claim_mapping_rules table
- RbacBeanConfig wires ClaimMappingRepository and ClaimMappingService beans
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Thread-safe AtomicReference-based license holder
- Defaults to open mode (all features enabled) when no license loaded
- Runtime license loading with feature/limit queries
- Unit tests for open mode and licensed mode
- Evaluates JWT claims against mapping rules
- Supports equals, contains (list + space-separated), regex match types
- Results sorted by priority
- 7 unit tests covering all match types and edge cases
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Validates payload.signature license tokens using Ed25519 public key
- Parses tier, features, limits, timestamps from JSON payload
- Rejects expired and tampered tokens
- Unit tests for valid, expired, and tampered license scenarios
- AssignmentOrigin enum (direct/managed)
- ClaimMappingRule record with match type and action enums
- ClaimMappingRepository interface for CRUD operations
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add origin and mapping_id columns to user_roles and user_groups
- Create claim_mapping_rules table with match_type and action constraints
- Update primary keys to include origin column
- Add indexes for fast managed assignment cleanup
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract inline fontSize/color styles from LogTab, LayoutShell,
UsersTab, GroupsTab, RolesTab, and LevelFilterBar into CSS modules.
Follows project convention of CSS modules over inline styles.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update @cameleer/design-system to v0.1.38 (12px minimum font size).
Replace all 10px and 11px font sizes with 12px across 25 CSS modules
and 5 TSX inline styles to match the new DS floor.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Expose getDirectRolesForUser on RbacService interface so syncOidcRoles
compares against directly-assigned roles only, not group-inherited ones
- Remove early-return that preserved existing roles when OIDC returned
none — now always applies defaultRoles as fallback
- Update CLAUDE.md and SERVER-CAPABILITIES.md to reflect changes
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Logto returns opaque access tokens unless the resource parameter is
included in both the authorization request AND the token exchange.
Append resource to the token endpoint POST body per RFC 8707 so Logto
returns a JWT access token with Custom JWT claims.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backend holds client_secret and does the token exchange server-side,
making PKCE redundant. Removes code_verifier/code_challenge from all
frontend auth paths and backend exchange method. Eliminates the source
of "grant request is invalid" errors from verifier mismatches.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The OIDC login flow now reads roles from the access_token (JWT) in
addition to the id_token. This fixes role extraction with providers
like Logto that put scopes/roles in access tokens rather than id_tokens.
- Add audience and additionalScopes to OidcConfig for RFC 8707 resource
indicator support and configurable extra scopes
- OidcTokenExchanger decodes access_token with at+jwt-compatible processor,
falls back to id_token if access_token is opaque or has no roles
- syncOidcRoles preserves existing local roles when OIDC returns none
- SPA includes resource and additionalScopes in authorization requests
- Admin UI exposes new config fields
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
extractRoles() only handled List claims (JSON arrays). When rolesClaim
is configured as "scope", the JWT value is a space-delimited string,
which was silently returning [] and falling back to defaultRoles.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Logs received scopes, rolesClaim path, extracted roles, and all claim
keys at each stage of the OIDC auth flow to help debug Logto integration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Docker build copies package.json before source, so public/ doesn't
exist when npm ci runs postinstall. Use mkdirSync(recursive:true).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
favicon.svg is now copied from @cameleer/design-system/assets on
npm install via postinstall hook. Removed from git tracking
(.gitignore). Updates automatically when DS version changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fire end-session via fetch(no-cors) instead of window.location redirect.
Always navigate to /login?local regardless of whether end-session
succeeds, preventing broken JSON responses from blocking logout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace PNG favicons and brand logos with cameleer3-logo.svg from
@cameleer/design-system/assets. Favicon, login dialog, and sidebar
all use the same SVG. Remove PNG favicon files from public/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DS now exports ./assets/* — import PNGs directly via Vite instead of
copying to public/. Removes duplicated brand files from public/.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The @cameleer/design-system package.json exports field doesn't include
assets/, causing production build failures. Copy PNGs to public/ and
reference via basePath until DS adds asset exports.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add VITE_APP_VERSION build arg to UI Dockerfile, pass short SHA from
CI docker build step. vite.config.ts truncates to 7 chars so both
CI build and Docker build produce consistent short hashes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Import PNGs via Vite from @cameleer/design-system/assets instead of
copying to public/. Only favicons remain in public/ (needed by HTML).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The SVG uses fill=currentColor (inherits text color). Switch to the
full-color PNG brand icons: 192px for login dialog, 48px for sidebar.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace hand-crafted favicon.svg with official brand assets from
@cameleer/design-system v0.1.32: PNG favicons (16/32px) and
camel-logo.svg for login dialog and sidebar. Update SecurityConfig
public endpoints accordingly. Update documentation for architecture
cleanup (PKCE, OidcProviderHelper, role normalization, K8s hardening,
Dockerfile credential removal, CI deduplication, sidebar path fix).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The standard nginx image requires root to modify /etc/nginx/conf.d
and create /var/cache/nginx directories during startup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Extract OidcProviderHelper for shared discovery + JWK source construction
- Add SystemRole.normalizeScope() to centralize role normalization
- Merge duplicate claim extraction in OidcTokenExchanger
- Add PKCE (S256) to OIDC authorization flow (frontend + backend)
- Add SecurityContext (runAsNonRoot) to all K8s deployments
- Fix postgres probe to use $POSTGRES_USER instead of hardcoded username
- Remove default credentials from Dockerfile
- Extract sanitize_branch() to shared .gitea/sanitize-branch.sh
- Fix sidebar to use /exchanges/ paths directly, remove legacy redirects
- Centralize basePath computation in router.tsx via config module
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Logout now always redirects to /login?local, either via OIDC
end_session or as a direct fallback, preventing prompt=none
auto-redirect from logging the user back in immediately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When on Config tab: clicking an app navigates to /config/:appId (shows
that app's config with detail panel). Clicking a route navigates to
/config/:appId (same app config, since config is per-app not per-route).
Clicking Applications header navigates to /config (all apps table).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Config tab now always visible (not just when app selected). Shows all-
app config table at /config, single-app detail at /config/:appId.
Fixed 404 when clicking sidebar nodes while on Config tab — the sidebar
navigation built /config/appId/routeId which had no route. Now falls
back to exchanges tab for route-level navigation from config.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Hide Admin sidebar section for non-ADMIN users
- Add RequireAdmin route guard — /admin/* redirects to / for non-admin
- Move App Config from admin section to main Config tab (per-app,
visible when app selected). VIEWER sees read-only, OPERATOR+ can edit
- Hide diagram node toolbar for VIEWER (onNodeAction conditional)
- Add useIsAdmin/useCanControl helpers to centralize role checks
- Remove App Config from admin sidebar tree
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New input in the Claim Mapping section lets admins configure which
id_token claim is used as the unique user identifier (default: sub).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The OIDC user login ID is now configurable via the admin OIDC setup
dialog (userIdClaim field). Supports dot-separated claim paths (e.g.
'email', 'preferred_username', 'custom.user_id'). Defaults to 'sub'
for backwards compatibility. Throws if the configured claim is missing
from the id_token.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Roles from the id_token's rolesClaim are now diffed against stored
system roles on each OIDC login. Missing roles are added, revoked
roles are removed. Group memberships (manually assigned) are never
touched. This propagates scope revocations from the OIDC provider
on next user login.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
M2M scope mapping now accepts both 'server:admin' and 'admin' (case-
insensitive). OIDC user login role assignment strips the 'server:'
prefix before looking up SystemRole, so 'server:viewer' from the
id_token maps to VIEWER correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Without BASE_PATH the redirect fails behind a reverse proxy. Adding
?local prevents the SSO auto-redirect from immediately signing the
user back in after logout.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Logto signs id_tokens with ES384 by default. SecurityConfig already
included it but OidcTokenExchanger only had RS256 and ES256.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When prompt=none fails with consent_required (scopes not yet granted),
retry the OIDC flow without prompt=none so the user can grant consent
once. Uses sessionStorage flag to prevent infinite loops — falls back
to local login if the retry also fails.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When OIDC is configured, the login page automatically redirects to the
provider with prompt=none. If the user has an active OIDC session, they
are signed in without seeing a login page. If the provider returns
login_required (no session), falls back to the login form via ?local.
Users can bypass auto-redirect with /login?local.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Hardcoded /favicon.svg paths skip the <base> tag and fail when served
from a subpath like /server/. Now uses config.basePath in TSX and a
relative href in index.html so the <base> tag resolves correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Behind a reverse proxy with strip-prefix (e.g., Traefik at /server/),
the OIDC redirect_uri must include the prefix so the callback routes
back through the proxy. Now uses config.basePath (from <base href>)
instead of hardcoding '/'.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OidcTokenExchanger fetched the discovery document from the issuerUri
as-is, but the database stores the issuer URI (e.g. /oidc), not the
full discovery URL. Logto returns 404 for the bare issuer path.
SecurityConfig already appended the well-known suffix — now the token
exchanger does the same.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OidcTokenExchanger cached securityProperties.isOidcTlsSkipVerify() in
the constructor as a boolean field. If Spring constructed the bean
before property binding completed, the cached value was false even when
the env var was set. SecurityConfig worked because it read the property
at call time. Now OidcTokenExchanger stores the SecurityProperties
reference and reads the flag on each call, matching SecurityConfig's
pattern.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Java's automatic redirect following creates new connections that do NOT
inherit custom SSLSocketFactory/HostnameVerifier. This caused the OIDC
discovery fetch to fail on redirect even with TLS_SKIP_VERIFY=true.
Now disables auto-redirect and follows manually with SSL on each hop.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Behind a reverse proxy the browser sends Origin matching the proxy's
public URL, which the single-origin CAMELEER_UI_ORIGIN rejects.
New env var accepts comma-separated origins and takes priority over
UI_ORIGIN, which remains as a backwards-compatible fallback.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Self-signed CA certs on the OIDC provider (e.g. Logto behind a reverse
proxy) cause the login flow to fail because Java's truststore rejects
the connection. This adds an opt-in env var that creates a trust-all
SSLContext scoped to OIDC HTTP calls only (discovery, token exchange,
JWKS fetch) without affecting system-wide TLS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
config.apiBaseUrl now derives from <base> tag when no explicit config
is set (e.g., /server/api/v1 instead of /api/v1). commands.ts authFetch
prepends apiBaseUrl and uses relative paths.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The second sed matched the just-injected <base href="/server/"> and
rewrote it to <base href="/server/server/">. Since Vite builds with
base: './' (relative paths), the <base> tag alone is sufficient.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When BASE_PATH is set (e.g., /server/), the entrypoint script injects
a <base> tag and rewrites asset paths in index.html. React Router reads
the basename from the <base> tag. Vite builds with relative paths.
Default / for standalone mode (no changes).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When set, fetches JWKs from this URL directly instead of discovering
from the OIDC well-known endpoint. Needed when the public issuer URL
(e.g., https://domain.com/oidc) isn't reachable from inside containers
but the internal URL (http://logto:3001/oidc/jwks) is.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
pg_isready without -U defaults to OS user "root" which doesn't exist
as a PostgreSQL role, causing noisy log entries.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
K8s $(VAR) substitution only resolves env vars defined earlier in the
list. PG_USER and PG_PASSWORD must come before DB_URL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
LOGTO_ENDPOINT and LOGTO_ADMIN_ENDPOINT are public-facing URLs that
Logto uses for OIDC discovery, issuer URI, and redirects. When behind
a reverse proxy (e.g., Traefik), set these to the external URLs.
Logto requires its own subdomain (not a path prefix).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Explicit spring.datasource.url in YAML takes precedence over the env var,
causing deployed containers to connect to localhost instead of the postgres
service. Now the YAML uses ${SPRING_DATASOURCE_URL:...} so the env var
wins when set. Flyway inherits from the datasource (no separate URL).
Removed CAMELEER_DB_SCHEMA — schema is part of the datasource URL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Existing deployment has tables in public schema. The new tenant_default
default breaks startup because Flyway sees an empty schema. Override to
public for backward compat; new deployments use the tenant-derived default.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Schema now defaults to tenant_${cameleer.tenant.id} (e.g. tenant_default,
tenant_acme) instead of public. Flyway create-schemas: true ensures the
schema is auto-created on first startup. CAMELEER_DB_SCHEMA env var still
available as override for feature branch isolation. Removed hardcoded
public schema from K8s base and main overlay.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Comprehensive standalone document covering API surface, agent protocol,
security, storage, multi-tenancy, deployment, and configuration — designed
for external systems (like the SaaS orchestration layer) that need to
understand and manage Cameleer3 Server instances.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Environment selector was losing its value on navigation because URL search
params were silently dropped by navigate() calls. Moved to a Zustand store
with localStorage persistence so the selection survives navigation, page
refresh, and new tabs. Switching environment now resets all filters, clears
URL params, invalidates queries, and remounts pages via Outlet key. Also
syncs openapi.json schema with running backend.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
HeartbeatRequest now carries environmentId (cameleer3-common update).
Auto-heal prefers the heartbeat value (most current) over the JWT
claim, ensuring agents recover their correct environment immediately
on the first heartbeat after server restart.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add 'env' claim to agent JWTs (set at registration, carried through
refresh). Auto-heal on heartbeat/SSE now reads environment from the
JWT instead of hardcoding 'default', so agents retain their correct
environment after server restart.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
DS v0.1.31 changes .env wrapper to neutral button style matching
other TopBar controls. Simplified selector CSS to inherit all
font/color properties from the wrapper.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Make the select transparent (no border, no background) so it
inherits the DS .env pill styling (success-colored badge with
mono font). Negative margins compensate for the pill padding.
Dropdown chevron uses currentColor to match the pill text.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use unfiltered agent query to discover environments (avoids circular
filter). Always show selector even with single environment so it's
visible as a label. Default to ['default'] when no agents connected.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Update @cameleer/design-system to v0.1.30 which accepts ReactNode
for the environment prop. Move EnvironmentSelector from standalone
div into TopBar, rendering between theme toggle and user menu.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Backend: Added optional `environment` query parameter to catalog,
search, stats, timeseries, punchcard, top-errors, logs, and agents
endpoints. ClickHouse queries filter by environment when specified
(literal SQL for AggregatingMergeTree, ? binds for raw tables).
StatsStore interface methods all accept environment parameter.
UI: Added EnvironmentSelector component (compact native select).
LayoutShell extracts distinct environments from agent data and
passes selected environment to catalog and agent queries via URL
search param (?env=). TopBar shows current environment label.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds configurable tenant ID (CAMELEER_TENANT_ID env var, default:
"default") and environment as a first-class concept. Each server
instance serves one tenant with multiple environments.
Changes across 36 files:
- TenantProperties config bean for tenant ID injection
- AgentInfo: added environmentId field
- AgentRegistrationRequest: added environmentId field
- All 9 ClickHouse stores: inject tenant ID, replace hardcoded
"default" constant, add environment to writes/reads
- ChunkAccumulator: configurable tenant ID + environment resolver
- MergedExecution/ProcessorBatch/BufferedLogEntry: added environment
- ClickHouse init.sql: added environment column to all tables,
updated ORDER BY (tenant→time→env→app), added tenant_id to
usage_events, updated all MV GROUP BY clauses
- Controllers: pass environmentId through registration/auto-heal
- K8s deploy: added CAMELEER_TENANT_ID env var
- All tests updated for new signatures
Closes#123
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When no exchange is selected, the topology-only diagram now shows
the RouteControlBar above it (if the agent supports routeControl
or replay and the user has OPERATOR/ADMIN role). This fixes a gap
where suspended routes with no recent exchanges had no way to be
resumed from the UI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The heartbeat now carries capabilities (per protocol v2 update).
On each heartbeat, capabilities are updated in the agent registry.
On auto-heal (server restart), capabilities from the heartbeat
are used instead of empty Map.of(), so the agent's feature flags
(replay, routeControl, logForwarding, etc.) are restored
immediately on the first heartbeat.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Convert ClickHouseUsageTracker and ClickHouseMetricsQueryStore to
use JDBC parameterized queries (? binds) — these query raw tables
without AggregateFunction columns.
Fix lit(String) in RouteMetricsController and ClickHouseStatsStore
to escape backslashes before single quotes. Without this, an input
like \' breaks out of the string literal in ClickHouse (where \
is an escaped backslash). These must remain as literal SQL because
the ClickHouse JDBC 0.9.x driver wraps PreparedStatement in
sub-queries that strip AggregateFunction types, breaking -Merge
combinators.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Running mvn sonar:sonar as a separate invocation skips child
modules. Combining verify and sonar:sonar in a single mvn
command ensures the reactor processes all modules.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Maven sonar plugin auto-detects sources and tests from the POM
module structure. Passing sonar.sources as CLI args caused path
doubling (module-dir/module-dir/src) in multi-module projects.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The standalone sonar-scanner CLI has Java discovery issues in the
build container. Switch to the Maven sonar plugin (same approach
as cameleer3 agent repo), which uses Maven's own JDK. This also
removes the sonar-scanner download/install step entirely.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonar-scanner 6.x checks SONAR_SCANNER_JAVA_HOME, not JAVA_HOME.
Despite JAVA_HOME being correct and java being on PATH, the scanner
uses its own env var for Java discovery.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
java binary may not be on PATH directly in the build container.
Derive JAVA_HOME from the jar binary location (which we know works)
and prepend JAVA_HOME/bin to PATH so sonar-scanner can find java.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
sonar-scanner 6.x requires JAVA_HOME or java on PATH. The build
container has Java installed but doesn't export JAVA_HOME, so
derive it from the java binary location.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
jar xf doesn't preserve Unix file permissions from zip entries,
so the sonar-scanner binary lacks the execute bit.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diagnostics showed ~3,200 tiny inserts per 5 minutes:
- processor_executions: 2,376 inserts (14 rows avg) — one per chunk
- logs: 803 inserts (5 rows avg) — synchronous in HTTP handler
Fix 1: Consolidate processor inserts — new insertProcessorBatches() method
flattens all ProcessorBatch records into a single INSERT per flush cycle.
Fix 2: Buffer log inserts — route through WriteBuffer<BufferedLogEntry>,
flushed on the same 5s interval as executions. LogIngestionController now
pushes to buffer instead of inserting directly.
Also reverts async_insert config (doesn't work with JDBC inline VALUES).
Expected: ~3,200 inserts/5min → ~160 (20x reduction in part creation,
MV triggers, and background merge work).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Diagnostics showed 3,200 tiny inserts per 5 minutes (processor_executions:
2,376 at 14 rows avg, logs: 803 at 5 rows avg), each creating a new part
and triggering MV aggregations + background merges. This was the root cause
of ~400m CPU usage at 3 tx/s.
async_insert=1 with 5s busy timeout lets ClickHouse buffer incoming inserts
and consolidate them into fewer, larger parts before writing to disk.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Increase ingestion flush interval from 500ms to 5000ms to reduce MV merge storms
- Reduce ClickHouse background_schedule_pool_size from 8 to 4
- Rename LIVE/PAUSED badge labels to AUTO/MANUAL across all pages
- Update design system to v0.1.29
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ChunkAccumulator now extracts inputBody/outputBody/inputHeaders/outputHeaders
from ExecutionChunk.inputSnapshot/outputSnapshot instead of storing empty strings
- Set ClickHouse server log level to warning (was trace by default)
- Update CLAUDE.md to document Ed25519 key derivation
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace DB-persisted keypair with deterministic derivation from
CAMELEER_JWT_SECRET via HMAC-SHA256 seed + seeded SHA1PRNG KeyPairGenerator.
Same secret = same key pair across restarts, no private key in the database.
Closes#121
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The keypair was generated ephemerally on each startup, causing agents
to reject all commands after a server restart (signature mismatch).
Now persisted to PostgreSQL server_config table and restored on startup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Merge all V1-V11 migration scripts into one idempotent init.sql
- Simplify ClickHouseSchemaInitializer to load single file
- Replace route_diagrams projection with in-memory caches:
hashCache (routeId+instanceId → contentHash) warm-loaded on startup,
graphCache (contentHash → RouteGraph) lazy-populated on access
- Eliminates 9M+ row scans on diagram lookups
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse 24.12 requires this setting before adding projections to
ReplacingMergeTree tables. Using 'drop' mode which discards the projection
during deduplication merges and rebuilds it afterward.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Alphabetical sort put V10/V11 before V2-V9 ("V11" < "V1_" in ASCII),
causing the route_diagrams projection to run before the table existed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Set com.clickhouse log level to INFO and org.apache.hc.client5 to WARN
- Admin hooks (useUsers/useGroups/useRoles) now only fetch on admin pages,
eliminating AUDIT view_users entries on every UI click
- Add ClickHouse projection on route_diagrams for (tenant_id, route_id,
instance_id, created_at) to avoid full table scans on diagram lookups
- Bump @cameleer/design-system to v0.1.28 (PAUSED mode time range fix,
refreshTimeRange API)
- Call refreshTimeRange before invalidateQueries in PAUSED mode manual
refresh so sidebar clicks use current time window
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
On first click, Dashboard was in non-split mode. The click set
selectedId locally then triggered split view, which remounted
Dashboard — losing the selectedId state.
Added activeExchangeId prop passed from ExchangesPage so the
selection survives the remount. Also syncs via useEffect when
parent changes selection (e.g. correlated exchange navigation).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
useLiveQuery returned enabled:false when paused, which prevented
queries from running at all. Changed to enabled:true always —
PAUSED now means "fetch once, no polling" instead of "don't fetch".
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
After server restart, auto-healed agents register with empty
routeIds. The catalog only looked at agent registry for routes,
so routes and counts disappeared.
Now merges route IDs from ClickHouse stats_1m_route into the
catalog. Also includes apps that only exist in ClickHouse data
(no agent currently registered). Routes and exchange counts
survive server restarts.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When autoRefresh is disabled, sidebar clicks now invalidate all
queries (queryClient.invalidateQueries()), triggering a re-fetch.
This gives users "click to refresh" behavior instead of stale data.
When LIVE mode is on, queries already poll at intervals, so no
invalidation is needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When the Applications section is already expanded, clicking the
header now navigates to /{tab} (all applications) instead of
collapsing. When collapsed, clicking expands as before.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three related issues caused by in-memory agent registry being empty
after server restart:
1. JwtAuthenticationFilter rejected valid agent JWTs if agent wasn't
in registry — now authenticates any valid JWT regardless
2. Heartbeat returned 404 for unknown agents — now auto-registers
the agent from JWT claims (subject, application)
3. SSE endpoint returned 404 — same auto-registration fix
JWT validation result is stored as a request attribute so downstream
controllers can extract the application claim for auto-registration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The refresh endpoint required the agent to exist in the in-memory
registry. After server restart the registry is empty, so all refresh
attempts got 404. The refresh token itself is self-contained with
subject, application, and roles — the registry lookup is optional.
Now uses application from the JWT, falling back to registry only
if the agent happens to be registered.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The system.processes query was returning its own row. Added
filter: query NOT LIKE '%system.processes%'
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AdminLayout was a plain div with padding but no scroll. The parent
<main> has overflow:hidden, so admin page content beyond viewport
height was clipped. Added flex:1, overflow:auto, minHeight:0 to
make AdminLayout a proper scroll container.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
UsageFlushScheduler was a @Component with @ConditionalOnBean, but
ClickHouseUsageTracker is created via @Bean — component scan runs
first, so the condition always evaluated false. Events accumulated
in the WriteBuffer but flush() was never called.
Moved scheduler to @Bean in StorageBeanConfig with the same
@ConditionalOnProperty guard as the tracker.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Icons now reflect event type (UserPlus for registration, Skull
for dead, HeartPulse for recovery, Route for state changes, etc.)
while severity still drives the color. Updated in both
AgentInstance and AgentHealth pages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Disabled buttons now show reduced opacity (0.35) and muted icon
color instead of just changing the cursor.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Buttons are disabled based on route state: Started disables
Start/Resume, Stopped disables Stop/Suspend/Resume, Suspended
disables Start/Suspend. State looked up from catalog API.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Version injected at build time via VITE_APP_VERSION env var.
CI sets it to branch@sha. Falls back to 'dev' in local dev.
Displayed next to "Cameleer" in the sidebar header.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
collectStarredItems used 'app:' prefix for route keys but
buildAppTreeNodes uses 'route:' prefix. Routes were starred
but never matched in the starred section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Admin section stays in its fixed position (after Starred, before
Footer). Entering admin mode collapses Applications and Starred
but does not reorder sections.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Remove Agents and Routes sections from sidebar. Layout is now:
Header (camel logo + Cameleer) → Search → Applications section →
Starred section (when items exist) → Footer (Admin + API Docs).
Admin accordion: clicking Admin navigates to /admin/rbac and
expands Admin section at top while collapsing Applications and
Starred. Clicking Applications exits admin mode.
Removed buildAgentTreeNodes and buildRouteTreeNodes from
sidebar-utils (no longer needed).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Pool was hardcoded to 10 connections serving 7 concurrent write
streams + UI reads, causing "too many simultaneous queries" and
WriteBuffer overflow. Pool now defaults to 50 (configurable via
clickhouse.pool-size), flush interval reduced from 1000ms to 500ms.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace local HeartbeatRequest DTO with the shared model from
cameleer3-common. Message types exchanged between server and agent
belong in the common module.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace ACK-based route state inference with agent-reported state.
Heartbeats now carry optional routeStates map, and ROUTE_STATE_CHANGED
events update the registry immediately.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Defines two backward-compatible mechanisms for accurate route state
tracking: heartbeat extension (routeStates map in heartbeat body)
and ROUTE_STATE_CHANGED events for real-time updates. Covers
agent-side detection via Camel EventNotifier, server-side handling,
multi-agent conflict resolution, and migration path.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add routeState field to RouteSummary DTO (null for started, 'stopped'
or 'suspended' for non-default states). Sidebar shows stop/pause icons
and state badge for affected routes in both Apps and Routes sections.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
In-memory registry that infers route state (started/stopped/suspended)
from successful route-control command ACKs. Updates state only when all
agents in a group confirm success.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add CommandGroupResponse and ConfigUpdateResponse types. Switch
useSendGroupCommand and useSendRouteCommand from openapi-fetch to authFetch
returning CommandGroupResponse. Update useUpdateApplicationConfig to return
ConfigUpdateResponse and fix all consumer onSuccess callbacks to access
saved.config.version instead of saved.version.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add addGroupCommandWithReplies() to AgentRegistryService that sends commands
to all LIVE agents in a group and returns CompletableFuture per agent for
collecting replies. Update sendGroupCommand() and pushConfigToAgents() to
wait with a shared 10-second deadline, returning CommandGroupResponse with
per-agent status, timeouts, and overall success. Config update endpoint now
returns ConfigUpdateResponse wrapping both the saved config and push result.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Stop and suspend route commands now show a ConfirmDialog requiring
typed confirmation before dispatch. Start and resume execute
immediately without confirmation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Direct navigation to /admin/* now correctly opens Admin section
and collapses operational sections on first render. Previously
the accordion effect only triggered on route transitions.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Review feedback: buildRouteTreeNodes was defined but never rendered.
Added Routes section between Agents and Admin. Removed duplicate
padding on admin pages (AdminLayout handles its own padding).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Search: DS renders dumb input, app owns filterQuery state and
passes it to each SidebarTree. Icon-rail click: fires both
onCollapseToggle and onToggle simultaneously, no navigation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replaces the previous "hide sidebar on admin" approach with a
composable compound component design. DS provides shell + building
blocks (Sidebar, Section, Footer, SidebarTree); consuming app
controls all content, section ordering, accordion behavior, and
icon-rail collapse.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Review feedback: breadcrumb memo had an unused isAdminPage branch
(TopBar no longer renders on admin pages). Added aria-label to
icon-only logout button for screen readers.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
AdminLayout gains a self-contained header (Back / Admin / user+logout)
with CSS module styles, replacing the inline padding wrapper. Admin
pages now render fully without the main app chrome.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Pass null as sidebar prop, guard TopBar and CommandPalette with
!isAdminPage, and remove conditional admin padding from main element.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
LIKE is case-sensitive in ClickHouse. Switch to ILIKE for message,
stack_trace, and logger_name searches so queries match regardless
of casing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Recursive case-insensitive highlighting of the search query in
collapsed message, expanded full message, and stack trace. Uses the
project's amber accent color for the highlight mark.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Use attributeBadgeColor() (hash-based) instead of "auto" so the same
application name gets the same badge color across all pages.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse 24.12 new query analyzer resolves countMerge(total_count)
in the CASE WHEN to the SELECT alias (UInt64) instead of the original
AggregateFunction column when the alias has the same name. Renamed
aliases to tc/fc to avoid the collision.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse rejects countMerge() in ORDER BY after GROUP BY because the
column is already finalized to UInt64. Use the SELECT alias instead.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The build container lacks unzip. The JDK jar command handles zip
extraction natively.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The table and materialized view were missing the processor_type column,
causing the RouteMetricsController query to fail and the dashboard
processor metrics table to render empty.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add GET /search/attributes/keys endpoint that queries distinct
attribute key names from ClickHouse using JSONExtractKeys. Attribute
keys appear in the cmd-k Attributes tab alongside attribute value
matches from exchange results.
- SearchIndex.distinctAttributeKeys() interface method
- ClickHouseSearchIndex implementation using arrayJoin(JSONExtractKeys)
- SearchController /attributes/keys endpoint
- useAttributeKeys() React Query hook
- buildSearchData includes attribute keys as 'attribute' category items
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Routes with the same name across different applications (e.g., "route1"
in both QUARKUS-APP and BACKEND-APP) were deduplicated because they
shared the same id (routeId). Use appId/routeId as the id so all
routes appear in cmd-k results.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The searchData useMemo recomputed on every catalog poll cycle because
catalogData got a new array reference even when content was unchanged.
This caused the CommandPalette list to re-render and reset scroll.
Use a ref with deep equality check to keep a stable catalog reference,
only updating when the actual data changes.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The command palette renders matchContext via dangerouslySetInnerHTML
expecting HTML with <mark> tags, but extractSnippet() returned plain
text. Wrap the matched term in <mark> tags and escape surrounding
text to prevent XSS.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ID fields (execution_id, correlation_id, exchange_id) should use
exact equality, not LIKE with wildcards. LIKE is only needed for
the _search_text full-text columns.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The _search_text materialized column only contained error messages,
bodies, and headers — not execution_id, correlation_id, exchange_id,
or route_id. Searching by ID via cmd-k returned no results.
- Add ID fields to _search_text in ClickHouse DDL (covered by ngram
bloom filter index)
- Add direct LIKE matches on execution_id, correlation_id, exchange_id
in the text search WHERE clause for faster exact ID lookups
Requires ClickHouse table recreation (fresh install).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sidebar selectedPath now uses sidebarReveal on all tabs, not just
exchanges. This fixes sidebar highlighting on dashboard and runtime.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The chunked ingestion path hardcoded hasTraceData=false because the
execution envelope doesn't carry processor bodies. But the processor
records DO have inputBody/outputBody — we just need to check them.
Track hasTraceData across chunks in PendingExchange and pass it to
MergedExecution when the final chunk arrives or on stale sweep.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The identity rename (application→applicationId) broke search filtering
because the stale schema.d.ts still had 'application' as the field name.
The backend silently ignored the unknown field, returning unfiltered results.
- Regenerate openapi.json and schema.d.ts from live backend
- Fix Dashboard: application→applicationId in search request
- Fix RouteDetail: application→applicationId in search request (2 places)
- LayoutShell: scope command palette search by appId/routeId
- LayoutShell: pass sidebarReveal state on sidebar click navigation
Note for DS team: the Sidebar selectedPath logic (line 5451 in dist)
has a hardcoded pathname.startsWith("/exchanges/") guard. This should
be broadened to simply `S ? S : $.pathname` so sidebarReveal works on
all tabs (dashboard, runtime), not just exchanges.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Two fixes:
- Pass sidebarReveal state on sidebar navigation so the design system
can highlight the selected entry (it compares internal /apps/... paths
against this state value, not the browser URL)
- Command palette search now includes scope.appId and scope.routeId
so results are filtered to the current sidebar selection
Note: sidebar highlighting works on the exchanges tab. The design
system's selectedPath logic only checks pathname.startsWith("/exchanges/")
for sidebarReveal — a DS update is needed to support /dashboard/ and
/runtime/ tabs too.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The AgentInstanceResponse backend DTO uses instanceId, displayName,
applicationId, status — but the stale schema.d.ts still had id, name,
application, state. This caused the runtime table to show no data.
- Update schema.d.ts AgentInstanceResponse fields
- Fix AgentHealth: row.id→instanceId, row.name→displayName,
row.application→applicationId, inst.id→instanceId
- Fix AgentInstance: agent.id→instanceId, agent.name→displayName
- Fix ExchangeHeader: agent.id→instanceId, agent.state→status
- Fix LayoutShell search: agent.state→status, agentTps→tps
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Rewrite ClickHouse admin to show useful storage metrics instead of
often-empty system.events data. Add active queries section.
- Replace performance endpoint: query system.parts for disk size,
uncompressed size, compression ratio, total rows, part count
- Add /queries endpoint querying system.processes for active queries
- Frontend: storage overview strip, tables with total size, active
queries DataTable
- Fix AgentHealth.tsx type: agentId → instanceId in inline type cast
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Complete the ClickHouse migration by removing all PostgreSQL analytics
code. PostgreSQL now serves only RBAC, config, and audit — all
observability data is exclusively in ClickHouse.
- Delete 6 dead PostgreSQL store classes (executions, stats, diagrams,
events, metrics, metrics-query) and 2 integration tests
- Delete RetentionScheduler (ClickHouse TTL handles retention)
- Remove all 7 cameleer.storage.* feature flags from application.yml
- Remove all @ConditionalOnProperty from ClickHouse beans in StorageBeanConfig
- Consolidate 14 Flyway migrations (V1-V14) into single clean V1 with
only RBAC/config/audit tables (no TimescaleDB, no analytics tables)
- Switch from timescale/timescaledb-ha:pg16 to postgres:16 everywhere
(docker-compose, deploy/postgres.yaml, test containers)
- Remove TimescaleDB check and /metrics-pipeline from DatabaseAdminController
- Set clickhouse.enabled default to true
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Applications, routes within each app, and agents within each app
are now sorted by name using localeCompare.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The backend identity rename (applicationName → applicationId,
agentId → instanceId) was not reflected in the frontend. This caused
drilldown to fail (detail.applicationName was undefined, disabling
the diagram fetch) and various display issues.
Updated schema.d.ts, ExchangeHeader, ExecutionDiagram, Dashboard,
AgentHealth, AgentInstance, LayoutShell, LogTab, InfoTab, DetailPanel,
ExchangesPage, and tracing-store.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tracks authenticated UI user requests to understand usage patterns:
- New ClickHouse usage_events table with 90-day TTL
- UsageTrackingInterceptor captures method, path, duration, user
- Path normalization groups dynamic segments ({id}, {hash})
- Buffered writes via WriteBuffer + periodic flush
- Admin endpoint GET /api/v1/admin/usage with groupBy=endpoint|user|hour
- Skips agent requests, health checks, and data ingestion
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Path containers (EIP_WHEN, EIP_OTHERWISE, etc.) don't have their own
processor records, so they never get an overlay entry. Now inferred
from descendants: green if any descendant executed, red if any failed.
Gated (amber) only when no descendants executed at all.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
A container is only gated (amber) when filterMatched=false or
duplicateMessage=true AND no descendants were executed. Containers
with executed children (split, choice, idempotent that passed) now
correctly show green/red based on their execution status.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
CompoundNode now uses execution overlay status to color its header:
failed (red) > completed (green) > default. Previously only used
static type-based color regardless of execution state.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace synthetic wrapper node approach with direct iteration fields:
- ProcessorNode gains iteration (child's index) and iterationSize
(container's total) fields, populated from ClickHouse flat records
- Frontend hooks detect iteration containers from iterationSize != null
instead of scanning for wrapper processorTypes
- useExecutionOverlay filters children by iteration field instead of
wrapper nodes, eliminating ITERATION_WRAPPER_TYPES entirely
- Cleaner data contract: API returns exactly what the DB stores
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RouteCatalogController, RouteMetricsController, and AgentRegistrationController
had unqualified JdbcTemplate injection, receiving the PostgreSQL template
instead of ClickHouse. The stats queries silently failed (caught exception)
returning 0 counts. Added @Qualifier("clickHouseJdbcTemplate") to all three.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sets TZ=UTC and -Duser.timezone=UTC to guarantee all JVM time operations
use UTC regardless of the container's base image or host configuration.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Timestamp.toString() uses JVM local timezone which can mismatch with
ClickHouse's UTC timezone, causing time-filtered queries to return empty
results. Replaced with DateTimeFormatter.withZone(UTC) in all lit() methods.
Also added warn logging to RouteCatalogController catch blocks to surface
query errors instead of silently swallowing them.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse can't rename columns that are part of ORDER BY keys.
Updated V1-V8 DDL files directly with new column names (instance_id,
application_id) and removed V9 migration. Wipe ClickHouse and restart.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Align all internal naming with the agent team's protocol v2 identity rename:
- agentId → instanceId (unique per-JVM identifier)
- applicationName → applicationId (shared app identifier)
- AgentInfo: id → instanceId, name → displayName, application → applicationId
Add SHUTDOWN lifecycle state for graceful agent shutdowns:
- New POST /data/events endpoint receives agent lifecycle events
- AGENT_STOPPED event transitions agent to SHUTDOWN (skips STALE/DEAD)
- New POST /{id}/deregister endpoint removes agent from registry
- Server now distinguishes graceful shutdown from crash (heartbeat timeout)
Includes ClickHouse V9 and PostgreSQL V14 migrations for column renames.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ElkDiagramRenderer.getElkRoot(): add null guard to prevent NPE
when node is null (SQ java:S2259)
- WriteBuffer: add offerOrWarn() that logs when buffer is full instead
of silently dropping data. ChunkAccumulator now uses this method
so ingestion backpressure is visible in logs (SQ java:S899)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ChunkAccumulator now injects DiagramStore and looks up the content hash
when converting to MergedExecution. Without this, the detail page had
no diagram hash, so the overlay couldn't find the route diagram.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RouteCatalogController, RouteMetricsController, AgentRegistrationController
all had inline SQL using SUM() on AggregateFunction columns from stats_1m_*
AggregatingMergeTree tables. Replace with countMerge/countIfMerge/sumMerge.
Also fix time_bucket() → toStartOfInterval() and ::double → toFloat64().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES=false (required
by PROTOCOL.md) and explicit TypeReference<List<ExecutionChunk>> for
array parsing. Without this, batched chunks from ChunkedExporter
(2+ chunks in a JSON array) were silently rejected, causing final:true
chunks to be lost and all exchanges to go stale.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouseExecutionStore implements ExecutionStore, so the concrete bean
already satisfies the interface — remove redundant wrapper bean. Align
ChunkAccumulator and ExecutionFlushScheduler conditions to
cameleer.storage.executions flag.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add cameleer.storage.executions feature flag (default: clickhouse).
PostgresExecutionStore activates only when explicitly set to postgres.
Add by-seq snapshot endpoint for iteration-aware processor lookup.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dual-mode buildTree: detects seq presence and uses seq/parentSeq linkage
instead of processorId map. Handles duplicate processorIds across
iterations correctly. Old processorId-based mode kept for PG compat.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements ExecutionStore interface with findById (FINAL for
ReplacingMergeTree), findProcessors (ORDER BY seq), findProcessorById,
and findProcessorBySeq. Write methods unchanged.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add conditional beans for ClickHouseDiagramStore, ClickHouseAgentEventRepository,
and ClickHouseLogStore. All default to ClickHouse (matchIfMissing=true).
PG/OS stores activate only when explicitly configured.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract LogIndex interface from OpenSearchLogIndex. Both ClickHouseLogStore
and OpenSearchLogIndex implement it. Controllers now inject LogIndex.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace partial memory config with full Altinity low-memory guide
settings. Revert container limit from 6Gi back to 4Gi — proper
tuning (mlock=false, reduced caches/pools/threads, disk spill for
aggregations) makes the original budget sufficient.
Switch all storage feature flags to ClickHouse:
- CAMELEER_STORAGE_SEARCH: opensearch → clickhouse
- CAMELEER_STORAGE_METRICS: postgres → clickhouse
- CAMELEER_STORAGE_STATS: already clickhouse
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- ChunkIngestionController: /data/chunks → /data/executions (matches
PROTOCOL.md endpoint the agent actually posts to)
- ExecutionController: conditional on ClickHouse being disabled to
avoid mapping conflict
- Persist originalExchangeId and replayExchangeId from ExecutionChunk
envelope through to ClickHouse (was silently dropped)
- V5 migration adds the two new columns to executions table
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
cameleer3-common removed children, loopIndex, splitIndex,
multicastIndex from ProcessorExecution (flat model only now).
Iteration context lives on synthetic wrapper nodes via processorType.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse 24.12 auto-sizes caches from the cgroup limit, leaving
insufficient headroom for MV processing and background merges.
Adds a custom config that shrinks mark/index/expression caches and
caps per-query memory at 2 GiB. Bumps container limit 4Gi → 6Gi.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The V4 DDL had a semicolon inside a comment which caused the
split-on-semicolon logic to produce a comment-only segment that
ClickHouse rejected as empty query. Fixed the comment and made
the initializer strip comment-only segments before execution.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements StatsStore interface for ClickHouse using AggregatingMergeTree
tables with -Merge combinators (countMerge, countIfMerge, sumMerge,
quantileMerge). Uses literal SQL for aggregate table queries to avoid
ClickHouse JDBC driver PreparedStatement issues with AggregateFunction
columns. Raw table queries (SLA, topErrors, activeErrorTypes) use normal
prepared statements.
Includes 13 integration tests covering stats, timeseries, grouped
timeseries, SLA compliance, SLA counts by app/route, top errors, active
error types, punchcard, and processor stats. Also fixes AggregateFunction
type signatures in V4 DDL (count() takes no args, countIf takes UInt8).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The jdbcTemplate() method was calling dataSource(properties) directly,
creating a new DataSource instance instead of using the Spring-managed
@Primary bean. This caused some repositories to receive the ClickHouse
connection instead of PostgreSQL.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- DDL for executions (ReplacingMergeTree) and processor_executions (MergeTree with seq/parentSeq/iteration)
- ClickHouseExecutionStore with batch INSERT for both tables
- ChunkAccumulator: buffers exchange envelope across chunks, inserts processors immediately, writes execution on final chunk
- ExecutionFlushScheduler drains WriteBuffers to ClickHouse
- ChunkIngestionController: POST /api/v1/data/chunks endpoint
- ClickHouseSearchIndex: ngram-accelerated SQL search implementing SearchIndex interface
- Feature flags: cameleer.storage.search=opensearch|clickhouse
- Uses cameleer3-common ExecutionChunk and FlatProcessorRecord models
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ExecutionFlushScheduler drains MergedExecution and ProcessorBatch write
buffers on a fixed interval and delegates batch inserts to
ClickHouseExecutionStore. Also sweeps stale exchanges every 60s.
ChunkIngestionController exposes POST /api/v1/data/chunks, accepts
single or array ExecutionChunk payloads, and feeds them into the
ChunkAccumulator. Conditional on ChunkAccumulator bean (clickhouse.enabled).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When clickhouse.enabled=true, the ClickHouse JdbcTemplate bean prevents
Spring Boot auto-config from creating the default PG JdbcTemplate.
All PG repositories then get the CH JdbcTemplate and fail with
"Table cameleer.audit_log does not exist".
Fix: explicitly create @Primary DataSource and JdbcTemplate from
DataSourceProperties so PG remains the default for unqualified injections.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
clickhouse-jdbc 0.9.7 rejects async_insert and wait_for_async_insert as
unknown URL parameters. These are server-side settings, not driver config.
Can be set per-query later if needed via custom_settings.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Set CLICKHOUSE_USER/PASSWORD via k8s secret (fixes "disabling network
access for user 'default'" when no password is set)
- Add clickhouse-credentials secret to CI deploy + feature branch copy
- Pass CLICKHOUSE_USERNAME/PASSWORD env vars to server pod
- Make schema initializer non-fatal so server starts even if CH is
temporarily unavailable
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The deploy/clickhouse.yaml manifest was created but not referenced
in the CI workflow. Add kubectl apply between OpenSearch and Authentik.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse only has the 'default' database out of the box. The JDBC URL
connects to 'cameleer', so the database must exist before the server starts.
Uses /docker-entrypoint-initdb.d/ init script via ConfigMap.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Testcontainers tests need Docker which isn't available in CI.
Rename to *IT so Surefire skips them (Failsafe runs them with -DskipITs=false).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Implements MetricsQueryStore using ClickHouse toStartOfInterval() for
time-bucketed aggregation queries; verified with 4 Testcontainers tests.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
TDD implementation of MetricsStore backed by ClickHouse. Uses native
Map(String,String) column type (no JSON cast), relies on ClickHouse
DEFAULT for server_received_at, and handles null tags by substituting
an empty HashMap. All 4 Testcontainers tests pass.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds ClickHouseSchemaInitializer that runs on ApplicationReadyEvent,
scanning classpath:clickhouse/*.sql in filename order and executing each
statement. Adds V1__agent_metrics.sql with MergeTree table, tenant/agent
partitioning, and 365-day TTL.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds ClickHouseProperties (bound to clickhouse.*), ClickHouseConfig
(conditional HikariDataSource + JdbcTemplate beans), and extends
application.yml with clickhouse.enabled/url/username/password and
cameleer.storage.metrics properties.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
User-facing data and config endpoints live under `/api/v1/environments/{envSlug}/...`. Env is a path segment, never a query param. The `envSlug` is resolved to an `Environment` bean via the `@EnvPath` argument resolver (`web/EnvironmentPathResolver.java`) — 404 on unknown slug.
**Slugs are immutable after creation** for both environments and apps. Slug regex: `^[a-z0-9][a-z0-9-]{0,63}$`. Validated in `EnvironmentService.create` and `AppService.createApp`. Update endpoints (`PUT`) do not accept a slug field; Jackson drops it as an unknown property.
### Flat-endpoint allow-list
These paths intentionally stay flat (no `/environments/{envSlug}` prefix). Every new endpoint should be env-scoped unless it appears here and the reason is documented.
| Path prefix | Why flat |
|---|---|
| `/api/v1/data/**` | Agent ingestion. JWT `env` claim is authoritative; URL-embedded env would invite spoofing. |
| `/api/v1/diagrams/{contentHash}/render`, `POST /api/v1/diagrams/render` | Content-addressed or stateless. |
| `/api/v1/alerts/notifications/{id}/retry` | Notification IDs are globally unique; no env routing needed. |
| `/api/v1/auth/**` | Pre-auth; no env context exists. |
| `/api/v1/health`, `/prometheus`, `/api-docs/**`, `/swagger-ui/**` | Server metadata. |
## Tenant isolation invariant
ClickHouse is shared across tenants. Every ClickHouse query must filter by `tenant_id` (from `CAMELEER_SERVER_TENANT_ID` env var, resolved via `TenantContext`/config) in addition to `environment`. New controllers added under `/environments/{envSlug}/...` must preserve this — the env filter from the path does not replace the tenant filter.
## controller/ — REST endpoints
### Env-scoped (user-facing data & config)
-`AppController` — `/api/v1/environments/{envSlug}/apps`. GET list / POST create / GET `{appSlug}` / DELETE `{appSlug}` / GET `{appSlug}/versions` / POST `{appSlug}/versions` (JAR upload) / PUT `{appSlug}/container-config`. App slug uniqueness is per-env (`(env, app_slug)` is the natural key). `CreateAppRequest` body has no env (path), validates slug regex.
-`DeploymentController` — `/api/v1/environments/{envSlug}/apps/{appSlug}/deployments`. GET list / POST create (body `{ appVersionId }`) / POST `{id}/stop` / POST `{id}/promote` (body `{ targetEnvironment: slug }` — target app slug must exist in target env) / GET `{id}/logs`.
-`ApplicationConfigController` — `/api/v1/environments/{envSlug}`. GET `/config` (list), GET/PUT `/apps/{appSlug}/config`, GET `/apps/{appSlug}/processor-routes`, POST `/apps/{appSlug}/config/test-expression`. PUT also pushes `CONFIG_UPDATE` to LIVE agents in this env.
-`AppSettingsController` — `/api/v1/environments/{envSlug}`. GET `/app-settings` (list), GET/PUT/DELETE `/apps/{appSlug}/settings`. ADMIN/OPERATOR only.
-`SearchController` — `/api/v1/environments/{envSlug}`. GET `/executions`, POST `/executions/search`, GET `/stats`, `/stats/timeseries`, `/stats/timeseries/by-app`, `/stats/timeseries/by-route`, `/stats/punchcard`, `/attributes/keys`, `/errors/top`.
-`LogQueryController` — GET `/api/v1/environments/{envSlug}/logs` (filters: source (multi, comma-split, OR-joined), level (multi, comma-split, OR-joined), application, agentId, exchangeId, logger, q, time range; sort asc/desc). Cursor-paginated, returns `{ data, nextCursor, hasMore, levelCounts }`; cursor is base64url of `"{timestampIso}|{insert_id_uuid}"` — same-millisecond tiebreak via the `insert_id` UUID column on `logs`.
-`RouteCatalogController` — GET `/api/v1/environments/{envSlug}/routes` (merged route catalog from registry + ClickHouse; env filter unconditional).
-`RouteMetricsController` — GET `/api/v1/environments/{envSlug}/routes/metrics`, GET `/api/v1/environments/{envSlug}/routes/metrics/processors`.
-`AgentListController` — GET `/api/v1/environments/{envSlug}/agents` (registered agents with runtime metrics, filtered to env).
-`AgentEventsController` — GET `/api/v1/environments/{envSlug}/agents/events` (lifecycle events; cursor-paginated, returns `{ data, nextCursor, hasMore }`; order `(timestamp DESC, insert_id DESC)`; cursor is base64url of `"{timestampIso}|{insert_id_uuid}"` — `insert_id` is a stable UUID column used as a same-millisecond tiebreak).
-`AgentMetricsController` — GET `/api/v1/environments/{envSlug}/agents/{agentId}/metrics` (JVM/Camel metrics). Rejects cross-env agents (404) as defence-in-depth.
-`DiagramRenderController` — GET `/api/v1/environments/{envSlug}/apps/{appSlug}/routes/{routeId}/diagram` (env-scoped lookup). Also GET `/api/v1/diagrams/{contentHash}/render` (flat — content hashes are globally unique).
-`AlertRuleController` — `/api/v1/environments/{envSlug}/alerts/rules`. GET list / POST create / GET `{id}` / PUT `{id}` / DELETE `{id}` / POST `{id}/enable` / POST `{id}/disable` / POST `{id}/render-preview` / POST `{id}/test-evaluate`. OPERATOR+ for mutations, VIEWER+ for reads. CRITICAL: attribute keys in `ExchangeMatchCondition.filter.attributes` are validated at rule-save time against `^[a-zA-Z0-9._-]+$` — they are later inlined into ClickHouse SQL. Webhook validation: verifies `outboundConnectionId` exists and `isAllowedInEnvironment`. Null notification templates default to `""` (NOT NULL constraint). Audit: `ALERT_RULE_CHANGE`.
-`AlertController` — `/api/v1/environments/{envSlug}/alerts`. GET list (inbox filtered by userId/groupIds/roleNames via `InAppInboxQuery`) / GET `/unread-count` / GET `{id}` / POST `{id}/ack` / POST `{id}/read` / POST `/bulk-read`. VIEWER+ for all. Inbox SQL: `? = ANY(target_user_ids) OR target_group_ids && ? OR target_role_names && ?` — requires at least one matching target (no broadcast concept).
-`AlertSilenceController` — `/api/v1/environments/{envSlug}/alerts/silences`. GET list / POST create / DELETE `{id}`. 422 if `endsAt <= startsAt`. OPERATOR+ for mutations, VIEWER+ for list. Audit: `ALERT_SILENCE_CHANGE`.
-`AlertNotificationController` — Dual-path (no class-level prefix). GET `/api/v1/environments/{envSlug}/alerts/{alertId}/notifications` (VIEWER+); POST `/api/v1/alerts/notifications/{id}/retry` (OPERATOR+, flat — notification IDs globally unique). Retry resets attempts to 0 and sets `nextAttemptAt = now`.
### Env admin (env-slug-parameterized, not env-scoped data)
-`EnvironmentAdminController` — `/api/v1/admin/environments`. GET list / POST create / GET `{envSlug}` / PUT `{envSlug}` / DELETE `{envSlug}` / PUT `{envSlug}/default-container-config` / PUT `{envSlug}/jar-retention`. Slug immutable — PUT body has no slug field; any slug supplied is dropped by Jackson. Slug validated on POST.
-`AgentRegistrationController` — POST `/register` (requires `environmentId` in body; 400 if missing), POST `/{id}/refresh` (rejects tokens with no `env` claim), POST `/{id}/heartbeat` (env from body preferred, JWT fallback; 400 if neither), POST `/{id}/deregister`.
-`AgentSseController` — GET `/{id}/events` (SSE connection).
-`AgentCommandController` — POST `/{agentId}/commands`, POST `/groups/{group}/commands`, POST `/commands` (broadcast), POST `/{agentId}/commands/{commandId}/ack`, POST `/{agentId}/replay`.
-`AgentConfigController` — GET `/api/v1/agents/config`. Agent-authoritative config read: resolves (app, env) from JWT subject → registry (registry miss falls back to JWT env claim; no registry entry → 404 since application can't be derived).
### Ingestion (agent-only, JWT-authoritative)
-`LogIngestionController` — POST `/api/v1/data/logs` (accepts `List<LogEntry>`; WARNs on missing identity, unregistered agents, empty payloads, buffer-full drops).
-`EventIngestionController` — POST `/api/v1/data/events`.
-`ChunkIngestionController` — POST `/api/v1/ingestion/chunk/{executions|metrics|diagrams}`.
-`ExecutionController` — POST `/api/v1/data/executions` (legacy ingestion path when ClickHouse disabled).
-`MetricsController` — POST `/api/v1/data/metrics`.
-`DiagramController` — POST `/api/v1/data/diagrams` (resolves applicationId + environment from the agent registry keyed on JWT subject; stamps both on the stored `TaggedDiagram`).
### Cross-env discovery (flat)
-`CatalogController` — GET `/api/v1/catalog` (merges managed apps + in-memory agents + CH stats; optional `?environment=` filter). DELETE `/api/v1/catalog/{applicationId}` (ADMIN: dismiss app, purge all CH data + PG record).
### Admin (cross-env, flat)
-`UserAdminController` — CRUD `/api/v1/admin/users`, POST `/{id}/roles`, POST `/{id}/set-password`.
-`OidcConfigAdminController` — GET/POST `/api/v1/admin/oidc`, POST `/test`.
-`OutboundConnectionAdminController` — `/api/v1/admin/outbound-connections`. GET list / POST create / GET `{id}` / PUT `{id}` / DELETE `{id}` / POST `{id}/test` / GET `{id}/usage`. RBAC: list/get/usage ADMIN|OPERATOR; mutations + test ADMIN.
-`SensitiveKeysAdminController` — GET/PUT `/api/v1/admin/sensitive-keys`. GET returns 200 or 204 if not configured. PUT accepts `{ keys: [...] }` with optional `?pushToAgents=true`. Fan-out iterates every distinct `(application, environment)` slice — intentional global baseline + per-env overrides.
-`ClaimMappingAdminController` — CRUD `/api/v1/admin/claim-mappings`, POST `/test`.
-`TraefikLabelBuilder` — generates Traefik Docker labels for path-based or subdomain routing. Also emits `cameleer.replica` and `cameleer.instance-id` labels per container for labels-first identity.
-`PrometheusLabelBuilder` — generates Prometheus Docker labels (`prometheus.scrape/path/port`) per runtime type for `docker_sd_configs` auto-discovery
-`ContainerLogForwarder` — streams Docker container stdout/stderr to ClickHouse with `source='container'`. One follow-stream thread per container, batches lines every 2s/50 lines via `ClickHouseLogStore.insertBufferedBatch()`. 60-second max capture timeout.
-`DisabledRuntimeOrchestrator` — no-op when runtime not enabled
## metrics/ — Prometheus observability
-`ServerMetrics` — centralized business metrics: gauges (agents by state, SSE connections, buffer depths), counters (ingestion drops, agent transitions, deployment outcomes, auth failures), timers (flush duration, deployment duration). Exposed via `/api/v1/prometheus`.
-`PostgresAppSettingsRepository`, `PostgresApplicationConfigRepository`, `PostgresThresholdRepository`. Both `app_settings` and `application_config` are env-scoped (PK `(app_id, environment)` / `(application, environment)`); finders take `(app, env)` — no env-agnostic variants.
-`crypto/SecretCipher` — AES-GCM symmetric cipher with key derived via HMAC-SHA256(jwtSecret, "cameleer-outbound-secret-v1"). Ciphertext format: base64(IV(12 bytes) || GCM output with 128-bit tag). `encrypt` throws `IllegalStateException`; `decrypt` throws `IllegalArgumentException` on tamper/wrong-key/malformed.
-`storage/PostgresOutboundConnectionRepository` — JdbcTemplate impl. `save()` upserts by id; JSONB serialization via ObjectMapper; UUID arrays via `ConnectionCallback`. Reads `created_by`/`updated_by` as String (= users.user_id TEXT).
-`OutboundConnectionServiceImpl` — service layer. Tenant bound at construction via `cameleer.server.tenant.id` property. Uniqueness check via `findByName`. Narrowing-envs guard: rejects update that removes envs while rules reference the connection (rulesReferencing stubbed in Plan 01, wired in Plan 02). Delete guard: rejects if referenced by rules.
-`controller/OutboundConnectionAdminController` — REST controller. Class-level `@PreAuthorize("hasRole('ADMIN')")` defaults; GETs relaxed to ADMIN|OPERATOR. Extracts acting user id from `SecurityContextHolder.authentication.name`, strips "user:" prefix. Audit via `AuditCategory.OUTBOUND_CONNECTION_CHANGE`.
-`dto/OutboundConnectionRequest` — Bean Validation: `@NotBlank` name, `@Pattern("^https://.+")` url, `@NotNull` method/tlsTrustMode/auth. Compact ctor throws `IllegalArgumentException` if TRUST_PATHS with empty paths list.
-`dto/OutboundConnectionDto` — response DTO. `hmacSecretSet: boolean` instead of the ciphertext; `authKind: OutboundAuthKind` instead of the full auth config.
-`dto/OutboundConnectionTestResult` — result of POST `/{id}/test`: status, latencyMs, responseSnippet (first 512 chars), tlsProtocol/cipherSuite/peerCertSubject (protocol is "TLS" stub; enriched in Plan 02 follow-up), error (nullable).
- Docker: multi-stage build (`Dockerfile`), `$BUILDPLATFORM` for native Maven on ARM64 runner, amd64 runtime. `docker-entrypoint.sh` imports `/certs/ca.pem` into JVM truststore before starting the app (supports custom CAs for OIDC discovery without `CAMELEER_SERVER_SECURITY_OIDCTLSSKIPVERIFY`).
-`REGISTRY_TOKEN` build arg required for `cameleer-common` dependency resolution
- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `cameleer-postgres-credentials`, `cameleer-clickhouse-credentials`
- K8s probes: server uses `/api/v1/health`, PostgreSQL uses `pg_isready -U "$POSTGRES_USER"` (env var, not hardcoded)
- K8s security: server and database pods run with `securityContext.runAsNonRoot`. UI (nginx) runs without securityContext (needs root for entrypoint setup).
- Docker: server Dockerfile has no default credentials — all DB config comes from env vars at runtime
-`SensitiveKeysMerger` — pure function: merge(global, perApp) -> union with case-insensitive dedup, preserves first-seen casing. Returns null when both inputs null.
-`AppSettings`, `AppSettingsRepository` — per-app-per-env settings config and persistence. Record carries `(applicationId, environment, …)`; repository methods are `findByApplicationAndEnvironment`, `findByEnvironment`, `save`, `delete(appId, env)`. `AppSettings.defaults(appId, env)` produces a default instance scoped to an environment.
-`ThresholdConfig`, `ThresholdRepository` — alerting threshold config and persistence
-`AuditService` — audit logging facade
-`AuditRecord`, `AuditResult`, `AuditCategory` (enum: `INFRA, AUTH, USER_MGMT, CONFIG, RBAC, AGENT, OUTBOUND_CONNECTION_CHANGE, OUTBOUND_HTTP_TRUST_CHANGE`), `AuditRepository` — audit trail records and persistence
-`OutboundConnection` — record: id, tenantId, name, description, url, method, defaultHeaders, defaultBodyTmpl, tlsTrustMode, tlsCaPemPaths, hmacSecretCiphertext, auth, allowedEnvironmentIds, createdAt, createdBy (String user_id), updatedAt, updatedBy (String user_id). `isAllowedInEnvironment(envId)` returns true when allowed-envs list is empty OR contains the env.
-`OutboundAuth` — sealed interface + records: `None | Bearer(tokenCiphertext) | Basic(username, passwordCiphertext)`. Jackson `@JsonTypeInfo(use = DEDUCTION)` — wire shape has no discriminator, subtype inferred from fields.
-`OutboundAuthKind`, `OutboundMethod` — enums
-`OutboundConnectionRepository` — CRUD by (tenantId, id): save/findById/findByName/listByTenant/delete
-`OutboundConnectionService` — create/update/delete/get/list with uniqueness + narrow-envs + delete-if-referenced guards. `rulesReferencing(id)` stubbed in Plan 01 (returns `[]`); populated in Plan 02 against `AlertRuleRepository`.
-`ChunkAccumulator` — batches data for efficient flush
-`WriteBuffer` — bounded ring buffer for async flush
-`BufferedLogEntry` — log entry wrapper with metadata
-`MergedExecution`, `TaggedExecution`, `TaggedDiagram` — tagged ingestion records. `TaggedDiagram` carries `(instanceId, applicationId, environment, graph)` — env is resolved from the agent registry in the controller and stamped on the ClickHouse `route_diagrams` row.
When deployed via the cameleer-saas platform, this server orchestrates customer app containers using Docker. Key components:
- **ConfigMerger** (`core/runtime/ConfigMerger.java`) — pure function: resolve(globalDefaults, envConfig, appConfig) -> ResolvedContainerConfig. Three-layer merge: global (application.yml) -> environment (defaultContainerConfig JSONB) -> app (containerConfig JSONB). Includes `runtimeType` (default `"auto"`) and `customArgs` (default `""`).
- **TraefikLabelBuilder** (`app/runtime/TraefikLabelBuilder.java`) — generates Traefik Docker labels for path-based (`/{envSlug}/{appSlug}/`) or subdomain-based (`{appSlug}-{envSlug}.{domain}`) routing. Supports strip-prefix and SSL offloading toggles. Also sets per-replica identity labels: `cameleer.replica` (index) and `cameleer.instance-id` (`{envSlug}-{appSlug}-{replicaIndex}`). Internal processing uses labels (not container name parsing) for extensibility.
- **PrometheusLabelBuilder** (`app/runtime/PrometheusLabelBuilder.java`) — generates Prometheus `docker_sd_configs` labels per resolved runtime type: Spring Boot `/actuator/prometheus:8081`, Quarkus/native `/q/metrics:9000`, plain Java `/metrics:9464`. Labels merged into container metadata alongside Traefik labels at deploy time.
- **DockerNetworkManager** (`app/runtime/DockerNetworkManager.java`) — manages two Docker network tiers:
-`cameleer-traefik` — shared network; Traefik, server, and all app containers attach here. Server joined via docker-compose with `cameleer-server` DNS alias.
-`cameleer-env-{slug}` — per-environment isolated network; containers in the same environment discover each other via Docker DNS. In SaaS mode, env networks are tenant-scoped: `cameleer-env-{tenantId}-{envSlug}` (overloaded `envNetworkName(tenantId, envSlug)` method) to prevent cross-tenant collisions when multiple tenants have identically-named environments.
- **DockerEventMonitor** (`app/runtime/DockerEventMonitor.java`) — persistent Docker event stream listener for containers with `managed-by=cameleer-server` label. Detects die/oom/start/stop events and updates deployment replica states. Periodic reconciliation (@Scheduled every 30s) inspects actual container state and corrects deployment status mismatches (fixes stale DEGRADED with all replicas healthy).
- **ContainerLogForwarder** (`app/runtime/ContainerLogForwarder.java`) — streams Docker container stdout/stderr to ClickHouse `logs` table with `source='container'`. Uses `docker logs --follow` per container, batches lines every 2s or 50 lines. Parses Docker timestamp prefix, infers log level via regex. `DeploymentExecutor` starts capture after each replica launches with the replica's `instanceId` (`{envSlug}-{appSlug}-{replicaIndex}`); `DockerEventMonitor` stops capture on die/oom. 60-second max capture timeout with 30s cleanup scheduler. Thread pool of 10 daemon threads. Container logs use the same `instanceId` as the agent (set via `CAMELEER_AGENT_INSTANCEID` env var) for unified log correlation at the instance level.
- **StartupLogPanel** (`ui/src/components/StartupLogPanel.tsx`) — collapsible log panel rendered below `DeploymentProgress`. Queries `/api/v1/logs?source=container&application={appSlug}&environment={envSlug}`. Auto-polls every 3s while deployment is STARTING; shows green "live" badge during polling, red "stopped" badge on FAILED. Uses `useStartupLogs` hook and `LogViewer` (design system).
## DeploymentExecutor Details
Primary network for app containers is set via `CAMELEER_SERVER_RUNTIME_DOCKERNETWORK` env var (in SaaS mode: `cameleer-tenant-{slug}`); apps also connect to `cameleer-traefik` (routing) and `cameleer-env-{tenantId}-{envSlug}` (per-environment discovery) as additional networks. Resolves `runtimeType: auto` to concrete type from `AppVersion.detectedRuntimeType` at PRE_FLIGHT (fails deployment if unresolvable). Builds Docker entrypoint per runtime type (all JVM types use `-javaagent:/app/agent.jar -jar`, plain Java uses `-cp` with main class, native runs binary directly). Sets per-replica `CAMELEER_AGENT_INSTANCEID` env var to `{envSlug}-{appSlug}-{replicaIndex}` so container logs and agent logs share the same instance identity. Sets `CAMELEER_AGENT_*` env vars from `ResolvedContainerConfig` (routeControlEnabled, replayEnabled, health port). These are startup-only agent properties — changing them requires redeployment.
## Deployment Status Model
| Status | Meaning |
|--------|---------|
| `STOPPED` | Intentionally stopped or initial state |
| `STARTING` | Deploy in progress |
| `RUNNING` | All replicas healthy and serving |
| `DEGRADED` | Some replicas healthy, some dead |
| `STOPPING` | Graceful shutdown in progress |
| `FAILED` | Terminal failure (pre-flight, health check, or crash) |
**Replica support**: deployments can specify a replica count. `DEGRADED` is used when at least one but not all replicas are healthy.
**Blue/green strategy**: when re-deploying, new replicas are started and health-checked before old ones are stopped, minimising downtime.
**Deployment uniqueness**: `DeploymentService.createDeployment()` deletes any STOPPED/FAILED deployments for the same app+environment before creating a new one, preventing duplicate rows.
## JAR Management
- **Retention policy** per environment: configurable maximum number of JAR versions to keep. Older JARs are deleted automatically.
- **Nightly cleanup job** (`JarRetentionJob`, Spring `@Scheduled` 03:00): purges JARs exceeding the retention limit and removes orphaned files not referenced by any app version. Skips versions currently deployed.
- **Volume-based JAR mounting** for Docker-in-Docker setups: set `CAMELEER_SERVER_RUNTIME_JARDOCKERVOLUME` to the Docker volume name that contains the JAR storage directory. When set, the orchestrator mounts this volume into the container instead of bind-mounting the host path (required when the SaaS container itself runs inside Docker and the host path is not accessible from sibling containers).
## Runtime Type Detection
The server detects the app framework from uploaded JARs and builds Docker entrypoints. The agent shaded JAR bundles the log appender, so no separate `cameleer-log-appender.jar` or `PropertiesLauncher` is needed:
- **Detection** (`RuntimeDetector`): runs at JAR upload time. Checks ZIP magic bytes (non-ZIP = native binary), then probes `META-INF/MANIFEST.MF` Main-Class: Spring Boot loader prefix -> `spring-boot`, Quarkus entry point -> `quarkus`, other Main-Class -> `plain-java` (extracts class name). Results stored on `AppVersion` (`detected_runtime_type`, `detected_main_class`).
- **Runtime types** (`RuntimeType` enum): `AUTO`, `SPRING_BOOT`, `QUARKUS`, `PLAIN_JAVA`, `NATIVE`. Configurable per app/environment via `containerConfig.runtimeType` (default `"auto"`).
- **Entrypoint per type**: All JVM types use `java -javaagent:/app/agent.jar -jar app.jar`. Plain Java uses `-cp` with explicit main class instead of `-jar`. Native runs the binary directly.
- **Custom arguments** (`containerConfig.customArgs`): freeform string appended to the start command. Validated against a strict pattern to prevent shell injection (entrypoint uses `sh -c`).
- **AUTO resolution**: at deploy time (PRE_FLIGHT), `"auto"` resolves to the detected type from `AppVersion`. Fails deployment if detection was unsuccessful — user must set type explicitly.
- **UI**: Resources tab shows Runtime Type dropdown (with detection hint from latest uploaded version) and Custom Arguments text field.
## SaaS Multi-Tenant Network Isolation
In SaaS mode, each tenant's server and its deployed apps are isolated at the Docker network level:
- **Tenant network** (`cameleer-tenant-{slug}`) — primary internal bridge for all of a tenant's containers. Set as `CAMELEER_SERVER_RUNTIME_DOCKERNETWORK` for the tenant's server instance. Tenant A's apps cannot reach tenant B's apps.
- **Shared services network** — server also connects to the shared infrastructure network (PostgreSQL, ClickHouse, Logto) and `cameleer-traefik` for HTTP routing.
- **Tenant-scoped environment networks** (`cameleer-env-{tenantId}-{envSlug}`) — per-environment discovery is scoped per tenant, so `alpha-corp`'s "dev" environment network is separate from `beta-corp`'s "dev" environment network.
## nginx / Reverse Proxy
-`client_max_body_size 200m` is required in the nginx config to allow JAR uploads up to 200 MB. Without this, large JAR uploads return 413.
This project is indexed by GitNexus as **cameleer-server** (6306 symbols, 15892 relationships, 300 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
## Always Do
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`.
## When Debugging
1.`gitnexus_query({query: "<error or symptom>"})` — find execution flows related to the issue
2.`gitnexus_context({name: "<suspect function>"})` — see all callers, callees, and process participation
3.`READ gitnexus://repo/cameleer-server/process/{processName}` — trace the full execution flow step by step
4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed
## When Refactoring
- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`.
- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code.
- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed.
## Never Do
- NEVER edit a function, class, or method without first running `gitnexus_impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph.
- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope.
Before completing any code modification task, verify:
1.`gitnexus_impact` was run for all modified symbols
2. No HIGH/CRITICAL risk warnings were ignored
3.`gitnexus_detect_changes()` confirms changes match expected scope
4. All d=1 (WILL BREAK) dependents were updated
## Keeping the Index Fresh
After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it:
```bash
npx gitnexus analyze
```
If the index previously included embeddings, preserve them by adding `--embeddings`:
```bash
npx gitnexus analyze --embeddings
```
To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.**
> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`.
## CLI
| Task | Read this skill file |
|------|---------------------|
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
Server exposes `/api/v1/prometheus` (unauthenticated, Prometheus text format). Spring Boot Actuator provides JVM, GC, thread pool, and `http.server.requests` metrics automatically. Business metrics via `ServerMetrics` component:
All containers also get `prometheus.scrape=true`. These labels enable Prometheus `docker_sd_configs` auto-discovery.
## Agent Metric Names (Micrometer)
Agents send `MetricsSnapshot` records with Micrometer-convention metric names. The server stores them generically (ClickHouse `agent_metrics.metric_name`). The UI references specific names in `AgentInstance.tsx` for JVM charts.
### JVM metrics (used by UI)
| Metric name | UI usage |
|---|---|
| `process.cpu.usage.value` | CPU % stat card + chart |
Mean processing time = `camel.route.policy.total_time / camel.route.policy.count`. Min processing time is not available (Micrometer does not track minimums).
-`ui/src/components/StartupLogPanel.tsx` — deployment startup log viewer (container logs from ClickHouse, polls 3s while STARTING)
-`ui/src/api/queries/logs.ts` — `useStartupLogs` hook for container startup log polling, `useLogs`/`useApplicationLogs` for bounded log search (single page), `useInfiniteApplicationLogs` for streaming log views (cursor-paginated, server-side source/level filters)
-`ui/src/api/queries/agents.ts` — `useAgents` for agent list, `useInfiniteAgentEvents` for cursor-paginated timeline stream
-`ui/src/hooks/useInfiniteStream.ts` — tanstack `useInfiniteQuery` wrapper with top-gated auto-refetch, flattened `items[]`, and `refresh()` invalidator
-`ui/src/components/InfiniteScrollArea.tsx` — scrollable container with IntersectionObserver top/bottom sentinels. Streaming log/event views use this + `useInfiniteStream`. Bounded views (LogTab, StartupLogPanel) keep `useLogs`/`useStartupLogs`
## UI Styling
- Always use `@cameleer/design-system` CSS variables for colors (`var(--amber)`, `var(--error)`, `var(--success)`, etc.) — never hardcode hex values. This applies to CSS modules, inline styles, and SVG `fill`/`stroke` attributes. SVG presentation attributes resolve `var()` correctly. All colors use CSS variables (no hardcoded hex).
- Shared CSS modules in `ui/src/styles/` (table-section, log-panel, rate-colors, refresh-indicator, chart-card, section-card) — import these instead of duplicating patterns.
- Design system components used consistently: `Select`, `Tabs`, `Toggle`, `Button`, `LogViewer`, `Label` — prefer DS components over raw HTML elements. `LogViewer` renders optional source badges (`container`, `app`, `agent`) via `LogEntry.source` field (DS v0.1.49+).
- Environment slugs are auto-computed from display name (read-only in UI).
- Brand assets: `@cameleer/design-system/assets/` provides `camel-logo.svg` (currentColor), `cameleer-{16,32,48,192,512}.png`, and `cameleer-logo.png`. Copied to `ui/public/` for use as favicon (`favicon-16.png`, `favicon-32.png`) and logo (`camel-logo.svg` — login dialog 36px, sidebar 28x24px).
- Sidebar generates `/exchanges/` paths directly (no legacy `/apps/` redirects). basePath is centralized in `ui/src/config.ts`; router.tsx imports it instead of re-reading `<base>` tag.
- Global user preferences (environment selection) use Zustand stores with localStorage persistence — never URL search params. URL params are for page-specific state only (e.g. `?text=` search query). Switching environment resets all filters and remounts pages.
An observability server that receives, stores, and serves Apache Camel route execution data from distributed Cameleer3 agents. Think njams Server (by Integration Matters) — but built incrementally, API-first, with a modern stack. Users can search through millions of recorded transactions by state, time, duration, full text, and correlate executions across multiple Camel instances. The server also pushes configuration, tracing controls, and ad-hoc commands to agents via SSE.
An observability server that receives, stores, and serves Apache Camel route execution data from distributed Cameleer agents. Think njams Server (by Integration Matters) — but built incrementally, API-first, with a modern stack. Users can search through millions of recorded transactions by state, time, duration, full text, and correlate executions across multiple Camel instances. The server also pushes configuration, tracing controls, and ad-hoc commands to agents via SSE.
## Core Value
@@ -16,7 +16,7 @@ Users can reliably search and find any transaction across all connected Camel in
### Active
- [ ] Receive and ingest transaction/activity data from Cameleer3 agents via HTTP POST
- [ ] Receive and ingest transaction/activity data from Cameleer agents via HTTP POST
- [ ] Store transactions in a high-volume, horizontally scalable data store with 30-day retention
- [ ] Search transactions by state, execution date/time, duration, and full-text content
- [ ] Correlate activities across multiple routes and Camel instances within a single transaction
@@ -38,8 +38,8 @@ Users can reliably search and find any transaction across all connected Camel in
## Context
- **Agent side**: cameleer3 agent (`https://gitea.siegeln.net/cameleer/cameleer3`) is under active development; already supports creating diagrams and capturing executions
- **Shared library**: `com.cameleer3:cameleer3-common` contains shared models and the graph API; protocol defined in `cameleer3-common/PROTOCOL.md`
- **Agent side**: cameleer agent (`https://gitea.siegeln.net/cameleer/cameleer`) is under active development; already supports creating diagrams and capturing executions
- **Shared library**: `com.cameleer:cameleer-common` contains shared models and the graph API; protocol defined in `cameleer-common/PROTOCOL.md`
- **Data model**: Hierarchical — a **transaction** represents a message's full journey, containing **activities** per route execution. Transactions can span multiple Camel instances (e.g., route A calls route B on another instance via endpoint)
- **Scale target**: Millions of transactions per day, 50+ connected agents, 30-day data retention
- **Query pattern**: Incident-driven — mostly recent data queries, with deep historical dives during incidents
@@ -49,7 +49,7 @@ Users can reliably search and find any transaction across all connected Camel in
## Constraints
- **Tech stack**: Java 17+, Spring Boot 3.4.3, Maven multi-module — already established
- **Dependency**: Must consume `com.cameleer3:cameleer3-common` from Gitea Maven registry
- **Dependency**: Must consume `com.cameleer:cameleer-common` from Gitea Maven registry
- **Protocol**: Agent protocol is still evolving — server must adapt as it stabilizes
- **Incremental delivery**: Build step by step; storage and search first, then layer features on top
**Core Value:** Users can reliably search and find any transaction across all connected Camel instances — by any combination of state, time, duration, or content — even at millions of transactions per day with 30-day retention.
- MetricsRepository: void insertBatch(List<MetricsSnapshot> metrics) -- use a generic type or the cameleer3-common metrics model if available; if not, create a simple MetricsData record in core module
- MetricsRepository: void insertBatch(List<MetricsSnapshot> metrics) -- use a generic type or the cameleer-common metrics model if available; if not, create a simple MetricsData record in core module
4. Create IngestionConfig as @ConfigurationProperties("ingestion"):
- bufferCapacity (int, default 50000)
@@ -193,7 +193,7 @@ Output: Working ClickHouse via Docker Compose, DDL with TTL, WriteBuffer with un
- No custom bean needed if relying on auto-config; only create if explicit JdbcTemplate customization required
</action>
<verify>
<automated>mvn test -pl cameleer3-server-core -Dtest=WriteBufferTest -q 2>&1 | tail -10</automated>
<automated>mvn test -pl cameleer-server-core -Dtest=WriteBufferTest -q 2>&1 | tail -10</automated>
</verify>
<done>WriteBuffer passes all unit tests. Repository interfaces exist with correct method signatures. IngestionConfig reads from application.yml.</done>
</task>
@@ -201,7 +201,7 @@ Output: Working ClickHouse via Docker Compose, DDL with TTL, WriteBuffer with un
</tasks>
<verification>
-`mvn test -pl cameleer3-server-core -q` -- all WriteBuffer unit tests pass
-`mvn test -pl cameleer-server-core -q` -- all WriteBuffer unit tests pass
-`mvn clean compile -q` -- full project compiles with new dependencies
- POST /api/v1/data/executions with single RouteExecution JSON returns 202
@@ -245,7 +245,7 @@ public class IngestionConfig {
Note: All integration tests must include X-Cameleer-Protocol-Version:1 header (API-04 will be enforced by Plan 03's interceptor, but include the header now for forward compatibility).
</action>
<verify>
<automated>mvn test -pl cameleer3-server-app -Dtest="ExecutionControllerIT,DiagramControllerIT,MetricsControllerIT,BackpressureIT" -q 2>&1 | tail -15</automated>
<automated>mvn test -pl cameleer-server-app -Dtest="ExecutionControllerIT,DiagramControllerIT,MetricsControllerIT,BackpressureIT" -q 2>&1 | tail -15</automated>
</verify>
<done>All three ingestion endpoints return 202 on valid data. Data arrives in ClickHouse after flush. Buffer-full returns 503 with Retry-After. Unknown JSON fields accepted. Integration tests green.</done>
</task>
@@ -253,7 +253,7 @@ public class IngestionConfig {
</tasks>
<verification>
-`mvn test -pl cameleer3-server-app -Dtest="ExecutionControllerIT,DiagramControllerIT,MetricsControllerIT,BackpressureIT" -q` -- all integration tests pass
-`mvn test -pl cameleer-server-app -Dtest="ExecutionControllerIT,DiagramControllerIT,MetricsControllerIT,BackpressureIT" -q` -- all integration tests pass
-`cameleer-server-app/.../controller/ExecutionControllerIT.java` - 4 tests: single, array, flush, unknown fields
-`cameleer-server-app/.../controller/DiagramControllerIT.java` - 3 tests: single, array, flush
-`cameleer-server-app/.../controller/MetricsControllerIT.java` - 2 tests: POST, flush
-`cameleer-server-app/.../controller/BackpressureIT.java` - 2 tests: 503 response, data not lost
## Decisions Made
- Controllers accept raw String body and detect single vs array JSON (starts with `[`), supporting both payload formats per protocol spec
@@ -119,7 +119,7 @@ Each task was committed atomically:
- **Found during:** Task 2 (integration test context startup)
- **Issue:** IngestionConfig had both `@Configuration` and `@ConfigurationProperties`, while `@EnableConfigurationProperties(IngestionConfig.class)` on the app class created a second bean, causing "expected single matching bean but found 2"
- **Fix:** Removed `@Configuration` from IngestionConfig, relying solely on `@EnableConfigurationProperties`
<done>AbstractClickHouseIT base class ready for integration tests. ProtocolVersionInterceptor validates header on data/agent paths. Health, swagger, and api-docs paths excluded. Application class enables scheduling and config properties.</done>
</task>
@@ -129,10 +129,10 @@ Output: AbstractClickHouseIT base class, working health, Swagger UI, protocol he
- GET /api/v1/health returns 200 with JSON containing status field
@@ -178,7 +178,7 @@ Output: AbstractClickHouseIT base class, working health, Swagger UI, protocol he
Note: All tests that POST to data endpoints must include X-Cameleer-Protocol-Version:1 header.
</action>
<verify>
<automated>mvn test -pl cameleer3-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q 2>&1 | tail -15</automated>
<automated>mvn test -pl cameleer-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q 2>&1 | tail -15</automated>
</verify>
<done>Health returns 200. OpenAPI docs are available and list endpoints. Protocol version header enforced on data paths, not on health/docs. Unknown JSON fields accepted. TTL confirmed in ClickHouse DDL via HealthControllerIT test methods.</done>
</task>
@@ -186,7 +186,7 @@ Output: AbstractClickHouseIT base class, working health, Swagger UI, protocol he
</tasks>
<verification>
-`mvn test -pl cameleer3-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q` -- all tests pass
-`mvn test -pl cameleer-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q` -- all tests pass
- GET /api/v1/health returns 200
- GET /api/v1/api-docs returns OpenAPI spec
- Missing protocol header returns 400 on data endpoints
- ProtocolVersionInterceptor validates X-Cameleer-Protocol-Version:1 on /api/v1/data/** and /api/v1/agents/** paths, returning 400 JSON error for missing or wrong version
- AbstractClickHouseIT base class with Testcontainers ClickHouse 25.3, shared static container, schema init from 01-schema.sql
- 12 integration tests: health endpoint (2), OpenAPI docs (2), protocol version enforcement (5), forward compatibility (1), TTL verification (2)
- Cameleer3ServerApplication with @EnableScheduling, @EnableConfigurationProperties, and dual package scanning
- CameleerServerApplication with @EnableScheduling, @EnableConfigurationProperties, and dual package scanning
## Task Commits
@@ -80,17 +80,17 @@ Each task was committed atomically:
2.**Task 2: Integration tests for health, OpenAPI, protocol version, forward compat, and TTL** - `2d3fde3` (test)
## Files Created/Modified
-`cameleer3-server-app/src/main/java/.../Cameleer3ServerApplication.java` - Spring Boot entry point with scheduling and config properties
-`cameleer3-server-app/src/main/java/.../interceptor/ProtocolVersionInterceptor.java` - Validates protocol version header on data/agent paths
-`cameleer3-server-app/src/main/java/.../config/WebConfig.java` - Registers interceptor with path patterns and exclusions
-`cameleer3-server-app/src/test/java/.../AbstractClickHouseIT.java` - Shared Testcontainers base class for ITs
-`cameleer3-server-app/src/test/resources/application-test.yml` - Test profile with small buffer config
-`cameleer3-server-app/src/test/java/.../controller/HealthControllerIT.java` - Health endpoint and TTL tests
-`cameleer3-server-app/src/test/java/.../controller/OpenApiIT.java` - OpenAPI and Swagger UI tests
Phase 1 establishes the data pipeline and API skeleton for Cameleer3 Server. Agents POST execution data, diagrams, and metrics to REST endpoints; the server buffers these in memory and batch-flushes to ClickHouse. The ClickHouse schema design is the most critical and least reversible decision in this phase -- ORDER BY and partitioning cannot be changed without table recreation.
Phase 1 establishes the data pipeline and API skeleton for Cameleer Server. Agents POST execution data, diagrams, and metrics to REST endpoints; the server buffers these in memory and batch-flushes to ClickHouse. The ClickHouse schema design is the most critical and least reversible decision in this phase -- ORDER BY and partitioning cannot be changed without table recreation.
The ClickHouse Java ecosystem has undergone significant changes. The recommended approach is **clickhouse-jdbc v0.9.7** (JDBC V2 driver) with Spring Boot's JdbcTemplate for batch inserts. An alternative is the standalone **client-v2** artifact which offers a POJO-based insert API, but JDBC integration with Spring Boot is more conventional and better documented. ClickHouse now has a native full-text index (TYPE text, GA as of March 2026) that supersedes the older tokenbf_v1 bloom filter approach -- this is relevant for Phase 2 but should be accounted for in schema design now.
@@ -17,7 +17,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
| ID | Description | Research Support |
|----|-------------|-----------------|
| INGST-01 (#1) | Accept RouteExecution via POST /api/v1/data/executions, return 202 | REST controller + async write buffer pattern; Jackson deserialization of cameleer3-common models |
| INGST-01 (#1) | Accept RouteExecution via POST /api/v1/data/executions, return 202 | REST controller + async write buffer pattern; Jackson deserialization of cameleer-common models |
| INGST-02 (#2) | Accept RouteGraph via POST /api/v1/data/diagrams, return 202 | Same pattern; separate ClickHouse table for diagrams with content-hash dedup |
| INGST-03 (#3) | Accept metrics via POST /api/v1/data/metrics, return 202 | Same pattern; separate ClickHouse table for metrics |
| INGST-04 (#4) | In-memory batch buffer with configurable flush interval/size | ArrayBlockingQueue + @Scheduled flush; configurable via application.yml |
@@ -60,7 +60,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
| ArrayBlockingQueue | LMAX Disruptor | Disruptor is faster under extreme contention but adds complexity; ABQ is sufficient for this throughput |
| Spring JdbcTemplate | Raw JDBC PreparedStatement | JdbcTemplate provides cleaner error handling and resource management; no meaningful overhead |
**Installation (add to cameleer3-server-app/pom.xml):**
**Installation (add to cameleer-server-app/pom.xml):**
```xml
<!-- ClickHouse JDBC V2 -->
<dependency>
@@ -103,7 +103,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
</dependency>
```
**Add to cameleer3-server-core/pom.xml:**
**Add to cameleer-server-core/pom.xml:**
```xml
<!-- SLF4J for logging (no Spring dependency) -->
<dependency>
@@ -117,7 +117,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/ExecutionController.java` | POST /api/v1/data/executions endpoint | VERIFIED | 79 lines; `@PostMapping("/executions")`; handles single/array via raw String parsing; returns 202 or 503 + Retry-After |
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/ClickHouseExecutionRepository.java` | Batch insert to route_executions via JdbcTemplate | VERIFIED | 118 lines; `@Repository`; `BatchPreparedStatementSetter`; flattens processor tree to parallel arrays |
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/ingestion/ClickHouseFlushScheduler.java` | Scheduled drain of WriteBuffer into ClickHouse | VERIFIED | 160 lines; `@Scheduled(fixedDelayString="${ingestion.flush-interval-ms:1000}")`; implements `SmartLifecycle` for shutdown drain |
| `cameleer3-server-core/src/main/java/com/cameleer3/server/core/ingestion/IngestionService.java` | Routes data to appropriate WriteBuffer instances | VERIFIED | 115 lines; plain class; `acceptExecution`, `acceptExecutions`, `acceptDiagram`, `acceptDiagrams`, `acceptMetrics`; delegates to typed `WriteBuffer` instances |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/controller/ExecutionController.java` | POST /api/v1/data/executions endpoint | VERIFIED | 79 lines; `@PostMapping("/executions")`; handles single/array via raw String parsing; returns 202 or 503 + Retry-After |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/ClickHouseExecutionRepository.java` | Batch insert to route_executions via JdbcTemplate | VERIFIED | 118 lines; `@Repository`; `BatchPreparedStatementSetter`; flattens processor tree to parallel arrays |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/ingestion/ClickHouseFlushScheduler.java` | Scheduled drain of WriteBuffer into ClickHouse | VERIFIED | 160 lines; `@Scheduled(fixedDelayString="${ingestion.flush-interval-ms:1000}")`; implements `SmartLifecycle` for shutdown drain |
| `cameleer-server-core/src/main/java/com/cameleer/server/core/ingestion/IngestionService.java` | Routes data to appropriate WriteBuffer instances | VERIFIED | 115 lines; plain class; `acceptExecution`, `acceptExecutions`, `acceptDiagram`, `acceptDiagrams`, `acceptMetrics`; delegates to typed `WriteBuffer` instances |
#### Plan 01-03 Artifacts
| Artifact | Expected | Status | Details |
|---|---|---|---|
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java` | Validates X-Cameleer-Protocol-Version:1 header on data endpoints | VERIFIED | 47 lines; implements `HandlerInterceptor.preHandle`; returns 400 JSON on missing/wrong version |
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java` | Registers interceptor with path patterns | VERIFIED | 35 lines; `addInterceptors` registers interceptor on `/api/v1/data/**` and `/api/v1/agents/**`; excludes health, api-docs, swagger-ui |
| `cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractClickHouseIT.java` | Shared Testcontainers base class for integration tests | VERIFIED | 73 lines; static `ClickHouseContainer`; `@DynamicPropertySource`; `@BeforeAll` schema init from SQL file; `JdbcTemplate` exposed to subclasses |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/interceptor/ProtocolVersionInterceptor.java` | Validates X-Cameleer-Protocol-Version:1 header on data endpoints | VERIFIED | 47 lines; implements `HandlerInterceptor.preHandle`; returns 400 JSON on missing/wrong version |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/config/WebConfig.java` | Registers interceptor with path patterns | VERIFIED | 35 lines; `addInterceptors` registers interceptor on `/api/v1/data/**` and `/api/v1/agents/**`; excludes health, api-docs, swagger-ui |
| `cameleer-server-app/src/test/java/com/cameleer/server/app/AbstractClickHouseIT.java` | Shared Testcontainers base class for integration tests | VERIFIED | 73 lines; static `ClickHouseContainer`; `@DynamicPropertySource`; `@BeforeAll` schema init from SQL file; `JdbcTemplate` exposed to subclasses |
---
@@ -113,7 +113,7 @@ No orphaned requirements — all 11 IDs declared in plan frontmatter match the R
### Anti-Patterns Found
No anti-patterns detected. Scanned all source files in `cameleer3-server-app/src/main` and `cameleer3-server-core/src/main` for TODO/FIXME/PLACEHOLDER/stub return patterns. None found.
No anti-patterns detected. Scanned all source files in `cameleer-server-app/src/main` and `cameleer-server-core/src/main` for TODO/FIXME/PLACEHOLDER/stub return patterns. None found.
- Add tokenbf_v1 skip indexes on exchange_bodies and exchange_headers (GRANULARITY 4, same as idx_error)
- Add tokenbf_v1 skip index on error_stacktrace (it has no index yet, needed for SRCH-05 full-text search across stack traces)
2. Create core search domain types in `com.cameleer3.server.core.search`:
2. Create core search domain types in `com.cameleer.server.core.search`:
-`SearchRequest` record: status (String, nullable), timeFrom (Instant), timeTo (Instant), durationMin (Long), durationMax (Long), correlationId (String), text (String — global full-text), textInBody (String), textInHeaders (String), textInErrors (String), offset (int), limit (int). Compact constructor validates: limit defaults to 50 if <= 0, capped at 500; offset defaults to 0 if < 0.
-`SearchResult<T>` record: data (List<T>), total (long), offset (int), limit (int). Include static factory `empty(int offset, int limit)`.
-`ExecutionSummary` record: executionId (String), routeId (String), agentId (String), status (String), startTime (Instant), endTime (Instant), durationMs (long), correlationId (String), errorMessage (String), diagramContentHash (String). This is the lightweight list-view DTO — NOT the full processor arrays.
-`SearchEngine` interface with methods: `SearchResult<ExecutionSummary> search(SearchRequest request)` and `long count(SearchRequest request)`. This is the swappable backend (ClickHouse now, OpenSearch later per user decision).
-`SearchService` class: plain class (no Spring annotations, same pattern as IngestionService). Constructor takes SearchEngine. `search(SearchRequest)` delegates to engine.search(). This thin orchestration layer allows adding cross-cutting concerns later.
3. Create core detail domain types in `com.cameleer3.server.core.detail`:
3. Create core detail domain types in `com.cameleer.server.core.detail`:
-`ProcessorNode` record: processorId (String), processorType (String), status (String), startTime (Instant), endTime (Instant), durationMs (long), diagramNodeId (String), errorMessage (String), errorStackTrace (String), children (List<ProcessorNode>). This is the nested tree node.
-`ExecutionDetail` record: executionId (String), routeId (String), agentId (String), status (String), startTime (Instant), endTime (Instant), durationMs (long), correlationId (String), exchangeId (String), errorMessage (String), errorStackTrace (String), diagramContentHash (String), processors (List<ProcessorNode>). This is the full detail response.
-`DetailService` class: plain class (no Spring annotations). Constructor takes ExecutionRepository. Method `getDetail(String executionId)` returns `Optional<ExecutionDetail>`. Calls repository's new `findDetailById` method, then calls `reconstructTree()` to convert flat arrays into nested ProcessorNode tree. The `reconstructTree` method: takes parallel arrays (ids, types, statuses, starts, ends, durations, diagramNodeIds, errorMessages, errorStackTraces, depths, parentIndexes), creates ProcessorNode[] array, then wires children using parentIndexes (parentIndex == -1 means root).
Actually, use a different approach per the layering: add a `findRawById(String executionId)` method that returns `Optional<RawExecutionRow>` — a new record containing all parallel arrays. DetailService takes this and reconstructs. Create `RawExecutionRow` as a record in the detail package with all fields needed for reconstruction.
<done>Schema migration SQL exists, all core domain types compile, SearchEngine interface and SearchService defined, ExecutionRepository extended with query method, DetailService has tree reconstruction logic</done>
- Test: After inserting a RouteExecution with processors that have exchange snapshots and nested children, the route_executions row has non-empty exchange_bodies, exchange_headers, processor_depths (correct depth values), processor_parent_indexes (correct parent wiring), processor_input_bodies, processor_output_bodies, processor_input_headers, processor_output_headers, processor_diagram_node_ids, and diagram_content_hash columns
- Verifies a second insertion with null snapshots succeeds with empty defaults
</action>
<verify>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer3-server && mvn test -pl cameleer3-server-app -Dtest=IngestionSchemaIT</automated>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer-server && mvn test -pl cameleer-server-app -Dtest=IngestionSchemaIT</automated>
</verify>
<done>All new columns populated correctly during ingestion, tree metadata (depth/parent) correct for nested processors, exchange data concatenated for search, existing ingestion tests still pass</done>
- Used FlatProcessor record to carry depth and parentIndex alongside the ProcessorExecution during DFS flattening -- single pass, no separate traversal
@@ -116,9 +116,9 @@ Each task was committed atomically:
**1. [Rule 3 - Blocking] Created DiagramRenderer and DiagramLayout stub interfaces**
- **Found during:** Task 2 (compilation step)
- **Issue:** Pre-existing `ElkDiagramRenderer` in app module referenced `DiagramRenderer` and `DiagramLayout` interfaces that did not exist in core module, causing compilation failure
- **Fix:** Created minimal stub interfaces in `com.cameleer3.server.core.diagram` package
- **Fix:** Created minimal stub interfaces in `com.cameleer.server.core.diagram` package
2. Create core diagram rendering interfaces in `com.cameleer3.server.core.diagram`:
2. Create core diagram rendering interfaces in `com.cameleer.server.core.diagram`:
- `PositionedNode` record: id (String), label (String), type (String — NodeType name), x (double), y (double), width (double), height (double), children (List<PositionedNode> — for compound/swimlane groups). JSON-serializable for the JSON layout response.
- Unit test: ElkDiagramRenderer.renderSvg with a simple 3-node graph (from->process->to) produces valid SVG containing svg element, rect elements for nodes, line/path elements for edges
- Integration test: GET /api/v1/diagrams/{hash} with no Accept preference defaults to SVG
</behavior>
<action>
1. Create `ElkDiagramRenderer` implementing `DiagramRenderer` in `com.cameleer3.server.app.diagram`:
1. Create `ElkDiagramRenderer` implementing `DiagramRenderer` in `com.cameleer.server.app.diagram`:
**Layout phase (shared by both SVG and JSON):**
- Convert RouteGraph to ELK graph: create ElkNode root, set properties for LayeredOptions.ALGORITHM_ID, Direction.DOWN (top-to-bottom per user decision), spacing 40px node-node, 20px edge-node.
- GET /api/v1/diagrams/{hash}/render with no Accept header -> assert SVG response (default).
</action>
<verify>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer3-server && mvn test -pl cameleer3-server-app -Dtest="ElkDiagramRendererTest,DiagramRenderControllerIT"</automated>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer-server && mvn test -pl cameleer-server-app -Dtest="ElkDiagramRendererTest,DiagramRenderControllerIT"</automated>
</verify>
<done>Diagram rendering produces color-coded top-to-bottom SVG and JSON layout, content negotiation works via Accept header, compound nodes group nested processors, all tests pass</done>
- Test searchByStatus: Insert 3 executions (COMPLETED, FAILED, RUNNING). GET /api/v1/search/executions?status=FAILED returns only the FAILED execution. Response has envelope: {"data":[...],"total":1,"offset":0,"limit":50}
@@ -221,7 +221,7 @@ Established controller pattern (from Phase 1):
- Test emptyResults: Search with no matches returns {"data":[],"total":0,"offset":0,"limit":50}
</behavior>
<action>
1. Create `ClickHouseSearchEngine` in `com.cameleer3.server.app.search`:
1. Create `ClickHouseSearchEngine` in `com.cameleer.server.app.search`:
- Implements SearchEngine interface from core module.
- Constructor takes JdbcTemplate.
-`search(SearchRequest)` method:
@@ -244,13 +244,13 @@ Established controller pattern (from Phase 1):
-`escapeLike(String)` utility: escape `%`, `_`, `\` characters in user input to prevent LIKE injection. Replace `\` with `\\`, `%` with `\%`, `_` with `\_`.
-`count(SearchRequest)` method: same WHERE building, just count query.
2. Create `SearchBeanConfig` in `com.cameleer3.server.app.config`:
2. Create `SearchBeanConfig` in `com.cameleer.server.app.config`:
-`DetailService` bean (takes the execution query interface from Plan 01)
3. Create `SearchController` in `com.cameleer3.server.app.controller`:
3. Create `SearchController` in `com.cameleer.server.app.controller`:
- Inject SearchService.
-`GET /api/v1/search/executions` with @RequestParam for basic filters:
- status (optional String)
@@ -274,7 +274,7 @@ Established controller pattern (from Phase 1):
- Assert response structure matches the envelope format.
</action>
<verify>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer3-server && mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT</automated>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer-server && mvn test -pl cameleer-server-app -Dtest=SearchControllerIT</automated>
</verify>
<done>All search filter types work independently and in combination, response envelope has correct format, pagination works correctly, full-text search finds matches in all text fields, LIKE patterns are properly escaped</done>
</task>
@@ -282,10 +282,10 @@ Established controller pattern (from Phase 1):
<task type="auto" tdd="true">
<name>Task 2: DetailController, tree reconstruction, exchange snapshot endpoint, and integration tests</name>
- Unit test: reconstructTree with [root, child, grandchild], depths=[0,1,2], parents=[-1,0,1] produces single root with one child that has one grandchild
@@ -308,7 +308,7 @@ Established controller pattern (from Phase 1):
- Add `findRawById(String executionId)` method that queries all columns from route_executions WHERE execution_id = ?. Return Optional<RawExecutionRow> (use the record created in Plan 01 or create it here if needed). The RawExecutionRow should contain ALL columns including the parallel arrays for processors.
- Add `findProcessorSnapshot(String executionId, int processorIndex)` method: queries processor_input_bodies[index+1], processor_output_bodies[index+1], processor_input_headers[index+1], processor_output_headers[index+1] for the given execution. Returns a DTO with inputBody, outputBody, inputHeaders, outputHeaders. ClickHouse arrays are 1-indexed in SQL, so add 1 to the Java 0-based index.
3. Create `DetailController` in `com.cameleer3.server.app.controller`:
3. Create `DetailController` in `com.cameleer.server.app.controller`:
- Inject DetailService.
-`GET /api/v1/executions/{executionId}`: call detailService.getDetail(executionId). If empty, return 404. Otherwise return 200 with ExecutionDetail JSON. The processors field is a nested tree of ProcessorNode objects.
-`GET /api/v1/executions/{executionId}/processors/{index}/snapshot`: call repository's findProcessorSnapshot. If execution not found or index out of bounds, return 404. Return JSON with inputBody, outputBody, inputHeaders, outputHeaders. Per user decision: exchange snapshot data fetched separately per processor, not inlined in detail response.
@@ -323,7 +323,7 @@ Established controller pattern (from Phase 1):
- Test GET /api/v1/executions/{id}/processors/999/snapshot: returns 404 for out-of-bounds index.
</action>
<verify>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer3-server && mvn test -pl cameleer3-server-core -Dtest=TreeReconstructionTest && mvn test -pl cameleer3-server-app -Dtest=DetailControllerIT</automated>
<automated>cd C:/Users/Hendrik/Documents/projects/cameleer-server && mvn test -pl cameleer-server-core -Dtest=TreeReconstructionTest && mvn test -pl cameleer-server-app -Dtest=DetailControllerIT</automated>
</verify>
<done>Tree reconstruction correctly rebuilds nested processor trees from flat arrays, detail endpoint returns nested tree with all fields, snapshot endpoint returns per-processor exchange data, diagram hash included in detail response, all tests pass</done>
</task>
@@ -331,9 +331,9 @@ Established controller pattern (from Phase 1):
</tasks>
<verification>
-`mvn test -pl cameleer3-server-core -Dtest=TreeReconstructionTest` passes (unit test for tree rebuild)
-`mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT` passes (all search filters)
-`mvn test -pl cameleer3-server-app -Dtest=DetailControllerIT` passes (detail + snapshot)
-`mvn test -pl cameleer-server-core -Dtest=TreeReconstructionTest` passes (unit test for tree rebuild)
-`mvn test -pl cameleer-server-app -Dtest=SearchControllerIT` passes (all search filters)
-`mvn test -pl cameleer-server-app -Dtest=DetailControllerIT` passes (detail + snapshot)
- **Issue:** Core module only had JUnit Jupiter as test dependency, TreeReconstructionTest needed assertj for assertions and mockito for mock(ExecutionRepository.class)
- **Fix:** Added assertj-core and mockito-core test-scoped dependencies to cameleer3-server-core/pom.xml
- Test 1: When a RouteGraph is ingested before a RouteExecution for the same routeId+agentId, the execution's diagram_content_hash column contains the SHA-256 hash of the diagram (not empty string)
@@ -117,7 +117,7 @@ public class ClickHouseDiagramRepository implements DiagramRepository {
**Gap 2 — Surefire classloader isolation:**
5. In `cameleer3-server-app/pom.xml`, add a `<build><plugins>` section (after the existing `spring-boot-maven-plugin`) with `maven-surefire-plugin` configuration:
5. In `cameleer-server-app/pom.xml`, add a `<build><plugins>` section (after the existing `spring-boot-maven-plugin`) with `maven-surefire-plugin` configuration:
```xml
<plugin>
<groupId>org.apache.maven.plugins</groupId>
@@ -131,12 +131,12 @@ public class ClickHouseDiagramRepository implements DiagramRepository {
This forces Surefire to fork a fresh JVM for each test class, isolating ELK's static initializer (LayeredMetaDataProvider + xtext CollectionLiterals) from Spring Boot's classloader. Trade-off: slightly slower test execution, but correct results.
- DiagramRepository injected via constructor into ClickHouseExecutionRepository -- both are @Repository Spring beans, so constructor injection autowires cleanly
| SRCH-01 | Filter by status returns matching executions | integration | `mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT#searchByStatus` | No -- Wave 0 |
| SRCH-02 | Filter by time range returns matching executions | integration | `mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT#searchByTimeRange` | No -- Wave 0 |
| SRCH-03 | Filter by duration range returns matching | integration | `mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT#searchByDuration` | No -- Wave 0 |
| SRCH-04 | Filter by correlationId returns correlated | integration | `mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT#searchByCorrelationId` | No -- Wave 0 |
| SRCH-05 | Full-text search across bodies/headers/errors | integration | `mvn test -pl cameleer3-server-app -Dtest=SearchControllerIT#fullTextSearch` | No -- Wave 0 |
| SRCH-06 | Detail returns nested processor tree | integration | `mvn test -pl cameleer3-server-app -Dtest=DetailControllerIT#detailReturnsNestedTree` | No -- Wave 0 |
| DIAG-01 | Content-hash dedup stores identical defs once | integration | `mvn test -pl cameleer3-server-app -Dtest=DiagramControllerIT#contentHashDedup` | Partial (ingestion test exists) |
| DIAG-02 | Transaction links to active diagram version | integration | `mvn test -pl cameleer3-server-app -Dtest=DetailControllerIT#detailIncludesDiagramHash` | No -- Wave 0 |
| DIAG-03 | Diagram rendered as SVG or JSON layout | integration | `mvn test -pl cameleer3-server-app -Dtest=DiagramRenderControllerIT#renderSvg` | No -- Wave 0 |
| SRCH-01 | Filter by status returns matching executions | integration | `mvn test -pl cameleer-server-app -Dtest=SearchControllerIT#searchByStatus` | No -- Wave 0 |
| SRCH-02 | Filter by time range returns matching executions | integration | `mvn test -pl cameleer-server-app -Dtest=SearchControllerIT#searchByTimeRange` | No -- Wave 0 |
| SRCH-03 | Filter by duration range returns matching | integration | `mvn test -pl cameleer-server-app -Dtest=SearchControllerIT#searchByDuration` | No -- Wave 0 |
| SRCH-04 | Filter by correlationId returns correlated | integration | `mvn test -pl cameleer-server-app -Dtest=SearchControllerIT#searchByCorrelationId` | No -- Wave 0 |
| SRCH-05 | Full-text search across bodies/headers/errors | integration | `mvn test -pl cameleer-server-app -Dtest=SearchControllerIT#fullTextSearch` | No -- Wave 0 |
| SRCH-06 | Detail returns nested processor tree | integration | `mvn test -pl cameleer-server-app -Dtest=DetailControllerIT#detailReturnsNestedTree` | No -- Wave 0 |
| DIAG-01 | Content-hash dedup stores identical defs once | integration | `mvn test -pl cameleer-server-app -Dtest=DiagramControllerIT#contentHashDedup` | Partial (ingestion test exists) |
| DIAG-02 | Transaction links to active diagram version | integration | `mvn test -pl cameleer-server-app -Dtest=DetailControllerIT#detailIncludesDiagramHash` | No -- Wave 0 |
| DIAG-03 | Diagram rendered as SVG or JSON layout | integration | `mvn test -pl cameleer-server-app -Dtest=DiagramRenderControllerIT#renderSvg` | No -- Wave 0 |
### Sampling Rate
- **Per task commit:** `mvn test -pl cameleer3-server-app -Dtest=<relevant>IT`
- **Per task commit:** `mvn test -pl cameleer-server-app -Dtest=<relevant>IT`
- **Per wave merge:** `mvn clean verify`
- **Phase gate:** Full suite green before `/gsd:verify-work`
@@ -551,7 +551,7 @@ private void populateExchangeColumns(PreparedStatement ps, List<FlatProcessor> p
### Primary (HIGH confidence)
- ClickHouse JDBC 0.9.7, ClickHouse 25.3 -- verified from project pom.xml and AbstractClickHouseIT
- cameleer3-common 1.0-SNAPSHOT JAR -- decompiled to verify RouteGraph, RouteNode, RouteEdge, NodeType, ProcessorExecution, ExchangeSnapshot field structures
- cameleer-common 1.0-SNAPSHOT JAR -- decompiled to verify RouteGraph, RouteNode, RouteEdge, NodeType, ProcessorExecution, ExchangeSnapshot field structures
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/ClickHouseExecutionRepository.java` | VERIFIED | DiagramRepository injected via constructor (line 59); `findContentHashForRoute` called in `setValues()` (lines 144–147); former `""` placeholder removed |
| `cameleer3-server-app/pom.xml` | VERIFIED | `maven-surefire-plugin` with `forkCount=1``reuseForks=false` at lines 95–100; `maven-failsafe-plugin` same config at lines 103–108 |
| `cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/DiagramLinkingIT.java` | VERIFIED | 152 lines; 2 integration tests; positive case asserts 64-char hex hash; negative case asserts empty string; uses `ignoreExceptions()` for ClickHouse eventual consistency |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/ClickHouseExecutionRepository.java` | VERIFIED | DiagramRepository injected via constructor (line 59); `findContentHashForRoute` called in `setValues()` (lines 144–147); former `""` placeholder removed |
| `cameleer-server-app/pom.xml` | VERIFIED | `maven-surefire-plugin` with `forkCount=1``reuseForks=false` at lines 95–100; `maven-failsafe-plugin` same config at lines 103–108 |
| `cameleer-server-app/src/test/java/com/cameleer/server/app/storage/DiagramLinkingIT.java` | VERIFIED | 152 lines; 2 integration tests; positive case asserts 64-char hex hash; negative case asserts empty string; uses `ignoreExceptions()` for ClickHouse eventual consistency |
| `Surefire/Failsafe` | ELK classloader isolation | `reuseForks=false` forces fresh JVM per test class | WIRED | Lines 95–116 in `cameleer3-server-app/pom.xml` |
| `Surefire/Failsafe` | ELK classloader isolation | `reuseForks=false` forces fresh JVM per test class | WIRED | Lines 95–116 in `cameleer-server-app/pom.xml` |
### Requirements Coverage
@@ -108,7 +108,7 @@ Two blockers from the initial verification (2026-03-11T16:00:00Z) have been reso
**Gap 1 resolved — DIAG-02 diagram hash linking:**`ClickHouseExecutionRepository` now injects `DiagramRepository` via constructor and calls `findContentHashForRoute(exec.getRouteId(), "")` in `insertBatch()`. Both the diagram store path and the execution ingest path use `agent_id=""` consistently, so the lookup is correct. `DiagramLinkingIT` provides integration test coverage for both the positive case (hash populated when diagram exists) and negative case (empty string when no diagram exists for the route).
**Gap 2 resolved — Test suite stability:** Both `maven-surefire-plugin` and `maven-failsafe-plugin` in `cameleer3-server-app/pom.xml` are now configured with `forkCount=1``reuseForks=false`. This forces a fresh JVM per test class, isolating ELK's `LayeredMetaDataProvider` static initializer from Spring Boot's classloader. The SUMMARY reports 51 tests, 0 failures. Test count across 16 test files totals 80 `@Test` methods; the difference from 51 reflects how Surefire/Failsafe counts parameterized and nested tests vs. raw annotation count.
**Gap 2 resolved — Test suite stability:** Both `maven-surefire-plugin` and `maven-failsafe-plugin` in `cameleer-server-app/pom.xml` are now configured with `forkCount=1``reuseForks=false`. This forces a fresh JVM per test class, isolating ELK's `LayeredMetaDataProvider` static initializer from Spring Boot's classloader. The SUMMARY reports 51 tests, 0 failures. Test count across 16 test files totals 80 `@Test` methods; the difference from 51 reflects how Surefire/Failsafe counts parameterized and nested tests vs. raw annotation count.
No regressions were introduced. All 10 observable truths and all 9 phase requirements are now satisfied. Two items remain for human visual verification (SVG rendering correctness).
3.**Update Cameleer3ServerApplication**: Add AgentRegistryConfig.class to @EnableConfigurationProperties.
3.**Update CameleerServerApplication**: Add AgentRegistryConfig.class to @EnableConfigurationProperties.
4.**Update application.yml**: Add agent-registry section with all defaults (see RESEARCH.md code example). Also add `spring.mvc.async.request-timeout: -1` for SSE support (Plan 02 needs it, but set it now).
- Use TestRestTemplate (already available from AbstractClickHouseIT's @SpringBootTest)
</action>
<verify>
<automated>mvn test -pl cameleer3-server-core,cameleer3-server-app -Dtest="Agent*"</automated>
<automated>mvn test -pl cameleer-server-core,cameleer-server-app -Dtest="Agent*"</automated>
</verify>
<done>POST /register returns 200 with agentId + sseEndpoint + heartbeatIntervalMs. POST /{id}/heartbeat returns 200 for known agents, 404 for unknown. GET /agents returns all agents with optional ?status= filter. AgentLifecycleMonitor runs on schedule. All integration tests pass. mvn clean verify passes.</done>
Build the SSE infrastructure and command delivery system:
@@ -181,7 +181,7 @@ From cameleer3-server-app/.../config/AgentRegistryConfig.java:
5.**Update WebConfig**: The SSE endpoint GET /api/v1/agents/{id}/events is already covered by the interceptor pattern "/api/v1/agents/**". Agents send the protocol version header on all requests (per research recommendation), so no exclusion needed. However, if the SSE GET causes issues because browsers/clients may not easily add custom headers to EventSource, add the SSE events path to excludePathPatterns: `/api/v1/agents/*/events`. This is a practical consideration -- add the exclusion to be safe.
@@ -224,7 +224,7 @@ From cameleer3-server-app/.../config/AgentRegistryConfig.java:
**Test configuration**: If ping interval needs to be shorter for tests, add to test application.yml or use @TestPropertySource with agent-registry.ping-interval-ms=1000.
</action>
<verify>
<automated>mvn test -pl cameleer3-server-core,cameleer3-server-app -Dtest="Agent*"</automated>
<automated>mvn test -pl cameleer-server-core,cameleer-server-app -Dtest="Agent*"</automated>
</verify>
<done>All SSE integration tests pass: connect/disconnect, config-update/deep-trace/replay delivery via SSE, ping keepalive received, Last-Event-ID accepted, command targeting (single/group/broadcast), command acknowledgement. mvn clean verify passes with all existing tests still green.</done>
This phase adds agent registration, heartbeat-based lifecycle management (LIVE/STALE/DEAD), and real-time command push via SSE to the Cameleer3 server. The technology stack is straightforward: Spring MVC's `SseEmitter` for server-push, `ConcurrentHashMap` for the in-memory agent registry, and `@Scheduled` for periodic lifecycle checks (same pattern already used by `ClickHouseFlushScheduler`).
This phase adds agent registration, heartbeat-based lifecycle management (LIVE/STALE/DEAD), and real-time command push via SSE to the Cameleer server. The technology stack is straightforward: Spring MVC's `SseEmitter` for server-push, `ConcurrentHashMap` for the in-memory agent registry, and `@Scheduled` for periodic lifecycle checks (same pattern already used by `ClickHouseFlushScheduler`).
The main architectural challenge is managing per-agent SSE connections reliably -- handling disconnections, timeouts, and cleanup without leaking threads or emitters. The command delivery model (PENDING with 60s expiry, acknowledgement) adds a second concurrent data structure to manage alongside the registry itself.
@@ -93,7 +93,7 @@ No new dependencies required. Everything is already on the classpath.
| `cameleer3-server-core/src/main/java/com/cameleer3/server/core/agent/AgentInfo.java` | Immutable record with all fields and wither methods | VERIFIED | 63 lines; record with 10 fields and 5 wither-style methods |
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/AgentRegistrationController.java` | POST /register, POST /{id}/heartbeat, GET /agents | VERIFIED | 153 lines; all three endpoints implemented with OpenAPI annotations |
| `cameleer3-server-app/src/main/java/com/cameleer3/server/app/agent/AgentLifecycleMonitor.java` | @Scheduled LIVE->STALE->DEAD transitions | VERIFIED | 37 lines; calls `registryService.checkLifecycle()` and `expireOldCommands()` on schedule |
| `cameleer-server-core/src/main/java/com/cameleer/server/core/agent/AgentInfo.java` | Immutable record with all fields and wither methods | VERIFIED | 63 lines; record with 10 fields and 5 wither-style methods |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentRegistrationController.java` | POST /register, POST /{id}/heartbeat, GET /agents | VERIFIED | 153 lines; all three endpoints implemented with OpenAPI annotations |
| `cameleer-server-app/src/main/java/com/cameleer/server/app/agent/AgentLifecycleMonitor.java` | @Scheduled LIVE->STALE->DEAD transitions | VERIFIED | 37 lines; calls `registryService.checkLifecycle()` and `expireOldCommands()` on schedule |
1. Add Maven dependencies to cameleer3-server-app/pom.xml:
1. Add Maven dependencies to cameleer-server-app/pom.xml:
-`spring-boot-starter-security` (managed version)
-`com.nimbusds:nimbus-jose-jwt:9.47` (explicit, may not be transitive without OAuth2 resource server)
-`spring-security-test` scope test (managed version)
@@ -165,12 +165,12 @@ public class AgentRegistryConfig { ... }
5. Update application-test.yml: Add `security.bootstrap-token: test-bootstrap-token`, `security.bootstrap-token-previous: old-bootstrap-token`. Also set `CAMELEER_AUTH_TOKEN: test-bootstrap-token` as an env override if needed.
6. IMPORTANT: Adding spring-boot-starter-security will break ALL existing tests immediately (401 on all endpoints). To prevent this during Plan 01 (before the security filter chain is configured in Plan 02), add a temporary test security config class `src/test/java/com/cameleer3/server/app/security/TestSecurityConfig.java` annotated `@TestConfiguration` that creates a `SecurityFilterChain` permitting all requests. This keeps existing tests green while security services are built. Plan 02 will replace this with real security config and update tests.
6. IMPORTANT: Adding spring-boot-starter-security will break ALL existing tests immediately (401 on all endpoints). To prevent this during Plan 01 (before the security filter chain is configured in Plan 02), add a temporary test security config class `src/test/java/com/cameleer/server/app/security/TestSecurityConfig.java` annotated `@TestConfiguration` that creates a `SecurityFilterChain` permitting all requests. This keeps existing tests green while security services are built. Plan 02 will replace this with real security config and update tests.
7. Write unit tests per the behavior spec above. Tests should NOT require Spring context -- construct implementations directly with test SecurityProperties.
</action>
<verify>
<automated>cd /c/Users/Hendrik/Documents/projects/cameleer3-server && mvn test -pl cameleer3-server-app -Dtest="JwtServiceTest,Ed25519SigningServiceTest,BootstrapTokenValidatorTest" -Dsurefire.reuseForks=false</automated>
<automated>cd /c/Users/Hendrik/Documents/projects/cameleer-server && mvn test -pl cameleer-server-app -Dtest="JwtServiceTest,Ed25519SigningServiceTest,BootstrapTokenValidatorTest" -Dsurefire.reuseForks=false</automated>
</verify>
<done>
- JwtService creates and validates access/refresh JWTs with correct claims and expiry
1. Create `JwtAuthenticationFilter extends OncePerRequestFilter` (NOT annotated @Component -- constructed in SecurityConfig to avoid double registration):
@@ -176,7 +176,7 @@ public class AgentRegistryService {
6. Update `WebConfig` if needed: The `ProtocolVersionInterceptor` excluded paths should align with Spring Security public paths. The SSE events path is already excluded from protocol version check (Phase 3 decision). Verify no conflicts.
1. Replace the Plan 01 temporary `TestSecurityConfig` (permit-all) with real security active in tests. Remove the permit-all override so tests run with actual security enforcement.
@@ -259,7 +259,7 @@ public class AgentRegistryService {
- Test: New access token from refresh can access protected endpoints
@@ -155,7 +155,7 @@ public record AgentCommand(String id, CommandType type, String payload, String a
- NOTE: This test depends on Plan 02's bootstrap token and JWT auth being in place. If Plan 03 executes before Plan 02, the test will need the TestSecurityHelper or a different auth approach. Since both are Wave 2 but independent, document this: "If Plan 02 is not yet complete, use TestSecurityHelper from Plan 01's temporary permit-all config."
</action>
<verify>
<automated>cd /c/Users/Hendrik/Documents/projects/cameleer3-server && mvn test -pl cameleer3-server-app -Dtest="SsePayloadSignerTest,SseSigningIT" -Dsurefire.reuseForks=false</automated>
<automated>cd /c/Users/Hendrik/Documents/projects/cameleer-server && mvn test -pl cameleer-server-app -Dtest="SsePayloadSignerTest,SseSigningIT" -Dsurefire.reuseForks=false</automated>
</verify>
<done>
- SsePayloadSigner signs JSON payloads with Ed25519 and adds signature field
This phase adds authentication and integrity protection to the Cameleer3 server. The implementation uses Spring Security 6.4.3 (managed by Spring Boot 3.4.3) with a custom `OncePerRequestFilter` for JWT validation, JDK 17 built-in Ed25519 for signing SSE payloads, and environment variable-based bootstrap tokens for agent registration. The approach is deliberately simple -- no OAuth2 resource server, no external identity provider, just symmetric HMAC JWTs for access control and Ed25519 signatures for payload integrity.
This phase adds authentication and integrity protection to the Cameleer server. The implementation uses Spring Security 6.4.3 (managed by Spring Boot 3.4.3) with a custom `OncePerRequestFilter` for JWT validation, JDK 17 built-in Ed25519 for signing SSE payloads, and environment variable-based bootstrap tokens for agent registration. The approach is deliberately simple -- no OAuth2 resource server, no external identity provider, just symmetric HMAC JWTs for access control and Ed25519 signatures for payload integrity.
The existing codebase has clear integration points: `AgentRegistrationController.register()` already returns `serverPublicKey: null` as a placeholder, `SseConnectionManager.onCommandReady()` is the signing hook for SSE events, and `WebConfig` already defines excluded paths that align with the public endpoint list. Spring Security's `SecurityFilterChain` replaces the need for hand-rolled authorization logic -- endpoints are protected by default, with explicit `permitAll()` for health, register, and docs.
@@ -89,7 +89,7 @@ The existing codebase has clear integration points: `AgentRegistrationController
- **Ed25519 library:** Use JDK built-in. Zero external dependencies, native performance, well-tested in JDK 17+.
- **Refresh token storage:** Use stateless signed refresh tokens (also HMAC-signed JWTs with different claims/expiry). This avoids any in-memory storage for refresh tokens and scales naturally. The refresh token is just a JWT with `type=refresh`, `sub=agentId`, and 7-day expiry. On refresh, validate the refresh JWT, check agent still exists, issue new access JWT.
**Installation (add to cameleer3-server-app pom.xml):**
**Installation (add to cameleer-server-app pom.xml):**
**Domain:** Transaction observability server for Apache Camel integrations
**Researched:** 2026-03-11
@@ -6,7 +6,7 @@
## Executive Summary
Cameleer3 Server is a write-heavy, read-occasional observability system that receives millions of transaction records per day from distributed Apache Camel agents, stores them with 30-day retention, and provides structured + full-text search. The architecture closely parallels established observability platforms like Jaeger, Zipkin, and njams Server, with the key differentiator being Camel route diagram visualization tied to individual transactions.
Cameleer Server is a write-heavy, read-occasional observability system that receives millions of transaction records per day from distributed Apache Camel agents, stores them with 30-day retention, and provides structured + full-text search. The architecture closely parallels established observability platforms like Jaeger, Zipkin, and njams Server, with the key differentiator being Camel route diagram visualization tied to individual transactions.
The recommended stack centers on **ClickHouse** as the primary data store. ClickHouse's columnar MergeTree engine provides the exact properties this project needs: massive batch insert throughput, excellent time-range query performance, native TTL-based retention, and 10-20x compression on structured observability data. This is a well-established pattern used by production observability platforms (SigNoz, Uptrace, PostHog all run on ClickHouse).
@@ -93,9 +93,9 @@ Based on research, suggested phase structure:
## Gaps to Address
- **ClickHouse Java client API:** The clickhouse-java library has undergone significant changes. Exact API, connection pooling, and Spring Boot integration patterns need phase-specific research
- **cameleer3-common PROTOCOL.md:** Must read the agent protocol definition before designing ClickHouse schema -- this defines the exact data structures being ingested
- **cameleer-common PROTOCOL.md:** Must read the agent protocol definition before designing ClickHouse schema -- this defines the exact data structures being ingested
- **ClickHouse Docker setup:** Optimal ClickHouse Docker configuration (memory limits, merge settings) for development and production
- **Full-text search decision:** ClickHouse skip indexes may or may not meet the "search by any content" requirement. This needs prototyping with realistic data
- **Diagram rendering library:** Server-side route diagram rendering is a significant unknown; needs prototyping with actual Camel route graph data from cameleer3-common
- **Diagram rendering library:** Server-side route diagram rendering is a significant unknown; needs prototyping with actual Camel route graph data from cameleer-common
- **Frontend framework:** No research on UI technology -- deferred to UI phase
- **Agent protocol stability:** The cameleer3-common protocol is still evolving. Schema evolution strategy needs alignment with agent development
- **Agent protocol stability:** The cameleer-common protocol is still evolving. Schema evolution strategy needs alignment with agent development
This project is indexed by GitNexus as **alerting-02** (7810 symbols, 20082 relationships, 300 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
## Always Do
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`.
## When Debugging
1.`gitnexus_query({query: "<error or symptom>"})` — find execution flows related to the issue
2.`gitnexus_context({name: "<suspect function>"})` — see all callers, callees, and process participation
3.`READ gitnexus://repo/alerting-02/process/{processName}` — trace the full execution flow step by step
4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed
## When Refactoring
- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`.
- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code.
- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed.
## Never Do
- NEVER edit a function, class, or method without first running `gitnexus_impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph.
- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope.
Before completing any code modification task, verify:
1.`gitnexus_impact` was run for all modified symbols
2. No HIGH/CRITICAL risk warnings were ignored
3.`gitnexus_detect_changes()` confirms changes match expected scope
4. All d=1 (WILL BREAK) dependents were updated
## Keeping the Index Fresh
After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it:
```bash
npx gitnexus analyze
```
If the index previously included embeddings, preserve them by adding `--embeddings`:
```bash
npx gitnexus analyze --embeddings
```
To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.**
> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`.
## CLI
| Task | Read this skill file |
|------|---------------------|
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
@@ -4,18 +4,18 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project
Cameleer3 Server — observability server that receives, stores, and serves Camel route execution data and route diagrams from Cameleer3 agents. Pushes config and commands to agents via SSE.
Cameleer Server — observability server that receives, stores, and serves Camel route execution data and route diagrams from Cameleer agents. Pushes config and commands to agents via SSE. Also orchestrates Docker container deployments when running under cameleer-saas.
## Related Project
- **cameleer3** (`https://gitea.siegeln.net/cameleer/cameleer3`) — the Java agent that instruments Camel applications
- Protocol defined in `cameleer3-common/PROTOCOL.md` in the agent repo
- This server depends on `com.cameleer3:cameleer3-common` (shared models and graph API)
- **cameleer** (`https://gitea.siegeln.net/cameleer/cameleer`) — the Java agent that instruments Camel applications
- Protocol defined in `cameleer-common/PROTOCOL.md` in the agent repo
- This server depends on `com.cameleer:cameleer-common` (shared models and graph API)
- Depends on `com.cameleer3:cameleer3-common` from Gitea Maven registry
- Depends on `com.cameleer:cameleer-common` from Gitea Maven registry
- Jackson `JavaTimeModule` for `Instant` deserialization
- Communication: receives HTTP POST data from agents (executions, diagrams, metrics, logs), serves SSE event streams for config push/commands (config-update, deep-trace, replay, route-control)
-Maintains agent instance registry with states: LIVE → STALE → DEAD
- Storage: PostgreSQL (TimescaleDB) for structured data, OpenSearch for full-text search and application log storage
-Security: JWT auth with RBAC (AGENT/VIEWER/OPERATOR/ADMIN roles), Ed25519 config signing, bootstrap token for registration
-OIDC: Optional external identity provider support (token exchange pattern). Configured via admin API, stored in database (`server_config` table)
-URL taxonomy: user-facing data, config, and query endpoints live under `/api/v1/environments/{envSlug}/...`. Env is a path segment, resolved via the `@EnvPath` argument resolver (404 on unknown slug). Flat endpoints are only for: agent self-service (JWT-authoritative), cross-env admin (RBAC, OIDC, audit, license, thresholds, env CRUD), cross-env discovery (`/catalog`), content-addressed lookups (`/diagrams/{contentHash}/render`, `/executions/{id}`), and auth. See `.claude/rules/app-classes.md` for the full allow-list.
- Slug immutability: environment and app slugs are immutable after creation (both appear in URLs, Docker network names, container names, and ClickHouse partition keys). Slug regex `^[a-z0-9][a-z0-9-]{0,63}$` is enforced on POST; update endpoints silently drop any slug field in the request body via Jackson's default unknown-property handling.
-App uniqueness: `(environment_id, app_slug)` is the natural key. The same app slug can legitimately exist in multiple environments; `AppService.getByEnvironmentAndSlug(envId, slug)` is the canonical lookup for controllers. Bare `getBySlug(slug)` remains for internal use but is ambiguous across envs.
-Environment filtering: all data queries filter by the selected environment. All commands target only agents in the selected environment. Env is required on every env-scoped endpoint (path param); the legacy `?environment=` query form is retired.
- Maintains agent instance registry (in-memory) with states: LIVE -> STALE -> DEAD. Auto-heals from JWT `env` claim + heartbeat body on heartbeat/SSE after server restart (priority: heartbeat `environmentId` > JWT `env` claim; no silent default — missing env on heartbeat auto-heal returns 400). Registration (`POST /api/v1/agents/register`) requires `environmentId` in the request body; missing or blank returns 400. Capabilities and route states updated on every heartbeat (protocol v2). Route catalog merges three sources: in-memory agent registry, persistent `route_catalog` table (ClickHouse), and `stats_1m_route` execution stats. The persistent catalog tracks `first_seen`/`last_seen` per route per environment, updated on every registration and heartbeat. Routes appear in the sidebar when their lifecycle overlaps the selected time window (`first_seen <= to AND last_seen >= from`), so historical routes remain visible even after being dropped from newer app versions.
- Multi-tenancy: each server instance serves one tenant (configured via `CAMELEER_SERVER_TENANT_ID`, default: `"default"`). Environments (dev/staging/prod) are first-class. PostgreSQL isolated via schema-per-tenant (`?currentSchema=tenant_{id}`) and `ApplicationName=tenant_{id}` on the JDBC URL. ClickHouse shared DB with `tenant_id` + `environment` columns, partitioned by `(tenant_id, toYYYYMM(timestamp))`.
- Storage: PostgreSQL for RBAC, config, and audit; ClickHouse for all observability data (executions, search, logs, metrics, stats, diagrams). ClickHouse schema migrations in `clickhouse/*.sql`, run idempotently on startup by `ClickHouseSchemaInitializer`. Use `IF NOT EXISTS` for CREATE and ADD PROJECTION.
- Log exchange correlation: `ClickHouseLogStore` extracts `exchange_id` from log entry MDC, preferring `cameleer.exchangeId` over `camel.exchangeId` (fallback for older agents). For `ON_COMPLETION` exchange copies, the agent sets `cameleer.exchangeId` to the parent's exchange ID via `CORRELATION_ID`.
- Log processor correlation: The agent sets `cameleer.processorId` in MDC, identifying which processor node emitted a log line.
- Logging: ClickHouse JDBC set to INFO (`com.clickhouse`), HTTP client to WARN (`org.apache.hc.client5`) in application.yml
- Security: JWT auth with RBAC (AGENT/VIEWER/OPERATOR/ADMIN roles), Ed25519 config signing (key derived deterministically from JWT secret via HMAC-SHA256), bootstrap token for registration. CORS: `CAMELEER_SERVER_SECURITY_CORSALLOWEDORIGINS` (comma-separated) overrides `CAMELEER_SERVER_SECURITY_UIORIGIN` for multi-origin setups. Infrastructure access: `CAMELEER_SERVER_SECURITY_INFRASTRUCTUREENDPOINTS=false` disables Database and ClickHouse admin endpoints. Last-ADMIN guard: system prevents removal of the last ADMIN role (409 Conflict). Password policy: min 12 chars, 3-of-4 character classes, no username match. Brute-force protection: 5 failed attempts -> 15 min lockout. Token revocation: `token_revoked_before` column on users, checked in `JwtAuthenticationFilter`, set on password change.
- OIDC: Optional external identity provider support (token exchange pattern). Configured via admin API/UI, stored in database (`server_config` table). Resource server mode: accepts external access tokens (Logto M2M) via JWKS validation when `CAMELEER_SERVER_SECURITY_OIDCISSUERURI` is set. Scope-based role mapping via `SystemRole.normalizeScope()`. System roles synced on every OIDC login via `applyClaimMappings()` in `OidcAuthController` (calls `clearManagedAssignments` + `assignManagedRole` on `RbacService`) — always overwrites managed role assignments; uses managed assignment origin to avoid touching group-inherited or directly-assigned roles. Supports ES384, ES256, RS256.
- OIDC role extraction: `OidcTokenExchanger` reads roles from the **access_token** first (JWT with `at+jwt` type), then falls back to id_token. `OidcConfig` includes `audience` (RFC 8707 resource indicator) and `additionalScopes`. All provider-specific configuration is external — no provider-specific code in the server.
- Sensitive keys: Global enforced baseline for masking sensitive data in agent payloads. Merge rule: `final = global UNION per-app` (case-insensitive dedup, per-app can only add, never remove global keys).
- User persistence: PostgreSQL `users` table, admin CRUD at `/api/v1/admin/users`
- V13 — alert_instances open-rule unique index (alert_instances_open_rule_uq partial index on rule_id WHERE state IN PENDING/FIRING/ACKNOWLEDGED)
## UI Styling
ClickHouse: `cameleer-server-app/src/main/resources/clickhouse/init.sql` (run idempotently on startup)
- Always use `@cameleer/design-system` CSS variables for colors (`var(--amber)`, `var(--error)`, `var(--success)`, etc.) — never hardcode hex values. This applies to CSS modules, inline styles, and SVG `fill`/`stroke` attributes. SVG presentation attributes resolve `var()` correctly.
## Regenerating OpenAPI schema (SPA types)
After any change to REST controller paths, request/response DTOs, or `@PathVariable`/`@RequestParam`/`@RequestBody` signatures, regenerate the TypeScript types the SPA consumes. Required for every controller-level change.
```bash
# Backend must be running on :8081
cd ui && npm run generate-api:live # fetches fresh openapi.json AND regenerates schema.d.ts
# OR, if openapi.json was updated by other means:
cd ui && npm run generate-api # regenerates schema.d.ts from existing openapi.json
```
After regeneration, `ui/src/api/schema.d.ts` and `ui/src/api/openapi.json` will update. The TypeScript compiler then surfaces every SPA call site that needs updating — fix all compile errors before testing in the browser. Commit the regenerated files with the controller change.
## Maintaining .claude/rules/
When adding, removing, or renaming classes, controllers, endpoints, UI components, or metrics, update the corresponding `.claude/rules/` file as part of the same change. The rule files are the class/API map that future sessions rely on — stale rules cause wrong assumptions. Treat rule file updates like updating an import: part of the change, not a separate task.
## Disabled Skills
- Do NOT use any `gsd:*` skills in this project. This includes all `/gsd:` prefixed commands.
<!-- gitnexus:start -->
# GitNexus — Code Intelligence
This project is indexed by GitNexus as **alerting-02** (7810 symbols, 20082 relationships, 300 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
> If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.
## Always Do
- **MUST run impact analysis before editing any symbol.** Before modifying a function, class, or method, run `gitnexus_impact({target: "symbolName", direction: "upstream"})` and report the blast radius (direct callers, affected processes, risk level) to the user.
- **MUST run `gitnexus_detect_changes()` before committing** to verify your changes only affect expected symbols and execution flows.
- **MUST warn the user** if impact analysis returns HIGH or CRITICAL risk before proceeding with edits.
- When exploring unfamiliar code, use `gitnexus_query({query: "concept"})` to find execution flows instead of grepping. It returns process-grouped results ranked by relevance.
- When you need full context on a specific symbol — callers, callees, which execution flows it participates in — use `gitnexus_context({name: "symbolName"})`.
## When Debugging
1.`gitnexus_query({query: "<error or symptom>"})` — find execution flows related to the issue
2.`gitnexus_context({name: "<suspect function>"})` — see all callers, callees, and process participation
3.`READ gitnexus://repo/alerting-02/process/{processName}` — trace the full execution flow step by step
4. For regressions: `gitnexus_detect_changes({scope: "compare", base_ref: "main"})` — see what your branch changed
## When Refactoring
- **Renaming**: MUST use `gitnexus_rename({symbol_name: "old", new_name: "new", dry_run: true})` first. Review the preview — graph edits are safe, text_search edits need manual review. Then run with `dry_run: false`.
- **Extracting/Splitting**: MUST run `gitnexus_context({name: "target"})` to see all incoming/outgoing refs, then `gitnexus_impact({target: "target", direction: "upstream"})` to find all external callers before moving code.
- After any refactor: run `gitnexus_detect_changes({scope: "all"})` to verify only expected files changed.
## Never Do
- NEVER edit a function, class, or method without first running `gitnexus_impact` on it.
- NEVER ignore HIGH or CRITICAL risk warnings from impact analysis.
- NEVER rename symbols with find-and-replace — use `gitnexus_rename` which understands the call graph.
- NEVER commit changes without running `gitnexus_detect_changes()` to check affected scope.
Before completing any code modification task, verify:
1.`gitnexus_impact` was run for all modified symbols
2. No HIGH/CRITICAL risk warnings were ignored
3.`gitnexus_detect_changes()` confirms changes match expected scope
4. All d=1 (WILL BREAK) dependents were updated
## Keeping the Index Fresh
After committing code changes, the GitNexus index becomes stale. Re-run analyze to update it:
```bash
npx gitnexus analyze
```
If the index previously included embeddings, preserve them by adding `--embeddings`:
```bash
npx gitnexus analyze --embeddings
```
To check whether embeddings exist, inspect `.gitnexus/meta.json` — the `stats.embeddings` field shows the count (0 means no embeddings). **Running analyze without `--embeddings` will delete any previously generated embeddings.**
> Claude Code users: A PostToolUse hook handles this automatically after `git commit` and `git merge`.
## CLI
| Task | Read this skill file |
|------|---------------------|
| Understand architecture / "How does X work?" | `.claude/skills/gitnexus/gitnexus-exploring/SKILL.md` |
| Blast radius / "What breaks if I change X?" | `.claude/skills/gitnexus/gitnexus-impact-analysis/SKILL.md` |
| Trace bugs / "Why is X failing?" | `.claude/skills/gitnexus/gitnexus-debugging/SKILL.md` |
- Access to the Gitea Maven registry (for `cameleer3-common` dependency)
- Access to the Gitea Maven registry (for `cameleer-common` dependency)
## Build
@@ -21,31 +21,36 @@ mvn clean verify # compile + run all tests (needs Docker for integrati
## Infrastructure Setup
Start PostgreSQL and OpenSearch:
Start PostgreSQL:
```bash
docker compose up -d
```
This starts TimescaleDB (PostgreSQL 16) and OpenSearch 2.19. The database schema is applied automatically via Flyway migrations on server startup.
This starts PostgreSQL 16. The database schema is applied automatically via Flyway migrations on server startup. ClickHouse tables are created by the schema initializer on startup.
The server starts on **port 8081**. The `CAMELEER_AUTH_TOKEN` environment variable is **required** — the server fails fast on startup if it is not set.
> **Note:** The Docker image no longer includes default database credentials. When running via `docker run`, pass `-e SPRING_DATASOURCE_URL=...` etc. The docker-compose setup provides these automatically.
For token rotation without downtime, set `CAMELEER_AUTH_TOKEN_PREVIOUS` to the old token while rolling out the new one. The server accepts both during the overlap window.
The server starts on **port 8081**. The `CAMELEER_SERVER_SECURITY_BOOTSTRAPTOKEN` environment variable is **required** — the server fails fast on startup if it is not set.
For token rotation without downtime, set `CAMELEER_SERVER_SECURITY_BOOTSTRAPTOKENPREVIOUS` to the old token while rolling out the new one. The server accepts both during the overlap window.
## API Endpoints
@@ -84,7 +89,7 @@ curl -s -X POST http://localhost:8081/api/v1/auth/refresh \
-d '{"refreshToken":"<refreshToken>"}'
```
UI credentials are configured via `CAMELEER_UI_USER` / `CAMELEER_UI_PASSWORD` env vars (default: `admin` / `admin`).
UI credentials are configured via `CAMELEER_SERVER_SECURITY_UIUSER` / `CAMELEER_SERVER_SECURITY_UIPASSWORD` env vars (default: `admin` / `admin`).
The env-var local user gets `ADMIN` role. Agents get `AGENT` role at registration.
**UI role gating:** The sidebar hides the Admin section for non-ADMIN users. Admin routes (`/admin/*`) redirect to `/` for non-admin. The diagram node toolbar and route control bar are hidden for VIEWER. Config is a main tab (`/config` shows all apps, `/config/:appId` filters to one app with detail panel; sidebar clicks stay on config tab, route clicks resolve to parent app). VIEWER sees read-only, OPERATOR+ can edit.
### OIDC Login (Optional)
OIDC configuration is stored in PostgreSQL and managed via the admin API or UI. The SPA checks if OIDC is available:
@@ -139,7 +146,7 @@ curl -s -X PUT http://localhost:8081/api/v1/admin/oidc \
**Initial provisioning**: OIDC can also be seeded from `CAMELEER_OIDC_*` env vars on first startup (when DB is empty). After that, the admin API takes over.
**Initial provisioning**: OIDC can also be seeded from `CAMELEER_SERVER_SECURITY_OIDC*` env vars on first startup (when DB is empty). After that, the admin API takes over.
### Authentik Setup (OIDC Provider)
### Logto Setup (OIDC Provider)
Authentik is deployed alongside the Cameleer stack. After first deployment:
Logto is deployed alongside the Cameleer stack. After first deployment:
1.**Initial setup**: Open`http://192.168.50.86:30950/if/flow/initial-setup/` and create the admin account
- Redirect URIs: `http://192.168.50.86:30090/callback` (or your UI URL)
Logto is proxy-aware via `TRUST_PROXY_HEADER=1`. The `LOGTO_ENDPOINT` and `LOGTO_ADMIN_ENDPOINT` secrets define the public-facing URLs that Logto uses for OIDC discovery, issuer URI, and redirect URLs. When behind a reverse proxy (e.g., Traefik), set these to the external URLs (e.g.,`https://auth.cameleer.my.domain`). Logto needs its own subdomain — it cannot be path-prefixed under another app.
1.**Initial setup**: Open the Logto admin console (the `LOGTO_ADMIN_ENDPOINT` URL) and create the admin account
2.**Create SPA application**: Applications → Create → Single Page App
-Name: `Cameleer UI`
- Redirect URI: your UI URL + `/oidc/callback`
- Note the **Client ID**
3.**Create API Resource**: API Resources → Create
- Name: `Cameleer Server API`
- Indicator: your API URL (e.g., `https://cameleer.siegeln.net/api`)
4.**Configure roles** (optional): Create groups in Authentik and map them to Cameleer roles via the `roles-claim` config. Default claim path is `realm_access.roles`. For Authentik, you may need to customize the OIDC scope to include group claims.
5.**Configure Cameleer**: Use the admin API (`PUT /api/v1/admin/oidc`) or set env vars for initial seeding:
5.**Configure Cameleer OIDC login**: Use the admin API (`PUT /api/v1/admin/oidc`) or the admin UI. OIDC login configuration is stored in the database — no env vars needed for the SPA OIDC flow.
CAMELEER_SERVER_SECURITY_OIDCTLSSKIPVERIFY=true # optional — skip cert verification for self-signed CAs
```
`OIDCJWKSETURI` is needed when the public issuer URL isn't reachable from inside containers — it fetches JWKS directly from the internal Logto service. `OIDCTLSSKIPVERIFY` disables certificate verification for all OIDC HTTP calls (discovery, token exchange, JWKS); use only when the provider has a self-signed CA.
### SSO Behavior
When OIDC is configured and enabled, the UI automatically redirects to the OIDC provider for silent SSO (`prompt=none`). Users with an active provider session are signed in without seeing a login form. On first login, the provider may show a consent screen (scopes), after which subsequent logins are seamless. If auto-signup is enabled, new users are automatically provisioned with the configured default roles.
- **Bypass SSO**: Navigate to `/login?local` to see the local login form
- **Subpath deployments**: The OIDC redirect_uri respects `BASE_PATH` (e.g., `https://host/server/oidc/callback`)
- **Role sync**: System roles (ADMIN/OPERATOR/VIEWER) are synced from OIDC scopes on every login — revoking a scope in the provider takes effect on next login. Manually assigned group memberships are preserved.
### User Management (ADMIN only)
@@ -221,19 +242,18 @@ curl -s -X POST http://localhost:8081/api/v1/data/metrics \
# Post application log entries (raw JSON array — no wrapper)
curl -s -X POST http://localhost:8081/api/v1/data/logs \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $TOKEN" \
-d '{
"entries": [{
"timestamp": "2026-03-25T10:00:00Z",
"level": "INFO",
"loggerName": "com.acme.MyService",
"message": "Processing order #12345",
"threadName": "main"
}]
}'
-d '[{
"timestamp": "2026-03-25T10:00:00Z",
"level": "INFO",
"loggerName": "com.acme.MyService",
"message": "Processing order #12345",
"threadName": "main",
"source": "app"
}]'
```
**Note:** The `X-Protocol-Version: 1` header is required on all `/api/v1/data/**` endpoints. Missing or wrong version returns 400.
@@ -344,45 +364,98 @@ curl -s -X POST http://localhost:8081/api/v1/agents/agent-1/commands/{commandId}
**Agent lifecycle:** LIVE (heartbeat within 90s) → STALE (missed 3 heartbeats) → DEAD (5min after STALE). DEAD agents kept indefinitely.
**Server restart resilience:** The agent registry is in-memory and lost on server restart. Agents auto-re-register on their next heartbeat or SSE connection — the server reconstructs registry entries from JWT claims (subject, application). Route catalog uses ClickHouse execution data as fallback until agents re-register with full route IDs. Agents should also handle 404 on heartbeat by triggering a full re-registration.
**SSE events:** `config-update`, `deep-trace`, `replay`, `route-control` commands pushed in real time. Server sends ping keepalive every 15s.
**Command expiry:** Unacknowledged commands expire after 60 seconds.
**Route control responses:** Route control commands return `CommandGroupResponse` with per-agent status, response count, and timed-out agent IDs.
### Backpressure
When the write buffer is full (default capacity: 50,000), ingestion endpoints return **503 Service Unavailable**. Already-buffered data is not lost.
## Configuration
Key settings in `cameleer3-server-app/src/main/resources/application.yml`:
Key settings in `cameleer-server-app/src/main/resources/application.yml`. All custom properties live under `cameleer.server.*`. Env vars are a mechanical 1:1 mapping (dots to underscores, uppercase).
| Setting | Default | Description |
|---------|---------|-------------|
| `server.port` | 8081 | Server port |
| `ingestion.buffer-capacity` | 50000 | Max items in write buffer |
| `ingestion.batch-size` | 5000 | Items per batch insert |
**Note:** OIDC *login* configuration (issuer, client ID, client secret, roles claim, default roles) is stored in the database and managed via the admin API (`PUT /api/v1/admin/oidc`) or admin UI. The env vars above are for resource server mode (M2M token validation) only.
**Ingestion** (`cameleer.server.ingestion.*`):
| Setting | Default | Env var | Description |
|---------|---------|---------|-------------|
| `cameleer.server.ingestion.buffercapacity` | `50000` | `CAMELEER_SERVER_INGESTION_BUFFERCAPACITY` | Max items in write buffer |
| `cameleer.server.agentregistry.stalethresholdms` | `90000` | `CAMELEER_SERVER_AGENTREGISTRY_STALETHRESHOLDMS` | Time before agent marked STALE (ms) |
| `cameleer.server.agentregistry.deadthresholdms` | `300000` | `CAMELEER_SERVER_AGENTREGISTRY_DEADTHRESHOLDMS` | Time after STALE before DEAD (ms) |
| `cameleer.server.runtime.container.cpushares` | `512` | `CAMELEER_SERVER_RUNTIME_CONTAINER_CPUSHARES` | Default CPU shares for app containers |
**Other** (`cameleer.server.*`):
| Setting | Default | Env var | Description |
|---------|---------|---------|-------------|
| `cameleer.server.catalog.discoveryttldays` | `7` | `CAMELEER_SERVER_CATALOG_DISCOVERYTTLDAYS` | Days before stale discovered apps auto-hide from sidebar |
Push to `main` triggers: **build** (UI npm + Maven, unit tests) → **docker** (buildx amd64 for server + UI, push to Gitea registry) → **deploy** (kubectl apply + rolling update).
The Cameleer3 dashboard has a **distinctive, well-crafted warm amber design language** that stands out in the observability space. The core monitoring pages (Dashboard, Exchange Detail, Routes, Agents) are polished and consistent. The design system provides a solid foundation.
The Cameleer dashboard has a **distinctive, well-crafted warm amber design language** that stands out in the observability space. The core monitoring pages (Dashboard, Exchange Detail, Routes, Agents) are polished and consistent. The design system provides a solid foundation.
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.