docs: add sensitive keys feature documentation

- CLAUDE.md: add SensitiveKeysConfig, SensitiveKeysRepository, SensitiveKeysMerger to core admin classes; add SensitiveKeysAdminController endpoint; add PostgresSensitiveKeysRepository; add sensitive keys convention; add admin page to UI structure - Design spec and implementation plan for the feature Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 18:29:15 +02:00
parent 7b73b5c9c5
commit 891abbfcfd
3 changed files with 1623 additions and 2 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -67,6 +67,11 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
 - `SystemRole` — enum: AGENT, VIEWER, OPERATOR, ADMIN; `normalizeScope()` maps scopes
 - `UserDetail`, `RoleDetail`, `GroupDetail` — records

+**admin/** — Server-wide admin config
+- `SensitiveKeysConfig` — record: keys (List<String>, immutable)
+- `SensitiveKeysRepository` — interface: find(), save()
+- `SensitiveKeysMerger` — pure function: merge(global, perApp) -> union with case-insensitive dedup, preserves first-seen casing. Returns null when both inputs null.
+
 **security/** — Auth
 - `JwtService` — interface: createAccessToken, validateAccessToken
 - `Ed25519SigningService` — interface: sign, verify (config signing)
@@ -95,6 +100,7 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
 - `RoleAdminController` — CRUD /api/v1/admin/roles
 - `GroupAdminController` — CRUD /api/v1/admin/groups
 - `OidcConfigAdminController` — GET/POST /api/v1/admin/oidc, POST /test
+- `SensitiveKeysAdminController` — GET/PUT /api/v1/admin/sensitive-keys. GET returns 200 with config or 204 if not configured. PUT accepts `{ keys: [...] }` with optional `?pushToAgents=true` to fan out merged keys to all LIVE agents. Stored in `server_config` table (key `sensitive_keys`).
 - `AuditLogController` — GET /api/v1/admin/audit
 - `MetricsController` — GET /api/v1/metrics, GET /timeseries
 - `DiagramController` — GET /api/v1/diagrams/{id}, POST /
@@ -117,7 +123,7 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
 - `PostgresAppRepository`, `PostgresAppVersionRepository`, `PostgresEnvironmentRepository`
 - `PostgresDeploymentRepository` — includes JSONB replica_states, deploy_stage, findByContainerId
 - `PostgresUserRepository`, `PostgresRoleRepository`, `PostgresGroupRepository`
- `PostgresAuditRepository`, `PostgresOidcConfigRepository`, `PostgresClaimMappingRepository`
+- `PostgresAuditRepository`, `PostgresOidcConfigRepository`, `PostgresClaimMappingRepository`, `PostgresSensitiveKeysRepository`

 **storage/** — ClickHouse stores
 - `ClickHouseExecutionStore`, `ClickHouseMetricsStore`, `ClickHouseLogStore`
@@ -163,6 +169,7 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
 - Security: JWT auth with RBAC (AGENT/VIEWER/OPERATOR/ADMIN roles), Ed25519 config signing (key derived deterministically from JWT secret via HMAC-SHA256), bootstrap token for registration. CORS: `CAMELEER_SERVER_SECURITY_CORSALLOWEDORIGINS` (comma-separated) overrides `CAMELEER_SERVER_SECURITY_UIORIGIN` for multi-origin setups (e.g., reverse proxy). Infrastructure access: `CAMELEER_SERVER_SECURITY_INFRASTRUCTUREENDPOINTS=false` disables Database and ClickHouse admin endpoints (set by SaaS provisioner on tenant servers). Health endpoint exposes the flag for UI tab visibility. UI role gating: Admin sidebar/routes hidden for non-ADMIN; diagram toolbar and route control hidden for VIEWER. Read-only for VIEWER, editable for OPERATOR+. Role helpers: `useIsAdmin()`, `useCanControl()` in `auth-store.ts`. Route guard: `RequireAdmin` in `auth/RequireAdmin.tsx`. Last-ADMIN guard: system prevents removal of the last ADMIN role (409 Conflict on role removal, user deletion, group role removal). Password policy: min 12 chars, 3-of-4 character classes, no username match (enforced on user creation and admin password reset). Brute-force protection: 5 failed attempts -> 15 min lockout (tracked via `failed_login_attempts` / `locked_until` on users table). Token revocation: `token_revoked_before` column on users, checked in `JwtAuthenticationFilter`, set on password change.
 - OIDC: Optional external identity provider support (token exchange pattern). Configured via admin API/UI, stored in database (`server_config` table). Configurable `userIdClaim` (default `sub`) determines which id_token claim is used as the user identifier. Resource server mode: accepts external access tokens (Logto M2M) via JWKS validation when `CAMELEER_SERVER_SECURITY_OIDCISSUERURI` is set. `CAMELEER_SERVER_SECURITY_OIDCJWKSETURI` overrides JWKS discovery for container networking. `CAMELEER_SERVER_SECURITY_OIDCTLSSKIPVERIFY=true` disables TLS cert verification for OIDC calls (self-signed CAs). Scope-based role mapping via `SystemRole.normalizeScope()` (case-insensitive, strips `server:` prefix): `admin`/`server:admin` -> ADMIN, `operator`/`server:operator` -> OPERATOR, `viewer`/`server:viewer` -> VIEWER. SSO: when OIDC enabled, UI auto-redirects to provider with `prompt=none` for silent sign-in; falls back to `/login?local` on `login_required`, retries without `prompt=none` on `consent_required`. Logout always redirects to `/login?local` (via OIDC end_session or direct fallback) to prevent SSO re-login loops. Auto-signup provisions new OIDC users with default roles. System roles synced on every OIDC login via `syncOidcRoles` — always overwrites directly-assigned roles (falls back to `defaultRoles` when OIDC returns none); uses `getDirectRolesForUser` to avoid touching group-inherited roles. Group memberships are never touched. Supports ES384, ES256, RS256. Shared OIDC logic in `OidcProviderHelper` (discovery, JWK source, algorithm set).
 - OIDC role extraction: `OidcTokenExchanger` reads roles from the **access_token** first (JWT with `at+jwt` type, decoded by a separate processor), then falls back to id_token. `OidcConfig` includes `audience` (RFC 8707 resource indicator — included in both authorization request and token exchange POST body to trigger JWT access tokens) and `additionalScopes` (extra scopes for the SPA to request). The `rolesClaim` config points to the claim name in the token (e.g., `"roles"` for Custom JWT claims, `"realm_access.roles"` for Keycloak). All provider-specific configuration is external — no provider-specific code in the server.
+- Sensitive keys: Global enforced baseline for masking sensitive data in agent payloads. Admin configures via `PUT /api/v1/admin/sensitive-keys` (stored in `server_config` table, key `sensitive_keys`). Per-app additions stored in `ApplicationConfig.sensitiveKeys`. Merge rule: `final = global UNION per-app` (case-insensitive dedup, per-app can only add, never remove global keys). When no config exists, agents use built-in defaults. `ApplicationConfigController.getConfig()` returns `AppConfigResponse` wrapping config with `globalSensitiveKeys` and `mergedSensitiveKeys` for UI rendering. Config-update SSE payloads carry the merged list. SaaS propagation: platform calls the same admin API on each tenant server (no special protocol).
 - User persistence: PostgreSQL `users` table, admin CRUD at `/api/v1/admin/users`
 - Usage analytics: ClickHouse `usage_events` table tracks authenticated UI requests, flushed every 5s

@@ -211,6 +218,9 @@ The UI has 4 main tabs: **Exchanges**, **Dashboard**, **Runtime**, **Deployments
  - Create app: full page at `/apps/new` (not a modal)
  - Deployment progress: `ui/src/components/DeploymentProgress.tsx` (7-stage step indicator)

+**Admin pages** (ADMIN-only, under `/admin/`):
+- **Sensitive Keys** (`ui/src/pages/Admin/SensitiveKeysPage.tsx`) — global sensitive key masking config with tag/pill editor, push-to-agents toggle. Per-app additions shown in `AppConfigDetailPage.tsx` with read-only global pills (greyed Badge) + editable per-app pills (Tag with remove).
+
 ### Key UI Files

 - `ui/src/router.tsx` — React Router v6 routes
@@ -381,7 +391,7 @@ Mean processing time = `camel.route.policy.total_time / camel.route.policy.count
 <!-- gitnexus:start -->
 # GitNexus — Code Intelligence

-This project is indexed by GitNexus as **cameleer3-server** (6027 symbols, 15299 relationships, 300 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.
+This project is indexed by GitNexus as **cameleer3-server** (6155 symbols, 15501 relationships, 300 execution flows). Use the GitNexus MCP tools to understand code, assess impact, and navigate safely.

 > If any GitNexus tool warns the index is stale, run `npx gitnexus analyze` in terminal first.

--- a/docs/superpowers/plans/2026-04-14-sensitive-keys-server.md
+++ b/docs/superpowers/plans/2026-04-14-sensitive-keys-server.md
--- a/docs/superpowers/specs/2026-04-14-sensitive-keys-server-design.md
+++ b/docs/superpowers/specs/2026-04-14-sensitive-keys-server-design.md
@@ -0,0 +1,222 @@
+# Sensitive Keys Server-Side Support
+
+**Date:** 2026-04-14
+**Status:** Approved
+
+## Context
+
+The agent team is unifying `sensitiveHeaders` and `sensitiveProperties` into a single `sensitiveKeys` field on `ApplicationConfig` (see agent spec: "Sensitive Keys Unification + SSE Support + Pattern Matching"). The server must store, merge, and push these keys to agents.
+
+Key requirements beyond the agent contract:
+- Global enforced baseline that admins control
+- Per-app additions (cannot weaken the global baseline)
+- Immediate fan-out option when global keys change
+- SaaS tenant admins configure via the same REST API (no special protocol)
+
+## Design Decisions
+
+| Decision | Choice | Rationale |
+|----------|--------|-----------|
+| Override hierarchy | Global enforced + per-app additive | Security: devs cannot accidentally un-mask patterns |
+| Merge semantics | Union only, no remove | Simplicity; weakening the baseline is not a valid use case |
+| Server defaults | None (null until configured) | Agents have built-in defaults; avoids drift between server and agent hardcoded lists |
+| Fan-out on global change | Admin chooses via `pushToAgents` param | Flexibility; some admins want to batch changes |
+| SaaS propagation | SaaS platform calls the same admin API | No special SaaS-aware code in the server |
+| Storage | Existing `server_config` table | Follows OIDC and thresholds pattern |
+
+## Merge Rule
+
+```
+final_keys = global_keys UNION per_app_keys
+```
+
+- Global keys are always included (enforced baseline)
+- Per-app keys are additive only
+- Deduplication is case-insensitive (preserves the casing of the first occurrence)
+- If no global config exists AND no per-app keys exist, `sensitiveKeys` is omitted from the payload (agents use built-in defaults)
+- If global config exists but is an empty list `[]`, the server sends `[]` (agents mask nothing — explicit opt-out)
+
+## API
+
+### Global Sensitive Keys (new)
+
+**GET /api/v1/admin/sensitive-keys**
+- Auth: ADMIN only
+- Returns the stored global keys or `null` if not configured
+- Response:
+  ```json
+  {
+    "keys": ["Authorization", "Cookie", "Set-Cookie", "*password*", "*secret*"]
+  }
+  ```
+  Or `null` body (204 No Content) when not configured.
+
+**PUT /api/v1/admin/sensitive-keys?pushToAgents=false**
+- Auth: ADMIN only
+- Request body:
+  ```json
+  {
+    "keys": ["Authorization", "Cookie", "Set-Cookie", "*password*", "*secret*"]
+  }
+  ```
+- `pushToAgents` query param (boolean, default `false`):
+  - `false`: saves only, new keys take effect on each app's next config push
+  - `true`: saves, then fans out config-update to all LIVE agents (recomputes merged list per app)
+- Response: saved config + optional push results
+  ```json
+  {
+    "keys": ["Authorization", "Cookie", "Set-Cookie", "*password*", "*secret*"],
+    "pushResult": { ... }
+  }
+  ```
+  `pushResult` is `null` when `pushToAgents=false`.
+
+### Per-App Config (existing endpoint, enhanced behavior)
+
+**GET /api/v1/config/{application}**
+- `sensitiveKeys`: the **per-app additions only** (what the UI edits and PUTs back)
+- `globalSensitiveKeys`: read-only field, the current global baseline (for UI to render greyed-out pills)
+- `mergedSensitiveKeys`: the full merged list (global + per-app) — informational, shows what agents actually receive
+- Note: the agent-facing SSE payload uses `sensitiveKeys` for the merged list (agents don't need to know the split)
+
+**PUT /api/v1/config/{application}**
+- `sensitiveKeys` in the request body represents **per-app additions only** (not the merged list)
+- `globalSensitiveKeys` and `mergedSensitiveKeys` are ignored if sent in the PUT body
+- On save: only per-app `sensitiveKeys` are persisted
+- On push: server merges global + per-app into the `sensitiveKeys` field of the SSE payload sent to agents
+
+## Storage
+
+No new tables. Uses existing PostgreSQL schema.
+
+### Global Keys
+
+```sql
+-- server_config table
+-- config_key = 'sensitive_keys'
+-- config_val example:
+{ "keys": ["Authorization", "Cookie", "Set-Cookie", "*password*", "*secret*"] }
+```
+
+### Per-App Keys
+
+Stored in `application_config.config_val` as part of the existing `ApplicationConfig` JSONB. The `sensitiveKeys` field (added by the agent team in `cameleer3-common`) stores only the per-app additions.
+
+## Fan-Out on Global Change
+
+When admin PUTs global keys with `pushToAgents=true`:
+
+1. Save to `server_config`
+2. Collect all distinct applications from `application_config` table + live agents in registry
+3. For each application:
+   a. Load per-app config (or defaults if none)
+   b. Merge global + per-app keys
+   c. Set merged keys on the ApplicationConfig
+   d. Push via existing `pushConfigToAgents()` mechanism
+4. Return aggregate push results (total apps, total agents, per-app response summary)
+5. Audit log entry with results
+
+## Server Code Changes
+
+### New Files
+
+| File | Module | Purpose |
+|------|--------|---------|
+| `SensitiveKeysConfig.java` | core | `record SensitiveKeysConfig(List<String> keys)` |
+| `SensitiveKeysRepository.java` | core | Interface: `find()`, `save()` |
+| `SensitiveKeysMerger.java` | core | Pure function: `merge(List<String> global, List<String> perApp) -> List<String>`. Union, case-insensitive dedup, preserves first-seen casing. |
+| `PostgresSensitiveKeysRepository.java` | app/storage | Read/write `server_config` key `"sensitive_keys"` (follows `PostgresThresholdRepository` pattern) |
+| `SensitiveKeysAdminController.java` | app/controller | GET/PUT `/api/v1/admin/sensitive-keys`, fan-out logic, audit logging |
+
+### Modified Files
+
+| File | Change |
+|------|--------|
+| `ApplicationConfigController.java` | Inject `SensitiveKeysRepository`. On GET: merge global keys into response, add `globalSensitiveKeys` field. On PUT: merge before SSE push. |
+| `StorageBeanConfig.java` | Wire `PostgresSensitiveKeysRepository` bean |
+
+### No Schema Migration Required
+
+Uses existing `server_config` table (JSONB key-value store). The `sensitiveKeys` field on `ApplicationConfig` is added by the agent team in `cameleer3-common` — the server just reads/writes it as part of the existing JSONB blob.
+
+## Audit
+
+| Action | Category | Detail |
+|--------|----------|--------|
+| `view_sensitive_keys` | CONFIG | (none) |
+| `update_sensitive_keys` | CONFIG | `{ keys: [...], pushToAgents: true/false, appsPushed: N, totalAgents: N }` |
+
+Per-app changes are covered by the existing `update_app_config` audit entry.
+
+## UI
+
+### Global Sensitive Keys Admin Page
+
+- **Location:** Admin sidebar, new entry "Sensitive Keys"
+- **Access:** ADMIN only (sidebar entry hidden for non-ADMIN)
+- **Components:**
+  - Info banner at top: "Agents ship with built-in defaults (Authorization, Cookie, Set-Cookie, X-API-Key, X-Auth-Token, Proxy-Authorization). Configuring keys here replaces agent defaults for all applications. Leave unconfigured to use agent defaults."
+  - Tag/pill editor for the keys list. Type a key or glob pattern, press Enter to add as a pill. Each pill has an X to remove. Supports glob patterns (`*password*`, `X-Internal-*`).
+  - "Push to all connected agents immediately" toggle (default off)
+  - Save button
+- **Empty state:** Info banner + empty editor. Clear that agents use their own defaults.
+
+### Per-App Sensitive Keys (existing app config page)
+
+- **Location:** Within the existing per-app config editor, new section "Additional Sensitive Keys"
+- **Components:**
+  - Read-only pills showing current global keys (greyed out, no X button, visually distinct)
+  - Editable tag/pill editor for per-app additions (normal styling, X to remove)
+  - Info note: "Global keys (shown in grey) are enforced by your administrator and cannot be removed. Add application-specific keys below."
+- **When no global keys configured:** Section shows only the editable per-app editor with a note: "No global sensitive keys configured. Agents use their built-in defaults."
+
+## SaaS Integration
+
+No server-side changes needed for SaaS. The SaaS platform propagates tenant-level sensitive keys by calling the standard admin API:
+
+```
+PUT https://{tenant-server}/api/v1/admin/sensitive-keys?pushToAgents=true
+Authorization: Bearer {platform-admin-token}
+{
+  "keys": ["Authorization", "Cookie", "*password*", "*secret*"]
+}
+```
+
+Each tenant server handles merge + fan-out to its own agents independently.
+
+## Sequence Diagrams
+
+### Admin Updates Global Keys (with push)
+
+```
+Admin UI                    Server                      Agents
+   │                          │                           │
+   │ PUT /admin/sensitive-keys│                           │
+   │  { keys: [...] }        │                           │
+   │  ?pushToAgents=true      │                           │
+   │─────────────────────────>│                           │
+   │                          │ save to server_config     │
+   │                          │                           │
+   │                          │ for each app:             │
+   │                          │   merge(global, per-app)  │
+   │                          │   CONFIG_UPDATE SSE ──────>│
+   │                          │                    ACK <───│
+   │                          │                           │
+   │ { keys, pushResult }    │                           │
+   │<─────────────────────────│                           │
+```
+
+### Agent Startup
+
+```
+Agent                       Server
+  │                           │
+  │ GET /config/{app}         │
+  │──────────────────────────>│
+  │                           │ load per-app config
+  │                           │ load global sensitive keys
+  │                           │ merge(global, per-app)
+  │                           │
+  │ { ..., sensitiveKeys: [merged] }
+  │<──────────────────────────│
+```