docs: address spec review feedback for infrastructure overview
- Document SearchIndexerStats interface and required SearchIndexer changes - Add @EnableMethodSecurity prerequisite and retrofit of existing controllers - Limit audit log free-text search to indexed text columns (not JSONB) - Split migrations into V9 (thresholds) and V10 (audit_log) - Add user_agent field to audit records for SOC2 forensics - Add thresholds validation rules, pagination limits, error response shapes - Clarify SPA forwarding, single-row pattern, OpenSearch client reuse - Add audit log retention note for Phase 2 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -45,7 +45,7 @@ The gear icon expands/collapses an admin sub-menu in the sidebar:
|
|||||||
OpenSearch → /admin/opensearch
|
OpenSearch → /admin/opensearch
|
||||||
Audit Log → /admin/audit
|
Audit Log → /admin/audit
|
||||||
OIDC → /admin/oidc
|
OIDC → /admin/oidc
|
||||||
Users → (future)
|
Users → /admin/users (backend exists, UI page is future scope)
|
||||||
```
|
```
|
||||||
|
|
||||||
- Admin section visible only to users with `ADMIN` role
|
- Admin section visible only to users with `ADMIN` role
|
||||||
@@ -121,9 +121,17 @@ The gear icon expands/collapses an admin sub-menu in the sidebar:
|
|||||||
- Visual bar showing queue depth vs. max queue size
|
- Visual bar showing queue depth vs. max queue size
|
||||||
- Metrics: queue depth, failed document count, debounce interval, indexing rate (docs/s), time since last indexed
|
- Metrics: queue depth, failed document count, debounce interval, indexing rate (docs/s), time since last indexed
|
||||||
- Status badge based on configurable thresholds
|
- Status badge based on configurable thresholds
|
||||||
- Source: `SearchIndexer` internal stats (exposed via `SearchIndexerStats` interface)
|
- Source: `SearchIndexer` internal stats, exposed via a new `SearchIndexerStats` interface in `cameleer3-server-core`
|
||||||
- **Auto-refreshes every 15 seconds**
|
- **Auto-refreshes every 15 seconds**
|
||||||
|
|
||||||
|
**Implementation note:** `SearchIndexer` currently has no stats API. This requires adding:
|
||||||
|
- `AtomicLong failedCount` — incremented on indexing errors
|
||||||
|
- `AtomicLong indexedCount` — incremented on successful index operations
|
||||||
|
- `volatile Instant lastIndexedAt` — updated after each successful batch
|
||||||
|
- Rate calculation via a sliding window counter (e.g., count delta over last 15s interval)
|
||||||
|
- `SearchIndexerStats` interface in core module with getters for all above, implemented by `SearchIndexer`
|
||||||
|
- `queueDepth` and `maxQueueSize` already derivable from the internal `BlockingQueue`
|
||||||
|
|
||||||
### Indices Section
|
### Indices Section
|
||||||
|
|
||||||
- **Search/filter** by index name pattern (text input)
|
- **Search/filter** by index name pattern (text input)
|
||||||
@@ -193,7 +201,7 @@ Database-backed audit trail of all administrative actions across the system. Pro
|
|||||||
```
|
```
|
||||||
|
|
||||||
- **Filterable** by user, category, date range
|
- **Filterable** by user, category, date range
|
||||||
- **Searchable** by free text (matches action, target, detail)
|
- **Searchable** by free text (matches `action` and `target` columns only — JSONB `detail` excluded from text search for performance)
|
||||||
- **Sortable** by timestamp (default: newest first)
|
- **Sortable** by timestamp (default: newest first)
|
||||||
- **Pagination** — 25 per page, server-side
|
- **Pagination** — 25 per page, server-side
|
||||||
- **Detail expansion** — click a row to expand and show full `detail` JSON
|
- **Detail expansion** — click a row to expand and show full `detail` JSON
|
||||||
@@ -230,14 +238,17 @@ Every admin action across the system, not just infrastructure pages:
|
|||||||
| `detail` | JSONB with action-specific context (e.g., query text for killed query, old/new roles for role change) |
|
| `detail` | JSONB with action-specific context (e.g., query text for killed query, old/new roles for role change) |
|
||||||
| `result` | `SUCCESS` or `FAILURE` |
|
| `result` | `SUCCESS` or `FAILURE` |
|
||||||
| `ip_address` | Client IP address from the request |
|
| `ip_address` | Client IP address from the request |
|
||||||
|
| `user_agent` | Browser/client identification from request header (SOC2 forensics) |
|
||||||
|
|
||||||
### Backend Implementation
|
### Backend Implementation
|
||||||
|
|
||||||
- `AuditService` — central service injected into all admin controllers
|
- `AuditService` — central service in `cameleer3-server-core`, injected into all admin controllers via direct method calls (no AOP/interceptor — consistent with existing controller style)
|
||||||
- Single method: `log(action, category, target, detail, result)`
|
- Primary method: `log(action, category, target, detail, result)` — extracts username and IP from `SecurityContextHolder` and `HttpServletRequest`
|
||||||
- Extracts username and IP from `SecurityContextHolder` and `HttpServletRequest`
|
- Overloaded method for pre-auth contexts: `log(username, action, category, target, detail, result, request)` — used by auth controllers where `SecurityContext` is not yet populated (login success/failure)
|
||||||
|
- Captures `user_agent` from `HttpServletRequest` header
|
||||||
- Writes to both the `audit_log` table AND SLF4J (belt and suspenders)
|
- Writes to both the `audit_log` table AND SLF4J (belt and suspenders)
|
||||||
- Async write option not used — audit must be synchronous for compliance guarantees
|
- Async write option not used — audit must be synchronous for compliance guarantees
|
||||||
|
- Retrofit into existing controllers: add `auditService.log(...)` calls to `OidcConfigAdminController` (save, delete, test) and `UserAdminController` (update roles, delete user), and auth controllers (login, OIDC login, logout, failed login)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -265,6 +276,8 @@ All endpoints under `/api/v1/admin/` — secured by existing Spring Security fil
|
|||||||
| `DELETE` | `/admin/opensearch/indices/{name}` | Delete specific index (with audit log) |
|
| `DELETE` | `/admin/opensearch/indices/{name}` | Delete specific index (with audit log) |
|
||||||
| `GET` | `/admin/opensearch/performance` | Cache rates, latencies, JVM heap |
|
| `GET` | `/admin/opensearch/performance` | Cache rates, latencies, JVM heap |
|
||||||
|
|
||||||
|
**OpenSearch client:** All admin OpenSearch endpoints reuse the existing `OpenSearchClient` bean configured in `OpenSearchConfig.java`. No separate client or credentials needed — the admin endpoints call cluster-level APIs (`_cluster/health`, `_cat/indices`, `_nodes/stats`) using the same connection.
|
||||||
|
|
||||||
#### Indices Query Parameters
|
#### Indices Query Parameters
|
||||||
|
|
||||||
| Param | Type | Default | Description |
|
| Param | Type | Default | Description |
|
||||||
@@ -288,7 +301,7 @@ All endpoints under `/api/v1/admin/` — secured by existing Spring Security fil
|
|||||||
|-------|------|---------|-------------|
|
|-------|------|---------|-------------|
|
||||||
| `username` | string | — | Filter by username |
|
| `username` | string | — | Filter by username |
|
||||||
| `category` | enum | — | Filter by category: INFRA, AUTH, USER_MGMT, CONFIG |
|
| `category` | enum | — | Filter by category: INFRA, AUTH, USER_MGMT, CONFIG |
|
||||||
| `search` | string | — | Free text search across action, target, detail |
|
| `search` | string | — | Free text search across `action` and `target` columns (not JSONB `detail`) |
|
||||||
| `from` | ISO date | 7 days ago | Start of date range |
|
| `from` | ISO date | 7 days ago | Start of date range |
|
||||||
| `to` | ISO date | now | End of date range |
|
| `to` | ISO date | now | End of date range |
|
||||||
| `sort` | string | `timestamp` | Sort field |
|
| `sort` | string | `timestamp` | Sort field |
|
||||||
@@ -326,6 +339,37 @@ All endpoints under `/api/v1/admin/` — secured by existing Spring Security fil
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### Thresholds Validation Rules
|
||||||
|
|
||||||
|
- Warning must be <= critical for all numeric threshold pairs
|
||||||
|
- Percentage values must be 0–100
|
||||||
|
- Duration values must be > 0
|
||||||
|
- `clusterHealthWarning` must be less severe than `clusterHealthCritical` (GREEN < YELLOW < RED)
|
||||||
|
- Backend returns 400 Bad Request with field-level error messages on validation failure
|
||||||
|
|
||||||
|
### Pagination Limits
|
||||||
|
|
||||||
|
- All paginated endpoints enforce a maximum page size of 100 (`min(requested, 100)`)
|
||||||
|
- Applies to: indices listing, audit log
|
||||||
|
|
||||||
|
### Error Responses
|
||||||
|
|
||||||
|
All new endpoints return errors in a consistent shape:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": 404,
|
||||||
|
"error": "Not Found",
|
||||||
|
"message": "No active query with PID 12345"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
Specific error cases:
|
||||||
|
- `POST /admin/database/queries/{pid}/kill` — 404 if PID not found, 500 if `pg_terminate_backend` fails
|
||||||
|
- `DELETE /admin/opensearch/indices/{name}` — 404 if index not found, 502 if OpenSearch unreachable
|
||||||
|
- `GET /admin/database/status` — returns 200 with `"connected": false` if database is unreachable (not 503), so the frontend can render a red status badge rather than an error state
|
||||||
|
- `GET /admin/opensearch/status` — returns 200 with `"clusterHealth": "UNREACHABLE"` if OpenSearch is down
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## 6. Security
|
## 6. Security
|
||||||
@@ -333,8 +377,9 @@ All endpoints under `/api/v1/admin/` — secured by existing Spring Security fil
|
|||||||
### Enforcement Layers
|
### Enforcement Layers
|
||||||
|
|
||||||
1. **Spring Security filter chain** — `/api/v1/admin/**` requires `ROLE_ADMIN` (existing configuration)
|
1. **Spring Security filter chain** — `/api/v1/admin/**` requires `ROLE_ADMIN` (existing configuration)
|
||||||
2. **Controller annotation** — `@PreAuthorize("hasRole('ADMIN')")` on each controller class (defense-in-depth)
|
2. **Controller annotation** — `@PreAuthorize("hasRole('ADMIN')")` on each controller class (defense-in-depth). This is a new convention — existing controllers (`OidcConfigAdminController`, `UserAdminController`) must be retrofitted with this annotation as part of Phase 1.
|
||||||
3. **UI role check** — sidebar admin section hidden for non-admin users (cosmetic only, not a security boundary)
|
3. **`@EnableMethodSecurity`** — must be added to `SecurityConfig.java` to activate `@PreAuthorize` processing (prerequisite for layer 2)
|
||||||
|
4. **UI role check** — sidebar admin section hidden for non-admin users (cosmetic only, not a security boundary)
|
||||||
|
|
||||||
### Audit Logging
|
### Audit Logging
|
||||||
|
|
||||||
@@ -353,7 +398,7 @@ Design anticipates a read-only `OPERATOR` role:
|
|||||||
|
|
||||||
## 7. Data Storage
|
## 7. Data Storage
|
||||||
|
|
||||||
### New Flyway Migration: V9
|
### Flyway Migration V9: Admin Thresholds
|
||||||
|
|
||||||
```sql
|
```sql
|
||||||
CREATE TABLE admin_thresholds (
|
CREATE TABLE admin_thresholds (
|
||||||
@@ -363,7 +408,15 @@ CREATE TABLE admin_thresholds (
|
|||||||
updated_by TEXT NOT NULL,
|
updated_by TEXT NOT NULL,
|
||||||
CONSTRAINT single_row CHECK (id = 1)
|
CONSTRAINT single_row CHECK (id = 1)
|
||||||
);
|
);
|
||||||
|
```
|
||||||
|
|
||||||
|
- Single-row table using `CHECK (id = 1)` constraint — stricter than the `oidc_config` pattern (which uses a text PK defaulting to `'default'` without a constraint). The CHECK approach is preferred going forward as it explicitly prevents multiple rows.
|
||||||
|
- JSON column for flexibility — adding new thresholds doesn't require schema changes
|
||||||
|
- Tracks who last updated and when
|
||||||
|
|
||||||
|
### Flyway Migration V10: Audit Log
|
||||||
|
|
||||||
|
```sql
|
||||||
CREATE TABLE audit_log (
|
CREATE TABLE audit_log (
|
||||||
id BIGSERIAL PRIMARY KEY,
|
id BIGSERIAL PRIMARY KEY,
|
||||||
timestamp TIMESTAMPTZ NOT NULL DEFAULT now(),
|
timestamp TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||||
@@ -373,24 +426,24 @@ CREATE TABLE audit_log (
|
|||||||
target TEXT,
|
target TEXT,
|
||||||
detail JSONB,
|
detail JSONB,
|
||||||
result TEXT NOT NULL,
|
result TEXT NOT NULL,
|
||||||
ip_address TEXT
|
ip_address TEXT,
|
||||||
|
user_agent TEXT
|
||||||
);
|
);
|
||||||
|
|
||||||
CREATE INDEX idx_audit_log_timestamp ON audit_log (timestamp DESC);
|
CREATE INDEX idx_audit_log_timestamp ON audit_log (timestamp DESC);
|
||||||
CREATE INDEX idx_audit_log_username ON audit_log (username);
|
CREATE INDEX idx_audit_log_username ON audit_log (username);
|
||||||
CREATE INDEX idx_audit_log_category ON audit_log (category);
|
CREATE INDEX idx_audit_log_category ON audit_log (category);
|
||||||
|
CREATE INDEX idx_audit_log_action ON audit_log (action);
|
||||||
|
CREATE INDEX idx_audit_log_target ON audit_log (target);
|
||||||
```
|
```
|
||||||
|
|
||||||
**admin_thresholds:**
|
- Separate migration from thresholds so they can be developed and tested independently
|
||||||
- Single-row table (same pattern as `oidc_config`)
|
|
||||||
- JSON column for flexibility — adding new thresholds doesn't require schema changes
|
|
||||||
- Tracks who last updated and when
|
|
||||||
|
|
||||||
**audit_log:**
|
|
||||||
- Append-only table — no UPDATE or DELETE exposed via API
|
- Append-only table — no UPDATE or DELETE exposed via API
|
||||||
- Indexed on timestamp (primary query axis), username, and category for filtered views
|
- Indexed on timestamp (primary query axis), username, category, action, and target for filtered views and free-text search via `ILIKE` on indexed text columns
|
||||||
- JSONB `detail` column holds action-specific context without schema changes
|
- JSONB `detail` column holds action-specific context without schema changes (not searched via free text — use row expansion for detail inspection)
|
||||||
|
- `user_agent` field captures client identification for forensic analysis (SOC2)
|
||||||
- No foreign key to `users` table — username is denormalized so audit records survive user deletion
|
- No foreign key to `users` table — username is denormalized so audit records survive user deletion
|
||||||
|
- **Retention:** unbounded in Phase 1. Phase 2+ should add a retention/archival strategy (e.g., TimescaleDB hypertable with retention policy, or periodic archive to cold storage). Typical SOC2 retention is 7 years.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -417,7 +470,7 @@ CREATE INDEX idx_audit_log_category ON audit_log (category);
|
|||||||
|------|--------|
|
|------|--------|
|
||||||
| `components/layout/AppSidebar.tsx` | Refactor admin section to collapsible sub-menu with multiple items |
|
| `components/layout/AppSidebar.tsx` | Refactor admin section to collapsible sub-menu with multiple items |
|
||||||
| `router.tsx` | Add routes for `/admin/database`, `/admin/opensearch`, `/admin/audit`, redirect `/admin` |
|
| `router.tsx` | Add routes for `/admin/database`, `/admin/opensearch`, `/admin/audit`, redirect `/admin` |
|
||||||
| `SpaForwardController.java` | Ensure `/admin/*` forwarding covers new routes |
|
| `SpaForwardController.java` | Existing `/admin/{path:[^\\.]*}` pattern already covers single-segment routes — no change needed unless deeper routes are added |
|
||||||
|
|
||||||
### Auto-Refresh Strategy
|
### Auto-Refresh Strategy
|
||||||
|
|
||||||
@@ -438,7 +491,9 @@ CREATE INDEX idx_audit_log_category ON audit_log (category);
|
|||||||
5. Audit log — database-backed audit trail + admin viewer page
|
5. Audit log — database-backed audit trail + admin viewer page
|
||||||
6. Retrofit audit logging into existing admin controllers (OIDC, user management) and auth flow
|
6. Retrofit audit logging into existing admin controllers (OIDC, user management) and auth flow
|
||||||
7. Backend endpoints with RBAC enforcement
|
7. Backend endpoints with RBAC enforcement
|
||||||
8. Flyway migration V9 for thresholds + audit_log tables
|
8. Flyway migrations V9 (thresholds) and V10 (audit_log)
|
||||||
|
9. `SearchIndexerStats` interface and `SearchIndexer` stats instrumentation
|
||||||
|
10. `@EnableMethodSecurity` + `@PreAuthorize` retrofit on existing admin controllers
|
||||||
|
|
||||||
### Phase 2
|
### Phase 2
|
||||||
|
|
||||||
@@ -446,6 +501,7 @@ CREATE INDEX idx_audit_log_category ON audit_log (category);
|
|||||||
- OpenSearch operations (Force Reindex All, Flush)
|
- OpenSearch operations (Force Reindex All, Flush)
|
||||||
- Bulk index operations (checkbox selection)
|
- Bulk index operations (checkbox selection)
|
||||||
- Audit log CSV/JSON export for auditors
|
- Audit log CSV/JSON export for auditors
|
||||||
|
- Audit log retention/archival strategy (7-year SOC2 requirement)
|
||||||
- OPERATOR role with view-only permissions
|
- OPERATOR role with view-only permissions
|
||||||
|
|
||||||
### Phase 3
|
### Phase 3
|
||||||
|
|||||||
Reference in New Issue
Block a user