**Scope:** Phase 1 implementation; full vision documented with Phase 2+ sections marked
## Overview
Add Database and OpenSearch admin pages to the Cameleer3 Server UI, allowing administrators to monitor subsystem health, inspect metrics, and perform basic maintenance actions. Restructure admin navigation from a single OIDC page to a sidebar sub-menu with dedicated pages per concern.
## Goals
- Give admins real-time visibility into PostgreSQL and OpenSearch health, performance, and storage
- Metrics: query cache hit rate, request cache hit rate, average search latency, average indexing latency, JVM heap used (visual bar with used/max)
- Source: OpenSearch `_nodes/stats` API
- **Auto-refreshes every 15 seconds**
### Operations Section (Phase 2 — Visible but Disabled)
- Buttons: Force Reindex All, Flush Index, Delete Index (bulk via checkbox selection)
- Greyed out with tooltip: "Available in a future release"
### Thresholds Section
- Collapsible, collapsed by default
- Configurable values:
- Cluster health: warning level, critical level
- Queue depth: warning count, critical count
- JVM heap usage: warning %, critical %
- Failed docs: warning count, critical count
- Save button persists to database
---
## 4. Audit Log Page (`/admin/audit`)
### Purpose
Database-backed audit trail of all administrative actions across the system. Provides SOC2-compliant evidence of who did what, when, and from where. The audit log is append-only — entries cannot be modified or deleted through the UI or API.
-`AuditService` — central service in `cameleer3-server-core`, injected into all admin controllers via direct method calls (no AOP/interceptor — consistent with existing controller style)
- Primary method: `log(action, category, target, detail, result)` — extracts username and IP from `SecurityContextHolder` and `HttpServletRequest`
- Overloaded method for pre-auth contexts: `log(username, action, category, target, detail, result, request)` — used by auth controllers where `SecurityContext` is not yet populated (login success/failure)
- Captures `user_agent` from `HttpServletRequest` header
All endpoints under `/api/v1/admin/` — secured by existing Spring Security filter chain (`ROLE_ADMIN` required). Controllers additionally annotated with `@PreAuthorize("hasRole('ADMIN')")` for defense-in-depth.
**OpenSearch client:** All admin OpenSearch endpoints reuse the existing `OpenSearchClient` bean configured in `OpenSearchConfig.java`. No separate client or credentials needed — the admin endpoints call cluster-level APIs (`_cluster/health`, `_cat/indices`, `_nodes/stats`) using the same connection.
- Warning must be <= critical for all numeric threshold pairs
- Percentage values must be 0–100
- Duration values must be > 0
-`clusterHealthWarning` must be less severe than `clusterHealthCritical` (GREEN < YELLOW < RED)
- Backend returns 400 Bad Request with field-level error messages on validation failure
### Pagination Limits
- All paginated endpoints enforce a maximum page size of 100 (`min(requested, 100)`)
- Applies to: indices listing, audit log
### Error Responses
All new endpoints return errors in a consistent shape:
```json
{
"status": 404,
"error": "Not Found",
"message": "No active query with PID 12345"
}
```
Specific error cases:
-`POST /admin/database/queries/{pid}/kill` — 404 if PID not found, 500 if `pg_terminate_backend` fails
-`DELETE /admin/opensearch/indices/{name}` — 404 if index not found, 502 if OpenSearch unreachable
-`GET /admin/database/status` — returns 200 with `"connected": false` if database is unreachable (not 503), so the frontend can render a red status badge rather than an error state
-`GET /admin/opensearch/status` — returns 200 with `"clusterHealth": "UNREACHABLE"` if OpenSearch is down
2.**Controller annotation** — `@PreAuthorize("hasRole('ADMIN')")` on each controller class (defense-in-depth). This is a new convention — existing controllers (`OidcConfigAdminController`, `UserAdminController`) must be retrofitted with this annotation as part of Phase 1.
3.**`@EnableMethodSecurity`** — must be added to `SecurityConfig.java` to activate `@PreAuthorize` processing (prerequisite for layer 2)
4.**UI role check** — sidebar admin section hidden for non-admin users (cosmetic only, not a security boundary)
All admin actions are persisted to the `audit_log` database table (see Section 4 and Section 7 — Data Storage) AND logged via SLF4J at INFO level. The database record is the source of truth for compliance; the SLF4J log provides operational visibility.
The `AuditService` is injected into all admin controllers (infrastructure, OIDC, user management) and the authentication flow. See Section 4 (Audit Log Page) for full details on what is logged and the record structure.
- Single-row table using `CHECK (id = 1)` constraint — stricter than the `oidc_config` pattern (which uses a text PK defaulting to `'default'` without a constraint). The CHECK approach is preferred going forward as it explicitly prevents multiple rows.
- JSON column for flexibility — adding new thresholds doesn't require schema changes
- Indexed on timestamp (primary query axis), username, category, action, and target for filtered views and free-text search via `ILIKE` on indexed text columns
- JSONB `detail` column holds action-specific context without schema changes (not searched via free text — use row expansion for detail inspection)
-`user_agent` field captures client identification for forensic analysis (SOC2)
- **Retention:** unbounded in Phase 1. Phase 2+ should add a retention/archival strategy (e.g., TimescaleDB hypertable with retention policy, or periodic archive to cold storage). Typical SOC2 retention is 7 years.
