From 950f16be7a0d6990c812711a0b247ef37eaf4e7e Mon Sep 17 00:00:00 2001 From: hsiegeln <37154749+hsiegeln@users.noreply.github.com> Date: Tue, 17 Mar 2026 15:24:56 +0100 Subject: [PATCH] docs: fix plan review issues for infrastructure overview MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Fix AuthController → UiAuthController throughout - Flesh out PostgresAuditRepository.find() with full dynamic query implementation - Flesh out OpenSearchAdminController getStatus/getIndices/getPerformance methods - Fix HikariCP maxWait → getConnectionTimeout() - Add AuditServiceTest unit test task step - Add complete ThresholdConfigRequest with validation logic - Fix audit log from/to params: Instant → LocalDate with @DateTimeFormat - Fill in React Query hook placeholder bodies - Resolve extractUsername() duplication (inline in controller) Co-Authored-By: Claude Opus 4.6 (1M context) --- .../2026-03-17-infrastructure-overview.md | 1794 +++++++++++++++++ 1 file changed, 1794 insertions(+) create mode 100644 docs/superpowers/plans/2026-03-17-infrastructure-overview.md diff --git a/docs/superpowers/plans/2026-03-17-infrastructure-overview.md b/docs/superpowers/plans/2026-03-17-infrastructure-overview.md new file mode 100644 index 00000000..2146d846 --- /dev/null +++ b/docs/superpowers/plans/2026-03-17-infrastructure-overview.md @@ -0,0 +1,1794 @@ +# Infrastructure Overview Implementation Plan + +> **For agentic workers:** REQUIRED: Use superpowers:subagent-driven-development (if subagents available) or superpowers:executing-plans to implement this plan. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add Database, OpenSearch, and Audit Log admin pages with monitoring, basic management actions, configurable thresholds, and SOC2-compliant audit logging. + +**Architecture:** Backend-proxied monitoring — new Spring Boot controllers query PostgreSQL and OpenSearch, aggregate data, and return structured JSON. Central `AuditService` records all admin actions to a database table. Frontend consumes these via React Query hooks with auto-refresh for lightweight endpoints. + +**Tech Stack:** Java 17 / Spring Boot 3.4.3 / JdbcTemplate / OpenSearch Java Client / HikariCP MXBean / React 18 / TypeScript / TanStack React Query / React Router / openapi-fetch + +**Spec:** `docs/superpowers/specs/2026-03-17-infrastructure-overview-design.md` + +--- + +## File Map + +### New Files — Backend (Core Module) + +| File | Responsibility | +|------|---------------| +| `core/.../admin/AuditRecord.java` | Audit log record (immutable) | +| `core/.../admin/AuditCategory.java` | Enum: INFRA, AUTH, USER_MGMT, CONFIG | +| `core/.../admin/AuditResult.java` | Enum: SUCCESS, FAILURE | +| `core/.../admin/AuditRepository.java` | Interface: insert + paginated query | +| `core/.../admin/AuditService.java` | Central audit logging service | +| `core/.../admin/ThresholdConfig.java` | Threshold config record | +| `core/.../admin/ThresholdRepository.java` | Interface: find + save | +| `core/.../indexing/SearchIndexerStats.java` | Interface: queue depth, failed count, rate, etc. | + +> `core/...` = `cameleer3-server-core/src/main/java/com/cameleer3/server/core` + +### New Files — Backend (App Module) + +| File | Responsibility | +|------|---------------| +| `app/.../storage/PostgresAuditRepository.java` | JdbcTemplate impl of AuditRepository | +| `app/.../storage/PostgresThresholdRepository.java` | JdbcTemplate impl of ThresholdRepository | +| `app/.../controller/DatabaseAdminController.java` | Database monitoring + kill query | +| `app/.../controller/OpenSearchAdminController.java` | OpenSearch monitoring + delete index | +| `app/.../controller/ThresholdAdminController.java` | Threshold CRUD | +| `app/.../controller/AuditLogController.java` | Audit log viewer endpoint | +| `app/.../dto/DatabaseStatusResponse.java` | DTO: version, host, schema, connected | +| `app/.../dto/ConnectionPoolResponse.java` | DTO: active, idle, pending, maxWait, maxSize | +| `app/.../dto/TableSizeResponse.java` | DTO: table name, rows, dataSize, indexSize | +| `app/.../dto/ActiveQueryResponse.java` | DTO: pid, duration, state, query | +| `app/.../dto/OpenSearchStatusResponse.java` | DTO: version, host, health, nodes | +| `app/.../dto/PipelineStatsResponse.java` | DTO: queueDepth, failed, debounce, rate, lastIndexed | +| `app/.../dto/IndexInfoResponse.java` | DTO: name, docs, size, health, shards | +| `app/.../dto/IndicesPageResponse.java` | DTO: paginated indices + summary | +| `app/.../dto/PerformanceResponse.java` | DTO: cache rates, latencies, JVM heap | +| `app/.../dto/AuditLogPageResponse.java` | DTO: paginated audit entries | +| `app/.../dto/ThresholdConfigRequest.java` | DTO: threshold save payload | +| `resources/db/migration/V9__admin_thresholds.sql` | Flyway: admin_thresholds table | +| `resources/db/migration/V10__audit_log.sql` | Flyway: audit_log table | + +> `app/...` = `cameleer3-server-app/src/main/java/com/cameleer3/server/app` +> `resources/...` = `cameleer3-server-app/src/main/resources` + +### New Files — Backend (Tests) + +| File | Responsibility | +|------|---------------| +| `test/.../controller/DatabaseAdminControllerIT.java` | Integration test: DB endpoints | +| `test/.../controller/OpenSearchAdminControllerIT.java` | Integration test: OS endpoints | +| `test/.../controller/AuditLogControllerIT.java` | Integration test: audit endpoints | +| `test/.../controller/ThresholdAdminControllerIT.java` | Integration test: threshold endpoints | +| `test/.../admin/AuditServiceTest.java` | Unit test: audit service logic | + +> `test/...` = `cameleer3-server-app/src/test/java/com/cameleer3/server/app` + +### New Files — Frontend + +| File | Responsibility | +|------|---------------| +| `ui/src/api/queries/admin/database.ts` | React Query hooks: database endpoints | +| `ui/src/api/queries/admin/opensearch.ts` | React Query hooks: OpenSearch endpoints | +| `ui/src/api/queries/admin/thresholds.ts` | React Query hooks: threshold endpoints | +| `ui/src/api/queries/admin/audit.ts` | React Query hooks: audit log endpoint | +| `ui/src/components/admin/StatusBadge.tsx` | Green/yellow/red indicator | +| `ui/src/components/admin/StatusBadge.module.css` | Styles for StatusBadge | +| `ui/src/components/admin/RefreshableCard.tsx` | Card with refresh button | +| `ui/src/components/admin/RefreshableCard.module.css` | Styles for RefreshableCard | +| `ui/src/components/admin/ConfirmDeleteDialog.tsx` | Confirmation dialog requiring name input | +| `ui/src/components/admin/ConfirmDeleteDialog.module.css` | Styles for ConfirmDeleteDialog | +| `ui/src/pages/admin/DatabaseAdminPage.tsx` | Database monitoring page | +| `ui/src/pages/admin/DatabaseAdminPage.module.css` | Styles | +| `ui/src/pages/admin/OpenSearchAdminPage.tsx` | OpenSearch monitoring page | +| `ui/src/pages/admin/OpenSearchAdminPage.module.css` | Styles | +| `ui/src/pages/admin/AuditLogPage.tsx` | Audit log viewer page | +| `ui/src/pages/admin/AuditLogPage.module.css` | Styles | + +### Modified Files + +| File | Change | +|------|--------| +| `core/.../indexing/SearchIndexer.java` | Add stats counters, implement SearchIndexerStats | +| `app/.../security/SecurityConfig.java` | Add `@EnableMethodSecurity` | +| `app/.../controller/OidcConfigAdminController.java` | Add `@PreAuthorize`, inject AuditService | +| `app/.../controller/UserAdminController.java` | Add `@PreAuthorize`, inject AuditService | +| `app/.../security/UiAuthController.java` | Inject AuditService, log login/logout events | +| `app/.../security/OidcAuthController.java` | Inject AuditService, log OIDC login events | +| `app/.../config/StorageBeanConfig.java` | Wire new beans (AuditService, repositories) | +| `ui/src/router.tsx` | Add admin sub-routes, redirect `/admin` | +| `ui/src/components/layout/AppSidebar.tsx` | Collapsible admin sub-menu | + +--- + +## Task Breakdown + +### Task 1: Flyway Migrations + +**Files:** +- Create: `cameleer3-server-app/src/main/resources/db/migration/V9__admin_thresholds.sql` +- Create: `cameleer3-server-app/src/main/resources/db/migration/V10__audit_log.sql` + +- [ ] **Step 1: Create V9 migration** + +```sql +CREATE TABLE admin_thresholds ( + id INTEGER PRIMARY KEY DEFAULT 1, + config JSONB NOT NULL DEFAULT '{}', + updated_at TIMESTAMPTZ NOT NULL DEFAULT now(), + updated_by TEXT NOT NULL, + CONSTRAINT single_row CHECK (id = 1) +); +``` + +- [ ] **Step 2: Create V10 migration** + +```sql +CREATE TABLE audit_log ( + id BIGSERIAL PRIMARY KEY, + timestamp TIMESTAMPTZ NOT NULL DEFAULT now(), + username TEXT NOT NULL, + action TEXT NOT NULL, + category TEXT NOT NULL, + target TEXT, + detail JSONB, + result TEXT NOT NULL, + ip_address TEXT, + user_agent TEXT +); + +CREATE INDEX idx_audit_log_timestamp ON audit_log (timestamp DESC); +CREATE INDEX idx_audit_log_username ON audit_log (username); +CREATE INDEX idx_audit_log_category ON audit_log (category); +CREATE INDEX idx_audit_log_action ON audit_log (action); +CREATE INDEX idx_audit_log_target ON audit_log (target); +``` + +- [ ] **Step 3: Verify migrations compile** + +Run: `cd cameleer3-server && mvn clean compile -pl cameleer3-server-app` +Expected: BUILD SUCCESS + +- [ ] **Step 4: Commit** + +```bash +git add cameleer3-server-app/src/main/resources/db/migration/V9__admin_thresholds.sql \ + cameleer3-server-app/src/main/resources/db/migration/V10__audit_log.sql +git commit -m "feat: add Flyway V9 (thresholds) and V10 (audit_log) migrations" +``` + +--- + +### Task 2: Core Module — Audit Domain Model + Repository Interface + +**Files:** +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/AuditCategory.java` +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/AuditResult.java` +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/AuditRecord.java` +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/AuditRepository.java` +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/AuditService.java` + +- [ ] **Step 1: Create AuditCategory enum** + +```java +package com.cameleer3.server.core.admin; + +public enum AuditCategory { + INFRA, AUTH, USER_MGMT, CONFIG +} +``` + +- [ ] **Step 2: Create AuditResult enum** + +```java +package com.cameleer3.server.core.admin; + +public enum AuditResult { + SUCCESS, FAILURE +} +``` + +- [ ] **Step 3: Create AuditRecord** + +```java +package com.cameleer3.server.core.admin; + +import java.time.Instant; +import java.util.Map; + +public record AuditRecord( + long id, + Instant timestamp, + String username, + String action, + AuditCategory category, + String target, + Map detail, + AuditResult result, + String ipAddress, + String userAgent +) { + /** Factory for creating new records (id and timestamp assigned by DB) */ + public static AuditRecord create(String username, String action, AuditCategory category, + String target, Map detail, AuditResult result, + String ipAddress, String userAgent) { + return new AuditRecord(0, null, username, action, category, target, detail, result, ipAddress, userAgent); + } +} +``` + +- [ ] **Step 4: Create AuditRepository interface** + +```java +package com.cameleer3.server.core.admin; + +import java.time.Instant; +import java.util.List; + +public interface AuditRepository { + + void insert(AuditRecord record); + + record AuditQuery( + String username, + AuditCategory category, + String search, + Instant from, + Instant to, + String sort, + String order, + int page, + int size + ) {} + + record AuditPage(List items, long totalCount) {} + + AuditPage find(AuditQuery query); +} +``` + +- [ ] **Step 5: Create AuditService** + +The service lives in core so it can be referenced by any controller. It depends on `AuditRepository` (interface) and uses SLF4J for dual logging. It extracts username/IP/user-agent from Spring Security context and servlet request. + +```java +package com.cameleer3.server.core.admin; + +import jakarta.servlet.http.HttpServletRequest; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.security.core.Authentication; +import org.springframework.security.core.context.SecurityContextHolder; + +import java.util.Map; + +public class AuditService { + private static final Logger log = LoggerFactory.getLogger(AuditService.class); + private final AuditRepository repository; + + public AuditService(AuditRepository repository) { + this.repository = repository; + } + + /** Log an action using the current SecurityContext for username */ + public void log(String action, AuditCategory category, String target, + Map detail, AuditResult result, + HttpServletRequest request) { + String username = extractUsername(); + log(username, action, category, target, detail, result, request); + } + + /** Log an action with explicit username (for pre-auth contexts like login) */ + public void log(String username, String action, AuditCategory category, String target, + Map detail, AuditResult result, + HttpServletRequest request) { + String ip = request != null ? request.getRemoteAddr() : null; + String userAgent = request != null ? request.getHeader("User-Agent") : null; + AuditRecord record = AuditRecord.create(username, action, category, target, detail, result, ip, userAgent); + + repository.insert(record); + + log.info("AUDIT: user={} action={} category={} target={} result={}", + username, action, category, target, result); + } + + private String extractUsername() { + Authentication auth = SecurityContextHolder.getContext().getAuthentication(); + if (auth != null && auth.getName() != null) { + String name = auth.getName(); + return name.startsWith("user:") ? name.substring(5) : name; + } + return "unknown"; + } +} +``` + +**Note:** This class uses `jakarta.servlet` and `org.springframework.security` — the core POM must have these as `provided` scope dependencies if not already present. Check `cameleer3-server-core/pom.xml` and add if needed: +```xml + + jakarta.servlet + jakarta.servlet-api + provided + + + org.springframework.security + spring-security-core + provided + +``` + +- [ ] **Step 6: Write AuditService unit test** + +```java +class AuditServiceTest { + private AuditRepository mockRepository; + private AuditService auditService; + + @BeforeEach + void setUp() { + mockRepository = mock(AuditRepository.class); + auditService = new AuditService(mockRepository); + } + + @Test + void log_withExplicitUsername_insertsRecordWithCorrectFields() { + var request = mock(HttpServletRequest.class); + when(request.getRemoteAddr()).thenReturn("192.168.1.1"); + when(request.getHeader("User-Agent")).thenReturn("Mozilla/5.0"); + + auditService.log("admin", "kill_query", AuditCategory.INFRA, "PID 42", + Map.of("query", "SELECT 1"), AuditResult.SUCCESS, request); + + var captor = ArgumentCaptor.forClass(AuditRecord.class); + verify(mockRepository).insert(captor.capture()); + var record = captor.getValue(); + assertEquals("admin", record.username()); + assertEquals("kill_query", record.action()); + assertEquals(AuditCategory.INFRA, record.category()); + assertEquals("PID 42", record.target()); + assertEquals("192.168.1.1", record.ipAddress()); + assertEquals("Mozilla/5.0", record.userAgent()); + } + + @Test + void log_withNullRequest_handlesGracefully() { + auditService.log("admin", "test", AuditCategory.CONFIG, null, null, AuditResult.SUCCESS, null); + verify(mockRepository).insert(any(AuditRecord.class)); + } +} +``` + +- [ ] **Step 7: Verify core module compiles and test passes** + +Run: `mvn clean compile -pl cameleer3-server-core` +Run: `mvn test -pl cameleer3-server-app -Dtest=AuditServiceTest` +Expected: BUILD SUCCESS, tests PASS + +- [ ] **Step 8: Commit** + +```bash +git add cameleer3-server-core/ cameleer3-server-app/src/test/ +git commit -m "feat: add audit domain model, repository interface, AuditService, and unit test" +``` + +--- + +### Task 3: Core Module — Threshold Model + Repository Interface + +**Files:** +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/ThresholdConfig.java` +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/admin/ThresholdRepository.java` + +- [ ] **Step 1: Create ThresholdConfig record** + +```java +package com.cameleer3.server.core.admin; + +public record ThresholdConfig( + DatabaseThresholds database, + OpenSearchThresholds opensearch +) { + public record DatabaseThresholds( + int connectionPoolWarning, + int connectionPoolCritical, + double queryDurationWarning, + double queryDurationCritical + ) { + public static DatabaseThresholds defaults() { + return new DatabaseThresholds(80, 95, 1.0, 10.0); + } + } + + public record OpenSearchThresholds( + String clusterHealthWarning, + String clusterHealthCritical, + int queueDepthWarning, + int queueDepthCritical, + int jvmHeapWarning, + int jvmHeapCritical, + int failedDocsWarning, + int failedDocsCritical + ) { + public static OpenSearchThresholds defaults() { + return new OpenSearchThresholds("YELLOW", "RED", 100, 500, 75, 90, 1, 10); + } + } + + public static ThresholdConfig defaults() { + return new ThresholdConfig(DatabaseThresholds.defaults(), OpenSearchThresholds.defaults()); + } +} +``` + +- [ ] **Step 2: Create ThresholdRepository interface** + +```java +package com.cameleer3.server.core.admin; + +import java.util.Optional; + +public interface ThresholdRepository { + Optional find(); + void save(ThresholdConfig config, String updatedBy); +} +``` + +- [ ] **Step 3: Compile and commit** + +Run: `mvn clean compile -pl cameleer3-server-core` + +```bash +git add cameleer3-server-core/ +git commit -m "feat: add ThresholdConfig model and ThresholdRepository interface" +``` + +--- + +### Task 4: Core Module — SearchIndexerStats Interface + Instrumentation + +**Files:** +- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/indexing/SearchIndexerStats.java` +- Modify: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/indexing/SearchIndexer.java` + +- [ ] **Step 1: Create SearchIndexerStats interface** + +```java +package com.cameleer3.server.core.indexing; + +import java.time.Instant; + +public interface SearchIndexerStats { + int getQueueDepth(); + int getMaxQueueSize(); + long getFailedCount(); + long getIndexedCount(); + Instant getLastIndexedAt(); + long getDebounceMs(); + /** Approximate indexing rate in docs/sec over last measurement window */ + double getIndexingRate(); +} +``` + +- [ ] **Step 2: Add stats counters to SearchIndexer** + +Modify `SearchIndexer.java`: +- Add fields: `AtomicLong failedCount`, `AtomicLong indexedCount`, `volatile Instant lastIndexedAt` +- Add rate tracking: `AtomicLong rateWindowStart`, `AtomicLong rateWindowCount` +- Implement `SearchIndexerStats` +- Increment counters in `indexExecution()`: `indexedCount.incrementAndGet()` on success, `failedCount.incrementAndGet()` on catch +- Set `lastIndexedAt = Instant.now()` after successful indexing +- Return `pending.size()` for queue depth, `queueCapacity` for max size + +Key changes to `indexExecution()` method: +```java +private void indexExecution(String executionId) { + pending.remove(executionId); + try { + ExecutionRecord exec = executionStore.findById(executionId).orElse(null); + if (exec == null) return; + // ... existing indexing logic ... + indexedCount.incrementAndGet(); + lastIndexedAt = Instant.now(); + updateRate(); + } catch (Exception e) { + failedCount.incrementAndGet(); + log.error("Failed to index execution {}", executionId, e); + } +} +``` + +Rate calculation approach: track count delta between 15-second measurement windows. +```java +private void updateRate() { + long now = System.currentTimeMillis(); + long windowStart = rateWindowStartMs.get(); + long count = rateWindowCount.incrementAndGet(); + long elapsed = now - windowStart; + if (elapsed >= 15_000) { // 15-second window + lastRate = count / (elapsed / 1000.0); + rateWindowStartMs.set(now); + rateWindowCount.set(0); + } +} +``` + +- [ ] **Step 3: Compile and commit** + +Run: `mvn clean compile -pl cameleer3-server-core` + +```bash +git add cameleer3-server-core/ +git commit -m "feat: add SearchIndexerStats interface and instrument SearchIndexer" +``` + +--- + +### Task 5: App Module — Postgres Repository Implementations + +**Files:** +- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresAuditRepository.java` +- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresThresholdRepository.java` + +- [ ] **Step 1: Create PostgresAuditRepository** + +Follow the same `@Repository` + `JdbcTemplate` pattern used by `PostgresUserRepository` and `PostgresOidcConfigRepository`. + +```java +@Repository +public class PostgresAuditRepository implements AuditRepository { + private final JdbcTemplate jdbc; + private final ObjectMapper objectMapper; + + public PostgresAuditRepository(JdbcTemplate jdbc, ObjectMapper objectMapper) { + this.jdbc = jdbc; + this.objectMapper = objectMapper; + } + + @Override + public void insert(AuditRecord record) { + jdbc.update(""" + INSERT INTO audit_log (username, action, category, target, detail, result, ip_address, user_agent) + VALUES (?, ?, ?, ?, ?::jsonb, ?, ?, ?) + """, + record.username(), record.action(), record.category().name(), + record.target(), toJson(record.detail()), record.result().name(), + record.ipAddress(), record.userAgent()); + } + + @Override + public AuditPage find(AuditQuery query) { + int effectiveSize = Math.min(query.size(), 100); + var params = new ArrayList<>(); + var conditions = new ArrayList(); + + // Always filter by timestamp range + conditions.add("timestamp >= ?"); + params.add(Timestamp.from(query.from())); + conditions.add("timestamp <= ?"); + params.add(Timestamp.from(query.to())); + + if (query.username() != null && !query.username().isBlank()) { + conditions.add("username = ?"); + params.add(query.username()); + } + if (query.category() != null) { + conditions.add("category = ?"); + params.add(query.category().name()); + } + if (query.search() != null && !query.search().isBlank()) { + conditions.add("(action ILIKE ? OR target ILIKE ?)"); + String pattern = "%" + query.search() + "%"; + params.add(pattern); + params.add(pattern); + } + + String where = "WHERE " + String.join(" AND ", conditions); + + // Validate sort column against allowlist + String sortCol = switch (query.sort()) { + case "username" -> "username"; + case "action" -> "action"; + case "category" -> "category"; + default -> "timestamp"; + }; + String orderDir = "asc".equalsIgnoreCase(query.order()) ? "ASC" : "DESC"; + + // Count query + long total = jdbc.queryForObject( + "SELECT COUNT(*) FROM audit_log " + where, Long.class, params.toArray()); + + // Data query with pagination + String sql = "SELECT * FROM audit_log " + where + + " ORDER BY " + sortCol + " " + orderDir + + " LIMIT ? OFFSET ?"; + params.add(effectiveSize); + params.add(query.page() * effectiveSize); + + var items = jdbc.query(sql, this::mapRow, params.toArray()); + return new AuditPage(items, total); + } + + private String toJson(Map map) { + if (map == null || map.isEmpty()) return null; + try { return objectMapper.writeValueAsString(map); } + catch (Exception e) { return "{}"; } + } + + @SuppressWarnings("unchecked") + private AuditRecord mapRow(ResultSet rs, int rowNum) throws SQLException { + Map detail = null; + String detailJson = rs.getString("detail"); + if (detailJson != null) { + try { detail = objectMapper.readValue(detailJson, Map.class); } + catch (Exception e) { detail = Map.of("_raw", detailJson); } + } + Timestamp ts = rs.getTimestamp("timestamp"); + return new AuditRecord( + rs.getLong("id"), + ts != null ? ts.toInstant() : null, + rs.getString("username"), + rs.getString("action"), + AuditCategory.valueOf(rs.getString("category")), + rs.getString("target"), + detail, + AuditResult.valueOf(rs.getString("result")), + rs.getString("ip_address"), + rs.getString("user_agent")); + } +} +``` + +- [ ] **Step 2: Create PostgresThresholdRepository** + +```java +@Repository +public class PostgresThresholdRepository implements ThresholdRepository { + private final JdbcTemplate jdbc; + private final ObjectMapper objectMapper; + + public PostgresThresholdRepository(JdbcTemplate jdbc, ObjectMapper objectMapper) { + this.jdbc = jdbc; + this.objectMapper = objectMapper; + } + + @Override + public Optional find() { + var results = jdbc.query( + "SELECT config FROM admin_thresholds WHERE id = 1", + (rs, rowNum) -> { + try { + return objectMapper.readValue(rs.getString("config"), ThresholdConfig.class); + } catch (Exception e) { + return ThresholdConfig.defaults(); + } + }); + return results.isEmpty() ? Optional.empty() : Optional.of(results.get(0)); + } + + @Override + public void save(ThresholdConfig config, String updatedBy) { + String json; + try { json = objectMapper.writeValueAsString(config); } + catch (Exception e) { throw new RuntimeException("Failed to serialize thresholds", e); } + + jdbc.update(""" + INSERT INTO admin_thresholds (id, config, updated_by, updated_at) + VALUES (1, ?::jsonb, ?, now()) + ON CONFLICT (id) DO UPDATE SET config = ?::jsonb, updated_by = ?, updated_at = now() + """, json, updatedBy, json, updatedBy); + } +} +``` + +- [ ] **Step 3: Compile and commit** + +Run: `mvn clean compile -pl cameleer3-server-app` + +```bash +git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/ +git commit -m "feat: add Postgres implementations for AuditRepository and ThresholdRepository" +``` + +--- + +### Task 6: App Module — Bean Wiring + Security Retrofit + +**Files:** +- Modify: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/StorageBeanConfig.java` +- Modify: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/security/SecurityConfig.java` +- Modify: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/OidcConfigAdminController.java` +- Modify: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/UserAdminController.java` + +- [ ] **Step 1: Wire AuditService bean in StorageBeanConfig** + +Add to `StorageBeanConfig.java`: +```java +@Bean +public AuditService auditService(AuditRepository auditRepository) { + return new AuditService(auditRepository); +} +``` + +- [ ] **Step 2: Add @EnableMethodSecurity to SecurityConfig** + +```java +@Configuration +@EnableWebSecurity +@EnableMethodSecurity // <-- add this +public class SecurityConfig { +``` + +- [ ] **Step 3: Add @PreAuthorize to existing admin controllers** + +Add `@PreAuthorize("hasRole('ADMIN')")` to both `OidcConfigAdminController` and `UserAdminController` class-level annotations. + +- [ ] **Step 4: Inject AuditService into OidcConfigAdminController** + +Add `AuditService` as constructor parameter. Add audit logging calls: +- `save()` → `auditService.log("update_oidc", AuditCategory.CONFIG, "oidc", Map.of(...), AuditResult.SUCCESS, request)` +- `delete()` → `auditService.log("delete_oidc", AuditCategory.CONFIG, "oidc", null, AuditResult.SUCCESS, request)` +- `testConnection()` → `auditService.log("test_oidc", AuditCategory.CONFIG, "oidc", Map.of("result", ...), result, request)` + +Add `HttpServletRequest request` parameter to each endpoint method. + +- [ ] **Step 5: Inject AuditService into UserAdminController** + +Same pattern. Log: +- `updateRoles()` → `auditService.log("update_roles", AuditCategory.USER_MGMT, userId, Map.of("roles", roles), AuditResult.SUCCESS, request)` +- `deleteUser()` → `auditService.log("delete_user", AuditCategory.USER_MGMT, userId, null, AuditResult.SUCCESS, request)` + +- [ ] **Step 6: Inject AuditService into UiAuthController** + +The file is `UiAuthController.java` (not `AuthController`). Log login success/failure and logout. Use the explicit-username overload for login (SecurityContext not yet populated): +- Login success: `auditService.log(username, "login", AuditCategory.AUTH, null, null, AuditResult.SUCCESS, request)` +- Login failure: `auditService.log(username, "login_failed", AuditCategory.AUTH, null, Map.of("reason", reason), AuditResult.FAILURE, request)` + +- [ ] **Step 7: Inject AuditService into OidcAuthController** + +Log OIDC login: +- OIDC callback success: `auditService.log(username, "login_oidc", AuditCategory.AUTH, null, Map.of("provider", issuerUri), AuditResult.SUCCESS, request)` + +- [ ] **Step 8: Compile and commit** + +Run: `mvn clean compile` + +```bash +git add cameleer3-server-app/ +git commit -m "feat: wire AuditService, enable method security, retrofit audit logging into existing controllers" +``` + +--- + +### Task 7: App Module — Response DTOs + +**Files:** +- Create all DTO files listed in the File Map under `app/.../dto/` + +- [ ] **Step 1: Create database-related DTOs** + +All as Java records with `@Schema` annotations for OpenAPI: + +```java +// DatabaseStatusResponse +public record DatabaseStatusResponse( + @Schema(description = "Whether the database is reachable") boolean connected, + @Schema(description = "PostgreSQL version string") String version, + @Schema(description = "JDBC host") String host, + @Schema(description = "Current schema") String schema, + @Schema(description = "Whether TimescaleDB extension is present") boolean timescaleDb +) {} + +// ConnectionPoolResponse +public record ConnectionPoolResponse( + int activeConnections, int idleConnections, int pendingThreads, + long maxWaitMs, int maxPoolSize +) {} + +// TableSizeResponse +public record TableSizeResponse(String tableName, long rowCount, String dataSize, String indexSize, long dataSizeBytes, long indexSizeBytes) {} + +// ActiveQueryResponse +public record ActiveQueryResponse(int pid, double durationSeconds, String state, String query) {} +``` + +- [ ] **Step 2: Create OpenSearch-related DTOs** + +```java +// OpenSearchStatusResponse +public record OpenSearchStatusResponse( + boolean reachable, String clusterHealth, String version, int nodeCount, String host +) {} + +// PipelineStatsResponse +public record PipelineStatsResponse( + int queueDepth, int maxQueueSize, long failedCount, long indexedCount, + long debounceMs, double indexingRate, Instant lastIndexedAt +) {} + +// IndexInfoResponse +public record IndexInfoResponse(String name, long docCount, String size, long sizeBytes, String health, int primaryShards, int replicaShards) {} + +// IndicesPageResponse +public record IndicesPageResponse( + List indices, long totalIndices, long totalDocs, + String totalSize, int page, int pageSize, int totalPages +) {} + +// PerformanceResponse +public record PerformanceResponse( + double queryCacheHitRate, double requestCacheHitRate, + double searchLatencyMs, double indexingLatencyMs, + long jvmHeapUsedBytes, long jvmHeapMaxBytes +) {} +``` + +- [ ] **Step 3: Create audit + threshold DTOs** + +```java +// AuditLogPageResponse +public record AuditLogPageResponse( + List items, long totalCount, int page, int pageSize, int totalPages +) {} + +// ThresholdConfigRequest +public record ThresholdConfigRequest( + @Valid DatabaseThresholdsRequest database, + @Valid OpenSearchThresholdsRequest opensearch +) { + public record DatabaseThresholdsRequest( + @Min(0) @Max(100) int connectionPoolWarning, + @Min(0) @Max(100) int connectionPoolCritical, + @Positive double queryDurationWarning, + @Positive double queryDurationCritical + ) {} + + public record OpenSearchThresholdsRequest( + @NotBlank String clusterHealthWarning, + @NotBlank String clusterHealthCritical, + @Min(0) int queueDepthWarning, + @Min(0) int queueDepthCritical, + @Min(0) @Max(100) int jvmHeapWarning, + @Min(0) @Max(100) int jvmHeapCritical, + @Min(0) int failedDocsWarning, + @Min(0) int failedDocsCritical + ) {} + + /** Convert to domain model after validation */ + public ThresholdConfig toConfig() { + return new ThresholdConfig( + new ThresholdConfig.DatabaseThresholds( + database.connectionPoolWarning, database.connectionPoolCritical, + database.queryDurationWarning, database.queryDurationCritical), + new ThresholdConfig.OpenSearchThresholds( + opensearch.clusterHealthWarning, opensearch.clusterHealthCritical, + opensearch.queueDepthWarning, opensearch.queueDepthCritical, + opensearch.jvmHeapWarning, opensearch.jvmHeapCritical, + opensearch.failedDocsWarning, opensearch.failedDocsCritical)); + } + + /** Custom validation: warning <= critical for all pairs */ + public List validate() { + var errors = new ArrayList(); + if (database.connectionPoolWarning > database.connectionPoolCritical) + errors.add("database.connectionPoolWarning must be <= connectionPoolCritical"); + if (database.queryDurationWarning > database.queryDurationCritical) + errors.add("database.queryDurationWarning must be <= queryDurationCritical"); + if (opensearch.queueDepthWarning > opensearch.queueDepthCritical) + errors.add("opensearch.queueDepthWarning must be <= queueDepthCritical"); + if (opensearch.jvmHeapWarning > opensearch.jvmHeapCritical) + errors.add("opensearch.jvmHeapWarning must be <= jvmHeapCritical"); + if (opensearch.failedDocsWarning > opensearch.failedDocsCritical) + errors.add("opensearch.failedDocsWarning must be <= failedDocsCritical"); + // Cluster health severity: GREEN < YELLOW < RED + var severity = Map.of("GREEN", 0, "YELLOW", 1, "RED", 2); + int warnSev = severity.getOrDefault(opensearch.clusterHealthWarning.toUpperCase(), -1); + int critSev = severity.getOrDefault(opensearch.clusterHealthCritical.toUpperCase(), -1); + if (warnSev < 0) errors.add("opensearch.clusterHealthWarning must be GREEN, YELLOW, or RED"); + if (critSev < 0) errors.add("opensearch.clusterHealthCritical must be GREEN, YELLOW, or RED"); + if (warnSev >= 0 && critSev >= 0 && warnSev > critSev) + errors.add("opensearch.clusterHealthWarning must be less severe than clusterHealthCritical"); + return errors; + } +} +``` + +- [ ] **Step 4: Compile and commit** + +Run: `mvn clean compile -pl cameleer3-server-app` + +```bash +git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/dto/ +git commit -m "feat: add response/request DTOs for admin infrastructure endpoints" +``` + +--- + +### Task 8: App Module — DatabaseAdminController + +**Files:** +- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/DatabaseAdminController.java` +- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/DatabaseAdminControllerIT.java` + +- [ ] **Step 1: Write integration test** + +Extend `AbstractPostgresIT`. Test all 5 database endpoints: +- `GET /api/v1/admin/database/status` → 200, contains version, connected=true +- `GET /api/v1/admin/database/pool` → 200, contains activeConnections >= 0 +- `GET /api/v1/admin/database/tables` → 200, contains list with at least "users" table +- `GET /api/v1/admin/database/queries` → 200, returns list +- `POST /api/v1/admin/database/queries/99999/kill` → 404 (non-existent PID) + +Use `TestRestTemplate` with admin JWT for authentication. Create a helper that generates an admin JWT using `JwtService`. + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mvn test -pl cameleer3-server-app -Dtest=DatabaseAdminControllerIT` +Expected: FAIL — controller class does not exist + +- [ ] **Step 3: Implement DatabaseAdminController** + +```java +@RestController +@RequestMapping("/api/v1/admin/database") +@PreAuthorize("hasRole('ADMIN')") +@Tag(name = "Database Admin", description = "Database monitoring and management (ADMIN only)") +public class DatabaseAdminController { + + private final JdbcTemplate jdbc; + private final DataSource dataSource; + private final AuditService auditService; + + // Constructor injection + + @GetMapping("/status") + @Operation(summary = "Get database connection status and version") + public ResponseEntity getStatus() { + try { + String version = jdbc.queryForObject("SELECT version()", String.class); + boolean timescaleDb = Boolean.TRUE.equals( + jdbc.queryForObject("SELECT EXISTS(SELECT 1 FROM pg_extension WHERE extname = 'timescaledb')", Boolean.class)); + String schema = jdbc.queryForObject("SELECT current_schema()", String.class); + // Extract host from DataSource URL + String host = extractHost(dataSource); + return ResponseEntity.ok(new DatabaseStatusResponse(true, version, host, schema, timescaleDb)); + } catch (Exception e) { + return ResponseEntity.ok(new DatabaseStatusResponse(false, null, null, null, false)); + } + } + + @GetMapping("/pool") + @Operation(summary = "Get HikariCP connection pool stats") + public ResponseEntity getPool() { + HikariDataSource hds = (HikariDataSource) dataSource; + HikariPoolMXBean pool = hds.getHikariPoolMXBean(); + return ResponseEntity.ok(new ConnectionPoolResponse( + pool.getActiveConnections(), pool.getIdleConnections(), + pool.getThreadsAwaitingConnection(), hds.getConnectionTimeout(), + hds.getMaximumPoolSize())); + } + + @GetMapping("/tables") + @Operation(summary = "Get table sizes and row counts") + public ResponseEntity> getTables() { + // Query pg_stat_user_tables + pg_total_relation_size + pg_indexes_size + var tables = jdbc.query(""" + SELECT schemaname || '.' || relname AS table_name, + n_live_tup AS row_count, + pg_size_pretty(pg_total_relation_size(relid)) AS data_size, + pg_total_relation_size(relid) AS data_size_bytes, + pg_size_pretty(pg_indexes_size(relid)) AS index_size, + pg_indexes_size(relid) AS index_size_bytes + FROM pg_stat_user_tables + ORDER BY pg_total_relation_size(relid) DESC + """, (rs, row) -> new TableSizeResponse( + rs.getString("table_name"), rs.getLong("row_count"), + rs.getString("data_size"), rs.getString("index_size"), + rs.getLong("data_size_bytes"), rs.getLong("index_size_bytes"))); + return ResponseEntity.ok(tables); + } + + @GetMapping("/queries") + @Operation(summary = "Get active queries") + public ResponseEntity> getQueries() { + var queries = jdbc.query(""" + SELECT pid, EXTRACT(EPOCH FROM (now() - query_start)) AS duration_seconds, + state, query + FROM pg_stat_activity + WHERE state != 'idle' AND pid != pg_backend_pid() + ORDER BY query_start ASC + """, (rs, row) -> new ActiveQueryResponse( + rs.getInt("pid"), rs.getDouble("duration_seconds"), + rs.getString("state"), rs.getString("query"))); + return ResponseEntity.ok(queries); + } + + @PostMapping("/queries/{pid}/kill") + @Operation(summary = "Terminate a query by PID") + public ResponseEntity killQuery(@PathVariable int pid, HttpServletRequest request) { + // Check PID exists first + var exists = jdbc.queryForObject( + "SELECT EXISTS(SELECT 1 FROM pg_stat_activity WHERE pid = ? AND pid != pg_backend_pid())", + Boolean.class, pid); + if (!Boolean.TRUE.equals(exists)) { + throw new ResponseStatusException(HttpStatus.NOT_FOUND, "No active query with PID " + pid); + } + jdbc.queryForObject("SELECT pg_terminate_backend(?)", Boolean.class, pid); + auditService.log("kill_query", AuditCategory.INFRA, "PID " + pid, null, AuditResult.SUCCESS, request); + return ResponseEntity.ok().build(); + } +} +``` + +- [ ] **Step 4: Run integration test** + +Run: `mvn test -pl cameleer3-server-app -Dtest=DatabaseAdminControllerIT` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add cameleer3-server-app/ +git commit -m "feat: add DatabaseAdminController with status, pool, tables, queries, and kill endpoints" +``` + +--- + +### Task 9: App Module — OpenSearchAdminController + +**Files:** +- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/OpenSearchAdminController.java` +- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/OpenSearchAdminControllerIT.java` + +- [ ] **Step 1: Write integration test** + +Extend `AbstractPostgresIT` (which starts both PG and OpenSearch containers). Test: +- `GET /api/v1/admin/opensearch/status` → 200, reachable=true, clusterHealth in [green, yellow] +- `GET /api/v1/admin/opensearch/pipeline` → 200, contains queueDepth >= 0 +- `GET /api/v1/admin/opensearch/indices` → 200, returns paginated response +- `GET /api/v1/admin/opensearch/performance` → 200, contains jvmHeapMaxBytes > 0 +- `DELETE /api/v1/admin/opensearch/indices/nonexistent` → 404 + +- [ ] **Step 2: Run test to verify it fails** + +Run: `mvn test -pl cameleer3-server-app -Dtest=OpenSearchAdminControllerIT` +Expected: FAIL + +- [ ] **Step 3: Implement OpenSearchAdminController** + +```java +@RestController +@RequestMapping("/api/v1/admin/opensearch") +@PreAuthorize("hasRole('ADMIN')") +@Tag(name = "OpenSearch Admin", description = "OpenSearch monitoring and management (ADMIN only)") +public class OpenSearchAdminController { + + private final OpenSearchClient client; + private final RestClient restClient; + private final SearchIndexerStats indexerStats; + private final AuditService auditService; + @Value("${opensearch.url:http://localhost:9200}") + private String opensearchUrl; + + // Constructor injection + + @GetMapping("/status") + public ResponseEntity getStatus() { + try { + var health = client.cluster().health(); + var info = client.info(); + return ResponseEntity.ok(new OpenSearchStatusResponse( + true, health.status().jsonValue(), info.version().number(), + health.numberOfNodes(), opensearchUrl)); + } catch (Exception e) { + return ResponseEntity.ok(new OpenSearchStatusResponse( + false, "UNREACHABLE", null, 0, opensearchUrl)); + } + } + + @GetMapping("/pipeline") + public ResponseEntity getPipeline() { + return ResponseEntity.ok(new PipelineStatsResponse( + indexerStats.getQueueDepth(), indexerStats.getMaxQueueSize(), + indexerStats.getFailedCount(), indexerStats.getIndexedCount(), + indexerStats.getDebounceMs(), indexerStats.getIndexingRate(), + indexerStats.getLastIndexedAt())); + } + + @GetMapping("/indices") + public ResponseEntity getIndices( + @RequestParam(defaultValue = "") String search, + @RequestParam(defaultValue = "ALL") String health, + @RequestParam(defaultValue = "name") String sort, + @RequestParam(defaultValue = "asc") String order, + @RequestParam(defaultValue = "0") int page, + @RequestParam(defaultValue = "10") int size) { + int effectiveSize = Math.min(size, 100); + try { + // Use RestClient for _cat/indices with JSON format (richer than Java client's API) + Request catRequest = new Request("GET", "/_cat/indices?format=json&h=index,health,docs.count,store.size,pri,rep&bytes=b"); + Response catResponse = restClient.performRequest(catRequest); + var allIndices = objectMapper.readValue( + catResponse.getEntity().getContent(), + new TypeReference>>() {}); + + // Filter by search pattern + var filtered = allIndices.stream() + .filter(idx -> search.isEmpty() || idx.get("index").contains(search)) + .filter(idx -> "ALL".equals(health) || health.equalsIgnoreCase(idx.get("health"))) + .toList(); + + // Compute summary totals before pagination + long totalDocs = filtered.stream().mapToLong(idx -> parseLong(idx.get("docs.count"))).sum(); + long totalBytes = filtered.stream().mapToLong(idx -> parseLong(idx.get("store.size"))).sum(); + + // Sort + Comparator> comparator = switch (sort) { + case "docs" -> Comparator.comparingLong(m -> parseLong(m.get("docs.count"))); + case "size" -> Comparator.comparingLong(m -> parseLong(m.get("store.size"))); + case "health" -> Comparator.comparing(m -> m.get("health")); + default -> Comparator.comparing(m -> m.get("index")); + }; + if ("desc".equalsIgnoreCase(order)) comparator = comparator.reversed(); + + // Paginate + var sorted = filtered.stream().sorted(comparator).toList(); + int fromIndex = page * effectiveSize; + int toIndex = Math.min(fromIndex + effectiveSize, sorted.size()); + var pageItems = fromIndex < sorted.size() + ? sorted.subList(fromIndex, toIndex) : List.>of(); + + var indices = pageItems.stream().map(m -> new IndexInfoResponse( + m.get("index"), parseLong(m.get("docs.count")), + humanSize(parseLong(m.get("store.size"))), + parseLong(m.get("store.size")), + m.getOrDefault("health", "unknown"), + parseInt(m.get("pri")), parseInt(m.get("rep")) + )).toList(); + + int totalPages = (int) Math.ceil((double) filtered.size() / effectiveSize); + return ResponseEntity.ok(new IndicesPageResponse( + indices, filtered.size(), totalDocs, + humanSize(totalBytes), page, effectiveSize, totalPages)); + } catch (Exception e) { + throw new ResponseStatusException(HttpStatus.BAD_GATEWAY, "OpenSearch unreachable"); + } + } + + private long parseLong(String s) { try { return Long.parseLong(s); } catch (Exception e) { return 0; } } + private int parseInt(String s) { try { return Integer.parseInt(s); } catch (Exception e) { return 0; } } + private String humanSize(long bytes) { + if (bytes < 1024) return bytes + " B"; + if (bytes < 1024 * 1024) return String.format("%.1f KB", bytes / 1024.0); + if (bytes < 1024L * 1024 * 1024) return String.format("%.1f MB", bytes / (1024.0 * 1024)); + return String.format("%.1f GB", bytes / (1024.0 * 1024 * 1024)); + } + + @DeleteMapping("/indices/{name}") + public ResponseEntity deleteIndex(@PathVariable String name, HttpServletRequest request) { + try { + var exists = client.indices().exists(e -> e.index(name)); + if (!exists.value()) { + throw new ResponseStatusException(HttpStatus.NOT_FOUND, "Index not found: " + name); + } + client.indices().delete(d -> d.index(name)); + auditService.log("delete_index", AuditCategory.INFRA, name, null, AuditResult.SUCCESS, request); + return ResponseEntity.ok().build(); + } catch (ResponseStatusException e) { throw e; } + catch (Exception e) { + throw new ResponseStatusException(HttpStatus.BAD_GATEWAY, "OpenSearch unreachable"); + } + } + + @GetMapping("/performance") + public ResponseEntity getPerformance() { + try { + // Use RestClient for _nodes/stats — the Java client exposes this via NodesStatsRequest + Request statsRequest = new Request("GET", "/_nodes/stats/jvm,indices"); + Response statsResponse = restClient.performRequest(statsRequest); + var statsJson = objectMapper.readTree(statsResponse.getEntity().getContent()); + + // Aggregate across all nodes + var nodes = statsJson.get("nodes"); + long jvmHeapUsed = 0, jvmHeapMax = 0; + long queryCacheHits = 0, queryCacheMisses = 0; + long requestCacheHits = 0, requestCacheMisses = 0; + long searchTotal = 0, searchTimeMs = 0; + long indexingTotal = 0, indexingTimeMs = 0; + + var nodeIter = nodes.fields(); + while (nodeIter.hasNext()) { + var node = nodeIter.next().getValue(); + var jvm = node.get("jvm").get("mem"); + jvmHeapUsed += jvm.get("heap_used_in_bytes").asLong(); + jvmHeapMax += jvm.get("heap_max_in_bytes").asLong(); + + var indices = node.get("indices"); + var qc = indices.get("query_cache"); + queryCacheHits += qc.get("hit_count").asLong(); + queryCacheMisses += qc.get("miss_count").asLong(); + var rc = indices.get("request_cache"); + requestCacheHits += rc.get("hit_count").asLong(); + requestCacheMisses += rc.get("miss_count").asLong(); + var search = indices.get("search"); + searchTotal += search.get("query_total").asLong(); + searchTimeMs += search.get("query_time_in_millis").asLong(); + var indexing = indices.get("indexing"); + indexingTotal += indexing.get("index_total").asLong(); + indexingTimeMs += indexing.get("index_time_in_millis").asLong(); + } + + double qcHitRate = (queryCacheHits + queryCacheMisses) > 0 + ? 100.0 * queryCacheHits / (queryCacheHits + queryCacheMisses) : 0; + double rcHitRate = (requestCacheHits + requestCacheMisses) > 0 + ? 100.0 * requestCacheHits / (requestCacheHits + requestCacheMisses) : 0; + double avgSearchLatency = searchTotal > 0 ? (double) searchTimeMs / searchTotal : 0; + double avgIndexingLatency = indexingTotal > 0 ? (double) indexingTimeMs / indexingTotal : 0; + + return ResponseEntity.ok(new PerformanceResponse( + qcHitRate, rcHitRate, avgSearchLatency, avgIndexingLatency, + jvmHeapUsed, jvmHeapMax)); + } catch (Exception e) { + throw new ResponseStatusException(HttpStatus.BAD_GATEWAY, "OpenSearch unreachable"); + } + } +} +``` + +**Note on OpenSearch client access:** The `OpenSearchClient` bean is already available. For some operations (like `_cat/indices` with sorting), you may need to use the underlying `RestClient` directly with a raw HTTP request, since the Java client may not expose all `_cat` parameters. Alternatively, use the `_cat/indices` API via `RestClient.performRequest()` with `format=json`. + +- [ ] **Step 4: Run integration test** + +Run: `mvn test -pl cameleer3-server-app -Dtest=OpenSearchAdminControllerIT` +Expected: PASS + +- [ ] **Step 5: Commit** + +```bash +git add cameleer3-server-app/ +git commit -m "feat: add OpenSearchAdminController with status, pipeline, indices, performance, and delete endpoints" +``` + +--- + +### Task 10: App Module — ThresholdAdminController + AuditLogController + +**Files:** +- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/ThresholdAdminController.java` +- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/controller/AuditLogController.java` +- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/ThresholdAdminControllerIT.java` +- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/AuditLogControllerIT.java` + +- [ ] **Step 1: Write ThresholdAdminController integration test** + +- `GET /api/v1/admin/thresholds` → 200, returns defaults if no row exists +- `PUT /api/v1/admin/thresholds` with valid payload → 200 +- `GET /api/v1/admin/thresholds` after save → returns saved values +- `PUT /api/v1/admin/thresholds` with warning > critical → 400 + +- [ ] **Step 2: Write AuditLogController integration test** + +- `GET /api/v1/admin/audit` → 200, returns paginated results +- `GET /api/v1/admin/audit?category=INFRA` → 200, only INFRA entries +- `GET /api/v1/admin/audit?search=kill_query` → 200, filtered results +- Verify that the audit log captures actions from other admin endpoints (e.g., call kill query, then check audit log) + +- [ ] **Step 3: Implement ThresholdAdminController** + +```java +@RestController +@RequestMapping("/api/v1/admin/thresholds") +@PreAuthorize("hasRole('ADMIN')") +@Tag(name = "Threshold Admin", description = "Threshold configuration (ADMIN only)") +public class ThresholdAdminController { + private final ThresholdRepository repository; + private final AuditService auditService; + + @GetMapping + public ResponseEntity get() { + return ResponseEntity.ok(repository.find().orElse(ThresholdConfig.defaults())); + } + + @PutMapping + public ResponseEntity save(@Valid @RequestBody ThresholdConfigRequest request, + HttpServletRequest httpRequest) { + var errors = request.validate(); + if (!errors.isEmpty()) { + throw new ResponseStatusException(HttpStatus.BAD_REQUEST, + "Validation failed: " + String.join("; ", errors)); + } + ThresholdConfig config = request.toConfig(); + // Extract username from SecurityContext (same approach as AuditService) + Authentication auth = SecurityContextHolder.getContext().getAuthentication(); + String username = auth != null ? auth.getName() : "unknown"; + if (username.startsWith("user:")) username = username.substring(5); + + repository.save(config, username); + auditService.log("update_thresholds", AuditCategory.INFRA, "thresholds", + Map.of("config", config), AuditResult.SUCCESS, httpRequest); + return ResponseEntity.ok(config); + } +} +``` + +- [ ] **Step 4: Implement AuditLogController** + +```java +@RestController +@RequestMapping("/api/v1/admin/audit") +@PreAuthorize("hasRole('ADMIN')") +@Tag(name = "Audit Log", description = "Audit log viewer (ADMIN only)") +public class AuditLogController { + private final AuditRepository repository; + + @GetMapping + public ResponseEntity getAuditLog( + @RequestParam(required = false) String username, + @RequestParam(required = false) AuditCategory category, + @RequestParam(required = false) String search, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate from, + @RequestParam(required = false) @DateTimeFormat(iso = DateTimeFormat.ISO.DATE) LocalDate to, + @RequestParam(defaultValue = "timestamp") String sort, + @RequestParam(defaultValue = "desc") String order, + @RequestParam(defaultValue = "0") int page, + @RequestParam(defaultValue = "25") int size) { + size = Math.min(size, 100); + // Convert LocalDate to Instant (start of day / end of day in UTC) + Instant fromInstant = (from != null ? from : LocalDate.now().minusDays(7)) + .atStartOfDay(ZoneOffset.UTC).toInstant(); + Instant toInstant = (to != null ? to.plusDays(1) : LocalDate.now().plusDays(1)) + .atStartOfDay(ZoneOffset.UTC).toInstant(); + var query = new AuditRepository.AuditQuery(username, category, search, fromInstant, toInstant, sort, order, page, size); + var result = repository.find(query); + int totalPages = (int) Math.ceil((double) result.totalCount() / size); + return ResponseEntity.ok(new AuditLogPageResponse(result.items(), result.totalCount(), page, size, totalPages)); + } +} +``` + +- [ ] **Step 5: Run tests** + +Run: `mvn test -pl cameleer3-server-app -Dtest="ThresholdAdminControllerIT,AuditLogControllerIT"` +Expected: PASS + +- [ ] **Step 6: Commit** + +```bash +git add cameleer3-server-app/ +git commit -m "feat: add ThresholdAdminController and AuditLogController with integration tests" +``` + +--- + +### Task 11: Regenerate OpenAPI Spec + +**Files:** +- Modify: `ui/src/api/schema/openapi.json` (regenerated) +- Modify: `ui/src/api/schema.d.ts` (regenerated) + +- [ ] **Step 1: Start the server and regenerate** + +The project regenerates `openapi.json` from the running server. Follow the pattern from `feedback_regenerate_openapi.md`: + +1. Start the server (needs PG + OpenSearch running, or use test profile) +2. Fetch OpenAPI JSON: `curl http://localhost:8081/api/v1/api-docs > ui/src/api/schema/openapi.json` +3. Regenerate TypeScript types: `cd ui && npx openapi-typescript src/api/schema/openapi.json -o src/api/schema.d.ts` + +- [ ] **Step 2: Verify frontend compiles** + +Run: `cd ui && npm run build` +Expected: BUILD SUCCESS (or only warnings, no errors) + +- [ ] **Step 3: Commit** + +```bash +git add ui/src/api/schema/ +git commit -m "chore: regenerate OpenAPI spec and TypeScript types for admin endpoints" +``` + +--- + +### Task 12: Frontend — Shared Admin Components + +**Files:** +- Create: `ui/src/components/admin/StatusBadge.tsx` + `.module.css` +- Create: `ui/src/components/admin/RefreshableCard.tsx` + `.module.css` +- Create: `ui/src/components/admin/ConfirmDeleteDialog.tsx` + `.module.css` + +- [ ] **Step 1: Create StatusBadge component** + +A small colored dot + label. Props: `status: 'healthy' | 'warning' | 'critical' | 'unknown'`, `label?: string`. Colors: green/yellow/red/gray. Uses CSS modules consistent with existing component styling. + +```tsx +import styles from './StatusBadge.module.css'; + +type Status = 'healthy' | 'warning' | 'critical' | 'unknown'; + +export function StatusBadge({ status, label }: { status: Status; label?: string }) { + return ( + + + {label && {label}} + + ); +} +``` + +- [ ] **Step 2: Create RefreshableCard component** + +A collapsible card with title, optional auto-refresh indicator, and manual refresh button. Props: `title`, `onRefresh`, `isRefreshing`, `autoRefresh?: boolean`, `children`. + +- [ ] **Step 3: Create ConfirmDeleteDialog component** + +Modal dialog. Props: `isOpen`, `onClose`, `onConfirm`, `resourceName`, `resourceType`. Requires user to type the resource name to confirm. Disabled confirm button until input matches. + +- [ ] **Step 4: Verify frontend compiles** + +Run: `cd ui && npm run build` + +- [ ] **Step 5: Commit** + +```bash +git add ui/src/components/admin/ +git commit -m "feat: add shared admin UI components (StatusBadge, RefreshableCard, ConfirmDeleteDialog)" +``` + +--- + +### Task 13: Frontend — Admin Sidebar + Routing + +**Files:** +- Modify: `ui/src/components/layout/AppSidebar.tsx` +- Modify: `ui/src/router.tsx` + +- [ ] **Step 1: Update router.tsx** + +Add new admin routes inside the `AppShell` children array. Add redirect for `/admin`: + +```tsx +{ path: 'admin', element: }, +{ path: 'admin/database', element: }, +{ path: 'admin/opensearch', element: }, +{ path: 'admin/audit', element: }, +{ path: 'admin/oidc', element: }, +``` + +Use lazy imports for the new pages (consistent with SwaggerPage pattern). + +- [ ] **Step 2: Refactor AppSidebar admin section** + +Replace the single gear-icon link with a collapsible section: + +```tsx +// Admin section (visible only to ADMIN role) +{roles.includes('ADMIN') && ( +
+ + {adminOpen && ( + + )} +
+)} +``` + +Persist `adminOpen` state in localStorage (key: `cameleer-admin-sidebar-open`). Default: collapsed. + +- [ ] **Step 3: Verify frontend compiles and routing works** + +Run: `cd ui && npm run build` + +- [ ] **Step 4: Commit** + +```bash +git add ui/src/router.tsx ui/src/components/layout/ +git commit -m "feat: restructure admin sidebar with collapsible sub-navigation and new routes" +``` + +--- + +### Task 14: Frontend — React Query Hooks + +**Files:** +- Create: `ui/src/api/queries/admin/database.ts` +- Create: `ui/src/api/queries/admin/opensearch.ts` +- Create: `ui/src/api/queries/admin/thresholds.ts` +- Create: `ui/src/api/queries/admin/audit.ts` + +- [ ] **Step 1: Create database query hooks** + +Follow the exact pattern from `oidc-admin.ts`: + +```typescript +import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query'; +import { api } from '../../client'; + +export function useDatabaseStatus() { + return useQuery({ + queryKey: ['admin', 'database', 'status'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/database/status'); + if (error) throw new Error('Failed to load database status'); + return data!; + }, + }); +} + +export function useDatabasePool() { + return useQuery({ + queryKey: ['admin', 'database', 'pool'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/database/pool'); + if (error) throw new Error('Failed to load pool stats'); + return data!; + }, + refetchInterval: 15_000, + }); +} + +export function useDatabaseTables() { + return useQuery({ + queryKey: ['admin', 'database', 'tables'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/database/tables'); + if (error) throw new Error('Failed to load table sizes'); + return data!; + }, + // NO refetchInterval — manual refresh only + }); +} + +export function useDatabaseQueries() { + return useQuery({ + queryKey: ['admin', 'database', 'queries'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/database/queries'); + if (error) throw new Error('Failed to load active queries'); + return data!; + }, + refetchInterval: 15_000, + }); +} + +export function useKillQuery() { + const qc = useQueryClient(); + return useMutation({ + mutationFn: async (pid: number) => { + const { error } = await api.POST('/admin/database/queries/{pid}/kill', { params: { path: { pid } } }); + if (error) throw new Error('Failed to kill query'); + }, + onSuccess: () => qc.invalidateQueries({ queryKey: ['admin', 'database', 'queries'] }), + }); +} +``` + +- [ ] **Step 2: Create OpenSearch query hooks** + +Same pattern as database hooks. Create: + +```typescript +export function useOpenSearchStatus() { + return useQuery({ + queryKey: ['admin', 'opensearch', 'status'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/opensearch/status'); + if (error) throw new Error('Failed to load OpenSearch status'); + return data!; + }, + }); +} + +export function usePipelineStats() { + return useQuery({ + queryKey: ['admin', 'opensearch', 'pipeline'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/opensearch/pipeline'); + if (error) throw new Error('Failed to load pipeline stats'); + return data!; + }, + refetchInterval: 15_000, + }); +} + +export function useIndices(params: { search?: string; health?: string; sort?: string; order?: string; page?: number; size?: number }) { + return useQuery({ + queryKey: ['admin', 'opensearch', 'indices', params], + queryFn: async () => { + const { data, error } = await api.GET('/admin/opensearch/indices', { params: { query: params } }); + if (error) throw new Error('Failed to load indices'); + return data!; + }, + // NO refetchInterval — manual refresh only + }); +} + +export function usePerformanceStats() { + return useQuery({ + queryKey: ['admin', 'opensearch', 'performance'], + queryFn: async () => { + const { data, error } = await api.GET('/admin/opensearch/performance'); + if (error) throw new Error('Failed to load performance stats'); + return data!; + }, + refetchInterval: 15_000, + }); +} + +export function useDeleteIndex() { + const qc = useQueryClient(); + return useMutation({ + mutationFn: async (name: string) => { + const { error } = await api.DELETE('/admin/opensearch/indices/{name}', { params: { path: { name } } }); + if (error) throw new Error('Failed to delete index'); + }, + onSuccess: () => qc.invalidateQueries({ queryKey: ['admin', 'opensearch', 'indices'] }), + }); +} +``` + +- [ ] **Step 3: Create threshold hooks** + +`useThresholds()` query + `useSaveThresholds()` mutation. + +- [ ] **Step 4: Create audit log hooks** + +`useAuditLog()` query that accepts filter params (username, category, search, from, to, page, size) and includes them in the query key for proper cache management. + +- [ ] **Step 5: Verify frontend compiles** + +Run: `cd ui && npm run build` + +- [ ] **Step 6: Commit** + +```bash +git add ui/src/api/queries/admin/ +git commit -m "feat: add React Query hooks for database, OpenSearch, threshold, and audit log admin endpoints" +``` + +--- + +### Task 15: Frontend — Database Admin Page + +**Files:** +- Create: `ui/src/pages/admin/DatabaseAdminPage.tsx` + `.module.css` + +- [ ] **Step 1: Implement DatabaseAdminPage** + +Page structure (follow `OidcAdminPage` patterns for role check, loading states, error handling): + +1. **Header**: StatusBadge (connected/disconnected) + version + host + schema + manual refresh-all button +2. **Connection Pool card** (RefreshableCard, auto-refresh): progress bar showing active/max, metrics grid +3. **Table Sizes card** (RefreshableCard, manual): table with sortable columns, summary row +4. **Active Queries card** (RefreshableCard, auto-refresh): table with Kill button per row (behind confirmation dialog) +5. **Maintenance card** (Phase 2 placeholder): greyed-out buttons with tooltip +6. **Thresholds section** (collapsible): form inputs for warning/critical values, save button + +Use the `useThresholds()` hook to load current threshold values and apply them to StatusBadge calculations (e.g., pool usage % > warning → yellow badge). + +- [ ] **Step 2: Style the page** + +CSS modules, consistent with existing admin page styling from `OidcAdminPage.module.css`. + +- [ ] **Step 3: Verify page renders** + +Run: `cd ui && npm run build` + +- [ ] **Step 4: Commit** + +```bash +git add ui/src/pages/admin/DatabaseAdminPage.tsx ui/src/pages/admin/DatabaseAdminPage.module.css +git commit -m "feat: add Database admin page with pool, tables, queries, and thresholds UI" +``` + +--- + +### Task 16: Frontend — OpenSearch Admin Page + +**Files:** +- Create: `ui/src/pages/admin/OpenSearchAdminPage.tsx` + `.module.css` + +- [ ] **Step 1: Implement OpenSearchAdminPage** + +Page structure: + +1. **Header**: StatusBadge (cluster health) + version + nodes + host + manual refresh-all +2. **Indexing Pipeline card** (auto-refresh): queue depth bar, metrics grid, status badge +3. **Indices card** (manual refresh): search input, health filter dropdown, sortable/paginated table, delete button per row (ConfirmDeleteDialog with name-typing confirmation), summary row above table +4. **Performance card** (auto-refresh): cache hit rates, latencies, JVM heap bar +5. **Operations card** (Phase 2 placeholder): greyed-out buttons +6. **Thresholds section** (collapsible): form inputs for OpenSearch-specific thresholds + +- [ ] **Step 2: Style the page** + +- [ ] **Step 3: Verify page renders** + +Run: `cd ui && npm run build` + +- [ ] **Step 4: Commit** + +```bash +git add ui/src/pages/admin/OpenSearchAdminPage.tsx ui/src/pages/admin/OpenSearchAdminPage.module.css +git commit -m "feat: add OpenSearch admin page with pipeline, indices, performance, and thresholds UI" +``` + +--- + +### Task 17: Frontend — Audit Log Page + +**Files:** +- Create: `ui/src/pages/admin/AuditLogPage.tsx` + `.module.css` + +- [ ] **Step 1: Implement AuditLogPage** + +Page structure: + +1. **Header**: total event count + date range picker (two date inputs, default last 7 days) +2. **Filters row**: username dropdown (populated from distinct values or free text), category dropdown (INFRA/AUTH/USER_MGMT/CONFIG/All), free text search input +3. **Table**: columns — Timestamp, User, Category, Action, Target, Result. Click row to expand and show full `detail` JSON (formatted/pretty-printed) +4. **Pagination**: page controls below table, showing "Showing X-Y of Z" + +No auto-refresh. Read-only — no edit or delete buttons anywhere. + +- [ ] **Step 2: Style the page** + +- [ ] **Step 3: Verify page renders** + +Run: `cd ui && npm run build` + +- [ ] **Step 4: Commit** + +```bash +git add ui/src/pages/admin/AuditLogPage.tsx ui/src/pages/admin/AuditLogPage.module.css +git commit -m "feat: add Audit Log admin page with filtering, pagination, and detail expansion" +``` + +--- + +### Task 18: End-to-End Verification + +- [ ] **Step 1: Run full backend build** + +Run: `mvn clean verify` +Expected: BUILD SUCCESS with all tests passing + +- [ ] **Step 2: Run full frontend build** + +Run: `cd ui && npm run build` +Expected: BUILD SUCCESS + +- [ ] **Step 3: Manual smoke test** + +Start the full stack (PG + OpenSearch + server) and verify: +1. Admin sidebar shows collapsible sub-menu with Database, OpenSearch, Audit Log, OIDC +2. Database page shows connection status, pool stats, table sizes, active queries +3. OpenSearch page shows cluster health, pipeline stats, indices list, performance +4. Audit Log page shows entries from admin actions +5. Kill query and delete index work with confirmation dialogs +6. Thresholds save and load correctly +7. Non-admin users cannot see admin sidebar or access admin API endpoints (verify 403) + +- [ ] **Step 4: Final commit if any fixes needed** + +- [ ] **Step 5: Update HOWTO.md** + +Add section for new admin pages — how to access, what each page shows, how thresholds work. + +```bash +git add HOWTO.md +git commit -m "docs: update HOWTO.md with admin infrastructure pages" +``` + +- [ ] **Step 6: Push to Gitea** + +```bash +git push origin main +```