chore: update docs and config for PostgreSQL/OpenSearch storage layer

- Set failsafe reuseForks=true to reuse JVM across IT classes (faster test suite) - Replace ClickHouse with PostgreSQL+OpenSearch in docker-compose.yml - Remove redundant docker-compose.dev.yml - Update CLAUDE.md and HOWTO.md to reflect new storage stack Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-17 00:26:50 +01:00
parent 796be06a09
commit c316e80d7f
5 changed files with 49 additions and 69 deletions
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -38,10 +38,10 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
 - Jackson `JavaTimeModule` for `Instant` deserialization
 - Communication: receives HTTP POST data from agents, serves SSE event streams for config push/commands
 - Maintains agent instance registry with states: LIVE → STALE → DEAD
- Storage: ClickHouse for structured data, text index for full-text search
+- Storage: PostgreSQL (TimescaleDB) for structured data, OpenSearch for full-text search
 - Security: JWT auth with RBAC (AGENT/VIEWER/OPERATOR/ADMIN roles), Ed25519 config signing, bootstrap token for registration
 - OIDC: Optional external identity provider support (token exchange pattern). Configured via `CAMELEER_OIDC_*` env vars
- User persistence: ClickHouse `users` table, admin CRUD at `/api/v1/admin/users`
+- User persistence: PostgreSQL `users` table, admin CRUD at `/api/v1/admin/users`

 ## CI/CD & Deployment

@@ -50,8 +50,8 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
 - Docker: multi-stage build (`Dockerfile`), `$BUILDPLATFORM` for native Maven on ARM64 runner, amd64 runtime
 - `REGISTRY_TOKEN` build arg required for `cameleer3-common` dependency resolution
 - Registry: `gitea.siegeln.net/cameleer/cameleer3-server` (container images)
- K8s manifests in `deploy/` — ClickHouse StatefulSet + server Deployment + NodePort Service (30081)
+- K8s manifests in `deploy/` — PostgreSQL + OpenSearch StatefulSets, server Deployment + NodePort Service (30081)
 - Deployment target: k3s at 192.168.50.86, namespace `cameleer`
- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `clickhouse-credentials`, `CAMELEER_JWT_SECRET`
- K8s probes: server uses `/api/v1/health`, ClickHouse uses `/ping`
+- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `postgres-credentials`, `opensearch-credentials`, `CAMELEER_JWT_SECRET`
+- K8s probes: server uses `/api/v1/health`, PostgreSQL uses `pg_isready`, OpenSearch uses `/_cluster/health`
 - Docker build uses buildx registry cache + `--provenance=false` for Gitea compatibility
--- a/HOWTO.md
+++ b/HOWTO.md
@@ -21,20 +21,20 @@ mvn clean verify           # compile + run all tests (needs Docker for integrati

 ## Infrastructure Setup

-Start ClickHouse:
+Start PostgreSQL and OpenSearch:

 ```bash
 docker compose up -d
 ```

-This starts ClickHouse 25.3 and automatically runs the schema init scripts (`clickhouse/init/01-schema.sql`, `clickhouse/init/02-search-columns.sql`, `clickhouse/init/03-users.sql`).
+This starts TimescaleDB (PostgreSQL 16) and OpenSearch 2.19. The database schema is applied automatically via Flyway migrations on server startup.

 | Service    | Port | Purpose              |
-|------------|------|------------------|
-| ClickHouse | 8123 | HTTP API (JDBC)  |
-| ClickHouse | 9000 | Native protocol  |
+|------------|------|----------------------|
+| PostgreSQL | 5432 | JDBC (Spring JDBC)   |
+| OpenSearch | 9200 | REST API (full-text)  |

-ClickHouse credentials: `cameleer` / `cameleer_dev`, database `cameleer3`.
+PostgreSQL credentials: `cameleer` / `cameleer_dev`, database `cameleer3`.

 ## Run the Server

@@ -109,7 +109,7 @@ The env-var local user gets `ADMIN` role. Agents get `AGENT` role at registratio

 ### OIDC Login (Optional)

-OIDC configuration is stored in ClickHouse and managed via the admin API or UI. The SPA checks if OIDC is available:
+OIDC configuration is stored in PostgreSQL and managed via the admin API or UI. The SPA checks if OIDC is available:

 ```bash
 # 1. SPA checks if OIDC is available (returns 404 if not configured)
@@ -340,9 +340,8 @@ Key settings in `cameleer3-server-app/src/main/resources/application.yml`:
 |---------|---------|-------------|
 | `server.port` | 8081 | Server port |
 | `ingestion.buffer-capacity` | 50000 | Max items in write buffer |
-| `ingestion.batch-size` | 5000 | Items per ClickHouse batch insert |
+| `ingestion.batch-size` | 5000 | Items per batch insert |
 | `ingestion.flush-interval-ms` | 1000 | Buffer flush interval (ms) |
-| `ingestion.data-ttl-days` | 30 | ClickHouse TTL for auto-deletion |
 | `agent-registry.heartbeat-interval-seconds` | 30 | Expected heartbeat interval |
 | `agent-registry.stale-threshold-seconds` | 90 | Time before agent marked STALE |
 | `agent-registry.dead-threshold-seconds` | 300 | Time after STALE before DEAD |
@@ -386,7 +385,7 @@ npm run generate-api   # Requires backend running on :8081

 ## Running Tests

-Integration tests use Testcontainers (starts ClickHouse automatically — requires Docker):
+Integration tests use Testcontainers (starts PostgreSQL and OpenSearch automatically — requires Docker):

 ```bash
 # All tests
@@ -399,14 +398,13 @@ mvn test -pl cameleer3-server-core
 mvn test -pl cameleer3-server-app -Dtest=ExecutionControllerIT
 ```

-## Verify ClickHouse Data
+## Verify Database Data

 After posting data and waiting for the flush interval (1s default):

 ```bash
-docker exec -it cameleer3-server-clickhouse-1 clickhouse-client \
-  --user cameleer --password cameleer_dev -d cameleer3 \
-  -q "SELECT count() FROM route_executions"
+docker exec -it cameleer3-server-postgres-1 psql -U cameleer -d cameleer3 \
+  -c "SELECT count(*) FROM route_executions"
 ```

 ## Kubernetes Deployment
@@ -417,7 +415,8 @@ The full stack is deployed to k3s via CI/CD on push to `main`. K8s manifests are

 ```
 cameleer namespace:
-  ClickHouse (StatefulSet, 2Gi PVC)        ← clickhouse:8123 (ClusterIP)
+  PostgreSQL (StatefulSet, 10Gi PVC)       ← postgres:5432 (ClusterIP)
+  OpenSearch (StatefulSet, 10Gi PVC)       ← opensearch:9200 (ClusterIP)
  cameleer3-server (Deployment)            ← NodePort 30081
  cameleer3-ui (Deployment, Nginx)         ← NodePort 30090
  Authentik Server (Deployment)            ← NodePort 30950
@@ -439,7 +438,7 @@ cameleer namespace:

 Push to `main` triggers: **build** (UI npm + Maven, unit tests) → **docker** (buildx amd64 for server + UI, push to Gitea registry) → **deploy** (kubectl apply + rolling update).

-Required Gitea org secrets: `REGISTRY_TOKEN`, `KUBECONFIG_BASE64`, `CAMELEER_AUTH_TOKEN`, `CAMELEER_JWT_SECRET`, `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, `CAMELEER_UI_USER` (optional), `CAMELEER_UI_PASSWORD` (optional), `AUTHENTIK_PG_PASSWORD`, `AUTHENTIK_SECRET_KEY`, `CAMELEER_OIDC_ENABLED`, `CAMELEER_OIDC_ISSUER`, `CAMELEER_OIDC_CLIENT_ID`, `CAMELEER_OIDC_CLIENT_SECRET`.
+Required Gitea org secrets: `REGISTRY_TOKEN`, `KUBECONFIG_BASE64`, `CAMELEER_AUTH_TOKEN`, `CAMELEER_JWT_SECRET`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`, `OPENSEARCH_USER`, `OPENSEARCH_PASSWORD`, `CAMELEER_UI_USER` (optional), `CAMELEER_UI_PASSWORD` (optional), `AUTHENTIK_PG_USER`, `AUTHENTIK_PG_PASSWORD`, `AUTHENTIK_SECRET_KEY`, `CAMELEER_OIDC_ENABLED`, `CAMELEER_OIDC_ISSUER`, `CAMELEER_OIDC_CLIENT_ID`, `CAMELEER_OIDC_CLIENT_SECRET`.

 ### Manual K8s Commands

@@ -450,8 +449,11 @@ kubectl -n cameleer get pods
 # View server logs
 kubectl -n cameleer logs -f deploy/cameleer3-server

-# View ClickHouse logs
-kubectl -n cameleer logs -f statefulset/clickhouse
+# View PostgreSQL logs
+kubectl -n cameleer logs -f statefulset/postgres
+
+# View OpenSearch logs
+kubectl -n cameleer logs -f statefulset/opensearch

 # Restart server
 kubectl -n cameleer rollout restart deployment/cameleer3-server
--- a/cameleer3-server-app/pom.xml
+++ b/cameleer3-server-app/pom.xml
@@ -174,7 +174,7 @@
                <artifactId>maven-failsafe-plugin</artifactId>
                <configuration>
                    <forkCount>1</forkCount>
-                    <reuseForks>false</reuseForks>
+                    <reuseForks>true</reuseForks>
                </configuration>
                <executions>
                    <execution>
--- a/docker-compose.dev.yml
+++ b/docker-compose.dev.yml
@@ -1,29 +0,0 @@
-version: '3.8'
-
-services:
-  postgres:
-    image: timescale/timescaledb-ha:pg16
-    ports:
-      - "5432:5432"
-    environment:
-      POSTGRES_DB: cameleer3
-      POSTGRES_USER: cameleer
-      POSTGRES_PASSWORD: cameleer_dev
-    volumes:
-      - pgdata:/home/postgres/pgdata/data
-
-  opensearch:
-    image: opensearchproject/opensearch:2.19.0
-    ports:
-      - "9200:9200"
-      - "9300:9300"
-    environment:
-      discovery.type: single-node
-      DISABLE_SECURITY_PLUGIN: "true"
-      OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
-    volumes:
-      - osdata:/usr/share/opensearch/data
-
-volumes:
-  pgdata:
-  osdata:
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,20 +1,27 @@
 services:
-  clickhouse:
-    image: clickhouse/clickhouse-server:25.3
+  postgres:
+    image: timescale/timescaledb-ha:pg16
    ports:
-      - "8123:8123"
-      - "9000:9000"
-    volumes:
-      - clickhouse-data:/var/lib/clickhouse
-      - ./clickhouse/init:/docker-entrypoint-initdb.d
+      - "5432:5432"
    environment:
-      CLICKHOUSE_USER: cameleer
-      CLICKHOUSE_PASSWORD: cameleer_dev
-      CLICKHOUSE_DB: cameleer3
-    ulimits:
-      nofile:
-        soft: 262144
-        hard: 262144
+      POSTGRES_DB: cameleer3
+      POSTGRES_USER: cameleer
+      POSTGRES_PASSWORD: cameleer_dev
+    volumes:
+      - pgdata:/home/postgres/pgdata/data
+
+  opensearch:
+    image: opensearchproject/opensearch:2.19.0
+    ports:
+      - "9200:9200"
+      - "9300:9300"
+    environment:
+      discovery.type: single-node
+      DISABLE_SECURITY_PLUGIN: "true"
+      OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
+    volumes:
+      - osdata:/usr/share/opensearch/data

 volumes:
-  clickhouse-data:
+  pgdata:
+  osdata: