diff --git a/CLAUDE.md b/CLAUDE.md index 28513ac8..e3a8fddd 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -38,10 +38,10 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar - Jackson `JavaTimeModule` for `Instant` deserialization - Communication: receives HTTP POST data from agents, serves SSE event streams for config push/commands - Maintains agent instance registry with states: LIVE → STALE → DEAD -- Storage: ClickHouse for structured data, text index for full-text search +- Storage: PostgreSQL (TimescaleDB) for structured data, OpenSearch for full-text search - Security: JWT auth with RBAC (AGENT/VIEWER/OPERATOR/ADMIN roles), Ed25519 config signing, bootstrap token for registration - OIDC: Optional external identity provider support (token exchange pattern). Configured via `CAMELEER_OIDC_*` env vars -- User persistence: ClickHouse `users` table, admin CRUD at `/api/v1/admin/users` +- User persistence: PostgreSQL `users` table, admin CRUD at `/api/v1/admin/users` ## CI/CD & Deployment @@ -50,8 +50,8 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar - Docker: multi-stage build (`Dockerfile`), `$BUILDPLATFORM` for native Maven on ARM64 runner, amd64 runtime - `REGISTRY_TOKEN` build arg required for `cameleer3-common` dependency resolution - Registry: `gitea.siegeln.net/cameleer/cameleer3-server` (container images) -- K8s manifests in `deploy/` — ClickHouse StatefulSet + server Deployment + NodePort Service (30081) +- K8s manifests in `deploy/` — PostgreSQL + OpenSearch StatefulSets, server Deployment + NodePort Service (30081) - Deployment target: k3s at 192.168.50.86, namespace `cameleer` -- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `clickhouse-credentials`, `CAMELEER_JWT_SECRET` -- K8s probes: server uses `/api/v1/health`, ClickHouse uses `/ping` +- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `postgres-credentials`, `opensearch-credentials`, `CAMELEER_JWT_SECRET` +- K8s probes: server uses `/api/v1/health`, PostgreSQL uses `pg_isready`, OpenSearch uses `/_cluster/health` - Docker build uses buildx registry cache + `--provenance=false` for Gitea compatibility diff --git a/HOWTO.md b/HOWTO.md index a1fd9dec..54c52557 100644 --- a/HOWTO.md +++ b/HOWTO.md @@ -21,20 +21,20 @@ mvn clean verify # compile + run all tests (needs Docker for integrati ## Infrastructure Setup -Start ClickHouse: +Start PostgreSQL and OpenSearch: ```bash docker compose up -d ``` -This starts ClickHouse 25.3 and automatically runs the schema init scripts (`clickhouse/init/01-schema.sql`, `clickhouse/init/02-search-columns.sql`, `clickhouse/init/03-users.sql`). +This starts TimescaleDB (PostgreSQL 16) and OpenSearch 2.19. The database schema is applied automatically via Flyway migrations on server startup. -| Service | Port | Purpose | -|------------|------|------------------| -| ClickHouse | 8123 | HTTP API (JDBC) | -| ClickHouse | 9000 | Native protocol | +| Service | Port | Purpose | +|------------|------|----------------------| +| PostgreSQL | 5432 | JDBC (Spring JDBC) | +| OpenSearch | 9200 | REST API (full-text) | -ClickHouse credentials: `cameleer` / `cameleer_dev`, database `cameleer3`. +PostgreSQL credentials: `cameleer` / `cameleer_dev`, database `cameleer3`. ## Run the Server @@ -109,7 +109,7 @@ The env-var local user gets `ADMIN` role. Agents get `AGENT` role at registratio ### OIDC Login (Optional) -OIDC configuration is stored in ClickHouse and managed via the admin API or UI. The SPA checks if OIDC is available: +OIDC configuration is stored in PostgreSQL and managed via the admin API or UI. The SPA checks if OIDC is available: ```bash # 1. SPA checks if OIDC is available (returns 404 if not configured) @@ -340,9 +340,8 @@ Key settings in `cameleer3-server-app/src/main/resources/application.yml`: |---------|---------|-------------| | `server.port` | 8081 | Server port | | `ingestion.buffer-capacity` | 50000 | Max items in write buffer | -| `ingestion.batch-size` | 5000 | Items per ClickHouse batch insert | +| `ingestion.batch-size` | 5000 | Items per batch insert | | `ingestion.flush-interval-ms` | 1000 | Buffer flush interval (ms) | -| `ingestion.data-ttl-days` | 30 | ClickHouse TTL for auto-deletion | | `agent-registry.heartbeat-interval-seconds` | 30 | Expected heartbeat interval | | `agent-registry.stale-threshold-seconds` | 90 | Time before agent marked STALE | | `agent-registry.dead-threshold-seconds` | 300 | Time after STALE before DEAD | @@ -386,7 +385,7 @@ npm run generate-api # Requires backend running on :8081 ## Running Tests -Integration tests use Testcontainers (starts ClickHouse automatically — requires Docker): +Integration tests use Testcontainers (starts PostgreSQL and OpenSearch automatically — requires Docker): ```bash # All tests @@ -399,14 +398,13 @@ mvn test -pl cameleer3-server-core mvn test -pl cameleer3-server-app -Dtest=ExecutionControllerIT ``` -## Verify ClickHouse Data +## Verify Database Data After posting data and waiting for the flush interval (1s default): ```bash -docker exec -it cameleer3-server-clickhouse-1 clickhouse-client \ - --user cameleer --password cameleer_dev -d cameleer3 \ - -q "SELECT count() FROM route_executions" +docker exec -it cameleer3-server-postgres-1 psql -U cameleer -d cameleer3 \ + -c "SELECT count(*) FROM route_executions" ``` ## Kubernetes Deployment @@ -417,7 +415,8 @@ The full stack is deployed to k3s via CI/CD on push to `main`. K8s manifests are ``` cameleer namespace: - ClickHouse (StatefulSet, 2Gi PVC) ← clickhouse:8123 (ClusterIP) + PostgreSQL (StatefulSet, 10Gi PVC) ← postgres:5432 (ClusterIP) + OpenSearch (StatefulSet, 10Gi PVC) ← opensearch:9200 (ClusterIP) cameleer3-server (Deployment) ← NodePort 30081 cameleer3-ui (Deployment, Nginx) ← NodePort 30090 Authentik Server (Deployment) ← NodePort 30950 @@ -439,7 +438,7 @@ cameleer namespace: Push to `main` triggers: **build** (UI npm + Maven, unit tests) → **docker** (buildx amd64 for server + UI, push to Gitea registry) → **deploy** (kubectl apply + rolling update). -Required Gitea org secrets: `REGISTRY_TOKEN`, `KUBECONFIG_BASE64`, `CAMELEER_AUTH_TOKEN`, `CAMELEER_JWT_SECRET`, `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, `CAMELEER_UI_USER` (optional), `CAMELEER_UI_PASSWORD` (optional), `AUTHENTIK_PG_PASSWORD`, `AUTHENTIK_SECRET_KEY`, `CAMELEER_OIDC_ENABLED`, `CAMELEER_OIDC_ISSUER`, `CAMELEER_OIDC_CLIENT_ID`, `CAMELEER_OIDC_CLIENT_SECRET`. +Required Gitea org secrets: `REGISTRY_TOKEN`, `KUBECONFIG_BASE64`, `CAMELEER_AUTH_TOKEN`, `CAMELEER_JWT_SECRET`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`, `OPENSEARCH_USER`, `OPENSEARCH_PASSWORD`, `CAMELEER_UI_USER` (optional), `CAMELEER_UI_PASSWORD` (optional), `AUTHENTIK_PG_USER`, `AUTHENTIK_PG_PASSWORD`, `AUTHENTIK_SECRET_KEY`, `CAMELEER_OIDC_ENABLED`, `CAMELEER_OIDC_ISSUER`, `CAMELEER_OIDC_CLIENT_ID`, `CAMELEER_OIDC_CLIENT_SECRET`. ### Manual K8s Commands @@ -450,8 +449,11 @@ kubectl -n cameleer get pods # View server logs kubectl -n cameleer logs -f deploy/cameleer3-server -# View ClickHouse logs -kubectl -n cameleer logs -f statefulset/clickhouse +# View PostgreSQL logs +kubectl -n cameleer logs -f statefulset/postgres + +# View OpenSearch logs +kubectl -n cameleer logs -f statefulset/opensearch # Restart server kubectl -n cameleer rollout restart deployment/cameleer3-server diff --git a/cameleer3-server-app/pom.xml b/cameleer3-server-app/pom.xml index 81e21c59..4738822d 100644 --- a/cameleer3-server-app/pom.xml +++ b/cameleer3-server-app/pom.xml @@ -174,7 +174,7 @@ maven-failsafe-plugin 1 - false + true diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml deleted file mode 100644 index 1e6ecb58..00000000 --- a/docker-compose.dev.yml +++ /dev/null @@ -1,29 +0,0 @@ -version: '3.8' - -services: - postgres: - image: timescale/timescaledb-ha:pg16 - ports: - - "5432:5432" - environment: - POSTGRES_DB: cameleer3 - POSTGRES_USER: cameleer - POSTGRES_PASSWORD: cameleer_dev - volumes: - - pgdata:/home/postgres/pgdata/data - - opensearch: - image: opensearchproject/opensearch:2.19.0 - ports: - - "9200:9200" - - "9300:9300" - environment: - discovery.type: single-node - DISABLE_SECURITY_PLUGIN: "true" - OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m" - volumes: - - osdata:/usr/share/opensearch/data - -volumes: - pgdata: - osdata: diff --git a/docker-compose.yml b/docker-compose.yml index 4fa23d89..c5698b23 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -1,20 +1,27 @@ services: - clickhouse: - image: clickhouse/clickhouse-server:25.3 + postgres: + image: timescale/timescaledb-ha:pg16 ports: - - "8123:8123" - - "9000:9000" - volumes: - - clickhouse-data:/var/lib/clickhouse - - ./clickhouse/init:/docker-entrypoint-initdb.d + - "5432:5432" environment: - CLICKHOUSE_USER: cameleer - CLICKHOUSE_PASSWORD: cameleer_dev - CLICKHOUSE_DB: cameleer3 - ulimits: - nofile: - soft: 262144 - hard: 262144 + POSTGRES_DB: cameleer3 + POSTGRES_USER: cameleer + POSTGRES_PASSWORD: cameleer_dev + volumes: + - pgdata:/home/postgres/pgdata/data + + opensearch: + image: opensearchproject/opensearch:2.19.0 + ports: + - "9200:9200" + - "9300:9300" + environment: + discovery.type: single-node + DISABLE_SECURITY_PLUGIN: "true" + OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m" + volumes: + - osdata:/usr/share/opensearch/data volumes: - clickhouse-data: + pgdata: + osdata: