diff --git a/CLAUDE.md b/CLAUDE.md
index 28513ac8..e3a8fddd 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -38,10 +38,10 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
- Jackson `JavaTimeModule` for `Instant` deserialization
- Communication: receives HTTP POST data from agents, serves SSE event streams for config push/commands
- Maintains agent instance registry with states: LIVE → STALE → DEAD
-- Storage: ClickHouse for structured data, text index for full-text search
+- Storage: PostgreSQL (TimescaleDB) for structured data, OpenSearch for full-text search
- Security: JWT auth with RBAC (AGENT/VIEWER/OPERATOR/ADMIN roles), Ed25519 config signing, bootstrap token for registration
- OIDC: Optional external identity provider support (token exchange pattern). Configured via `CAMELEER_OIDC_*` env vars
-- User persistence: ClickHouse `users` table, admin CRUD at `/api/v1/admin/users`
+- User persistence: PostgreSQL `users` table, admin CRUD at `/api/v1/admin/users`
## CI/CD & Deployment
@@ -50,8 +50,8 @@ java -jar cameleer3-server-app/target/cameleer3-server-app-1.0-SNAPSHOT.jar
- Docker: multi-stage build (`Dockerfile`), `$BUILDPLATFORM` for native Maven on ARM64 runner, amd64 runtime
- `REGISTRY_TOKEN` build arg required for `cameleer3-common` dependency resolution
- Registry: `gitea.siegeln.net/cameleer/cameleer3-server` (container images)
-- K8s manifests in `deploy/` — ClickHouse StatefulSet + server Deployment + NodePort Service (30081)
+- K8s manifests in `deploy/` — PostgreSQL + OpenSearch StatefulSets, server Deployment + NodePort Service (30081)
- Deployment target: k3s at 192.168.50.86, namespace `cameleer`
-- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `clickhouse-credentials`, `CAMELEER_JWT_SECRET`
-- K8s probes: server uses `/api/v1/health`, ClickHouse uses `/ping`
+- Secrets managed in CI deploy step (idempotent `--dry-run=client | kubectl apply`): `cameleer-auth`, `postgres-credentials`, `opensearch-credentials`, `CAMELEER_JWT_SECRET`
+- K8s probes: server uses `/api/v1/health`, PostgreSQL uses `pg_isready`, OpenSearch uses `/_cluster/health`
- Docker build uses buildx registry cache + `--provenance=false` for Gitea compatibility
diff --git a/HOWTO.md b/HOWTO.md
index a1fd9dec..54c52557 100644
--- a/HOWTO.md
+++ b/HOWTO.md
@@ -21,20 +21,20 @@ mvn clean verify # compile + run all tests (needs Docker for integrati
## Infrastructure Setup
-Start ClickHouse:
+Start PostgreSQL and OpenSearch:
```bash
docker compose up -d
```
-This starts ClickHouse 25.3 and automatically runs the schema init scripts (`clickhouse/init/01-schema.sql`, `clickhouse/init/02-search-columns.sql`, `clickhouse/init/03-users.sql`).
+This starts TimescaleDB (PostgreSQL 16) and OpenSearch 2.19. The database schema is applied automatically via Flyway migrations on server startup.
-| Service | Port | Purpose |
-|------------|------|------------------|
-| ClickHouse | 8123 | HTTP API (JDBC) |
-| ClickHouse | 9000 | Native protocol |
+| Service | Port | Purpose |
+|------------|------|----------------------|
+| PostgreSQL | 5432 | JDBC (Spring JDBC) |
+| OpenSearch | 9200 | REST API (full-text) |
-ClickHouse credentials: `cameleer` / `cameleer_dev`, database `cameleer3`.
+PostgreSQL credentials: `cameleer` / `cameleer_dev`, database `cameleer3`.
## Run the Server
@@ -109,7 +109,7 @@ The env-var local user gets `ADMIN` role. Agents get `AGENT` role at registratio
### OIDC Login (Optional)
-OIDC configuration is stored in ClickHouse and managed via the admin API or UI. The SPA checks if OIDC is available:
+OIDC configuration is stored in PostgreSQL and managed via the admin API or UI. The SPA checks if OIDC is available:
```bash
# 1. SPA checks if OIDC is available (returns 404 if not configured)
@@ -340,9 +340,8 @@ Key settings in `cameleer3-server-app/src/main/resources/application.yml`:
|---------|---------|-------------|
| `server.port` | 8081 | Server port |
| `ingestion.buffer-capacity` | 50000 | Max items in write buffer |
-| `ingestion.batch-size` | 5000 | Items per ClickHouse batch insert |
+| `ingestion.batch-size` | 5000 | Items per batch insert |
| `ingestion.flush-interval-ms` | 1000 | Buffer flush interval (ms) |
-| `ingestion.data-ttl-days` | 30 | ClickHouse TTL for auto-deletion |
| `agent-registry.heartbeat-interval-seconds` | 30 | Expected heartbeat interval |
| `agent-registry.stale-threshold-seconds` | 90 | Time before agent marked STALE |
| `agent-registry.dead-threshold-seconds` | 300 | Time after STALE before DEAD |
@@ -386,7 +385,7 @@ npm run generate-api # Requires backend running on :8081
## Running Tests
-Integration tests use Testcontainers (starts ClickHouse automatically — requires Docker):
+Integration tests use Testcontainers (starts PostgreSQL and OpenSearch automatically — requires Docker):
```bash
# All tests
@@ -399,14 +398,13 @@ mvn test -pl cameleer3-server-core
mvn test -pl cameleer3-server-app -Dtest=ExecutionControllerIT
```
-## Verify ClickHouse Data
+## Verify Database Data
After posting data and waiting for the flush interval (1s default):
```bash
-docker exec -it cameleer3-server-clickhouse-1 clickhouse-client \
- --user cameleer --password cameleer_dev -d cameleer3 \
- -q "SELECT count() FROM route_executions"
+docker exec -it cameleer3-server-postgres-1 psql -U cameleer -d cameleer3 \
+ -c "SELECT count(*) FROM route_executions"
```
## Kubernetes Deployment
@@ -417,7 +415,8 @@ The full stack is deployed to k3s via CI/CD on push to `main`. K8s manifests are
```
cameleer namespace:
- ClickHouse (StatefulSet, 2Gi PVC) ← clickhouse:8123 (ClusterIP)
+ PostgreSQL (StatefulSet, 10Gi PVC) ← postgres:5432 (ClusterIP)
+ OpenSearch (StatefulSet, 10Gi PVC) ← opensearch:9200 (ClusterIP)
cameleer3-server (Deployment) ← NodePort 30081
cameleer3-ui (Deployment, Nginx) ← NodePort 30090
Authentik Server (Deployment) ← NodePort 30950
@@ -439,7 +438,7 @@ cameleer namespace:
Push to `main` triggers: **build** (UI npm + Maven, unit tests) → **docker** (buildx amd64 for server + UI, push to Gitea registry) → **deploy** (kubectl apply + rolling update).
-Required Gitea org secrets: `REGISTRY_TOKEN`, `KUBECONFIG_BASE64`, `CAMELEER_AUTH_TOKEN`, `CAMELEER_JWT_SECRET`, `CLICKHOUSE_USER`, `CLICKHOUSE_PASSWORD`, `CAMELEER_UI_USER` (optional), `CAMELEER_UI_PASSWORD` (optional), `AUTHENTIK_PG_PASSWORD`, `AUTHENTIK_SECRET_KEY`, `CAMELEER_OIDC_ENABLED`, `CAMELEER_OIDC_ISSUER`, `CAMELEER_OIDC_CLIENT_ID`, `CAMELEER_OIDC_CLIENT_SECRET`.
+Required Gitea org secrets: `REGISTRY_TOKEN`, `KUBECONFIG_BASE64`, `CAMELEER_AUTH_TOKEN`, `CAMELEER_JWT_SECRET`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`, `OPENSEARCH_USER`, `OPENSEARCH_PASSWORD`, `CAMELEER_UI_USER` (optional), `CAMELEER_UI_PASSWORD` (optional), `AUTHENTIK_PG_USER`, `AUTHENTIK_PG_PASSWORD`, `AUTHENTIK_SECRET_KEY`, `CAMELEER_OIDC_ENABLED`, `CAMELEER_OIDC_ISSUER`, `CAMELEER_OIDC_CLIENT_ID`, `CAMELEER_OIDC_CLIENT_SECRET`.
### Manual K8s Commands
@@ -450,8 +449,11 @@ kubectl -n cameleer get pods
# View server logs
kubectl -n cameleer logs -f deploy/cameleer3-server
-# View ClickHouse logs
-kubectl -n cameleer logs -f statefulset/clickhouse
+# View PostgreSQL logs
+kubectl -n cameleer logs -f statefulset/postgres
+
+# View OpenSearch logs
+kubectl -n cameleer logs -f statefulset/opensearch
# Restart server
kubectl -n cameleer rollout restart deployment/cameleer3-server
diff --git a/cameleer3-server-app/pom.xml b/cameleer3-server-app/pom.xml
index 81e21c59..4738822d 100644
--- a/cameleer3-server-app/pom.xml
+++ b/cameleer3-server-app/pom.xml
@@ -174,7 +174,7 @@
maven-failsafe-plugin
1
- false
+ true
diff --git a/docker-compose.dev.yml b/docker-compose.dev.yml
deleted file mode 100644
index 1e6ecb58..00000000
--- a/docker-compose.dev.yml
+++ /dev/null
@@ -1,29 +0,0 @@
-version: '3.8'
-
-services:
- postgres:
- image: timescale/timescaledb-ha:pg16
- ports:
- - "5432:5432"
- environment:
- POSTGRES_DB: cameleer3
- POSTGRES_USER: cameleer
- POSTGRES_PASSWORD: cameleer_dev
- volumes:
- - pgdata:/home/postgres/pgdata/data
-
- opensearch:
- image: opensearchproject/opensearch:2.19.0
- ports:
- - "9200:9200"
- - "9300:9300"
- environment:
- discovery.type: single-node
- DISABLE_SECURITY_PLUGIN: "true"
- OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
- volumes:
- - osdata:/usr/share/opensearch/data
-
-volumes:
- pgdata:
- osdata:
diff --git a/docker-compose.yml b/docker-compose.yml
index 4fa23d89..c5698b23 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1,20 +1,27 @@
services:
- clickhouse:
- image: clickhouse/clickhouse-server:25.3
+ postgres:
+ image: timescale/timescaledb-ha:pg16
ports:
- - "8123:8123"
- - "9000:9000"
- volumes:
- - clickhouse-data:/var/lib/clickhouse
- - ./clickhouse/init:/docker-entrypoint-initdb.d
+ - "5432:5432"
environment:
- CLICKHOUSE_USER: cameleer
- CLICKHOUSE_PASSWORD: cameleer_dev
- CLICKHOUSE_DB: cameleer3
- ulimits:
- nofile:
- soft: 262144
- hard: 262144
+ POSTGRES_DB: cameleer3
+ POSTGRES_USER: cameleer
+ POSTGRES_PASSWORD: cameleer_dev
+ volumes:
+ - pgdata:/home/postgres/pgdata/data
+
+ opensearch:
+ image: opensearchproject/opensearch:2.19.0
+ ports:
+ - "9200:9200"
+ - "9300:9300"
+ environment:
+ discovery.type: single-node
+ DISABLE_SECURITY_PLUGIN: "true"
+ OPENSEARCH_JAVA_OPTS: "-Xms512m -Xmx512m"
+ volumes:
+ - osdata:/usr/share/opensearch/data
volumes:
- clickhouse-data:
+ pgdata:
+ osdata: