Commit Graph

14 Commits

Author SHA1 Message Date
hsiegeln
f8505401d7 fix: ClickHouse auth credentials and non-fatal schema init
Some checks failed
CI / cleanup-branch (push) Has been skipped
CI / build (push) Successful in 1m5s
CI / docker (push) Successful in 43s
CI / deploy (push) Has been skipped
CI / deploy-feature (push) Failing after 13s
CI / cleanup-branch (pull_request) Has been skipped
CI / build (pull_request) Successful in 1m47s
CI / docker (pull_request) Has been skipped
CI / deploy (pull_request) Has been skipped
CI / deploy-feature (pull_request) Has been skipped
- Set CLICKHOUSE_USER/PASSWORD via k8s secret (fixes "disabling network
  access for user 'default'" when no password is set)
- Add clickhouse-credentials secret to CI deploy + feature branch copy
- Pass CLICKHOUSE_USERNAME/PASSWORD env vars to server pod
- Make schema initializer non-fatal so server starts even if CH is
  temporarily unavailable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 17:54:44 +02:00
hsiegeln
59dd629b0e fix: create cameleer database on ClickHouse startup
Some checks failed
CI / cleanup-branch (push) Has been skipped
CI / build (pull_request) Successful in 1m49s
CI / cleanup-branch (pull_request) Has been skipped
CI / docker (pull_request) Has been skipped
CI / deploy (pull_request) Has been skipped
CI / deploy-feature (pull_request) Has been skipped
CI / build (push) Successful in 1m7s
CI / docker (push) Successful in 10s
CI / deploy (push) Has been skipped
CI / deploy-feature (push) Has been cancelled
ClickHouse only has the 'default' database out of the box. The JDBC URL
connects to 'cameleer', so the database must exist before the server starts.
Uses /docker-entrypoint-initdb.d/ init script via ConfigMap.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-31 17:31:17 +02:00
hsiegeln
1b991f99a3 deploy: add ClickHouse StatefulSet and server env vars
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-31 17:08:42 +02:00
hsiegeln
ea687a342c deploy: remove obsolete ClickHouse K8s manifest 2026-03-16 19:01:26 +01:00
hsiegeln
4cdf2ac012 Fix ClickHouse OOM on batch insert: reduce batch size, increase memory
All checks were successful
CI / build (push) Successful in 1m3s
CI / docker (push) Successful in 41s
CI / deploy (push) Successful in 46s
Execution rows are wide (29 cols with serialized arrays/JSON), so 500
rows can exceed ClickHouse's memory limit. Reduce default batch size
from 500 to 100 and bump ClickHouse memory limit from 2Gi to 4Gi.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-13 22:44:03 +01:00
hsiegeln
9c2391e5d4 Move ClickHouse credentials to K8s Secret and add health probes
All checks were successful
CI / build (push) Successful in 41s
CI / docker (push) Successful in 13s
CI / deploy (push) Successful in 38s
- ClickHouse user/password now injected via `clickhouse-credentials` Secret
  instead of hardcoded plaintext in deploy manifests (#33)
- CI deploy step creates the secret idempotently from Gitea CI secrets
- Added liveness/readiness probes: server uses /api/v1/health, ClickHouse
  uses /ping (#35)
- Updated HOWTO.md and CLAUDE.md with new secrets and probe details

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 10:59:15 +01:00
hsiegeln
88da1a0dd8 Fix ClickHouse OOM during Proxmox nightly backups
All checks were successful
CI / build (push) Successful in 48s
CI / docker (push) Successful in 1m36s
CI / deploy (push) Successful in 20s
Increase ClickHouse memory limit from 1Gi to 2Gi and reduce default
batch size from 5000 to 500. During VM backup snapshots, I/O contention
prevents ClickHouse from flushing writes fast enough, causing buffer
accumulation that exceeds the 1Gi container limit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-13 09:55:46 +01:00
hsiegeln
48bdb46760 Server fully owns ClickHouse schema — create database + tables on startup
All checks were successful
CI / build (push) Successful in 48s
CI / docker (push) Successful in 39s
CI / deploy (push) Successful in 9s
ClickHouseConfig.ensureDatabaseExists() connects without the database path
to run CREATE DATABASE IF NOT EXISTS before the main DataSource is used.
Removes the ConfigMap-based init scripts from the K8s manifest — the server
is now the single owner of all ClickHouse schema management.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 21:41:35 +01:00
hsiegeln
f7ed91ef9c Use fully qualified cameleer3.* table names in ClickHouse init schema
All checks were successful
CI / build (push) Successful in 47s
CI / docker (push) Successful in 38s
CI / deploy (push) Successful in 10s
Init scripts run against the default database, not CLICKHOUSE_DB.
Prefix all table references with cameleer3.* and add CREATE DATABASE.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 21:38:09 +01:00
hsiegeln
5576b50a3a Add ClickHouse schema init via ConfigMap + docker-entrypoint-initdb.d
All checks were successful
CI / build (push) Successful in 47s
CI / docker (push) Successful in 37s
CI / deploy (push) Successful in 8s
Mounts the schema SQL files as a ConfigMap into ClickHouse's init
directory so tables are created automatically on fresh starts. All
statements use IF NOT EXISTS so they're safe to re-run. This ensures
the schema exists even if the PVC is lost or the pod is recreated.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 21:29:58 +01:00
hsiegeln
a1280609f6 Add NodePort service to expose ClickHouse externally
All checks were successful
CI / build (push) Successful in 1m21s
CI / docker (push) Successful in 38s
CI / deploy (push) Successful in 6s
HTTP on port 30123, native protocol on port 30900. Keeps the existing
headless service for internal StatefulSet DNS.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 21:16:45 +01:00
hsiegeln
9dffa9ea81 Move schema initialization from ClickHouse init scripts to server startup
All checks were successful
CI / build (push) Successful in 49s
CI / docker (push) Successful in 43s
CI / deploy (push) Successful in 15s
Server now applies schema via @PostConstruct using classpath SQL files.
All statements use IF NOT EXISTS/IF NOT EXISTS so it's idempotent and
safe to run on every startup. Removes ConfigMap and init script mount
from K8s manifest since ClickHouse no longer needs to manage the schema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:59:33 +01:00
hsiegeln
129b97183a Use fully qualified table names in ClickHouse init scripts
All checks were successful
CI / build (push) Successful in 49s
CI / docker (push) Successful in 40s
CI / deploy (push) Successful in 13s
ClickHouse Docker entrypoint runs init scripts against the default
database, not the one specified by CLICKHOUSE_DB. Prefix all table
names with cameleer3. to ensure they're created in the right database.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:54:45 +01:00
hsiegeln
c228c3201b Add Docker build, K8s manifests, and CI/CD deploy pipeline
Some checks failed
CI / docker (push) Has been cancelled
CI / deploy (push) Has been cancelled
CI / build (push) Has been cancelled
- Dockerfile: multi-stage build with $BUILDPLATFORM for native Maven
  builds on ARM64 runners, amd64 runtime target. Passes REGISTRY_TOKEN
  build arg for cameleer3-common dependency resolution.
- K8s manifests: ClickHouse StatefulSet with init scripts ConfigMap,
  server Deployment + NodePort (30081)
- CI: docker job (QEMU + buildx cross-compile, registry cache,
  provenance=false, old image cleanup) + deploy job (kubectl)
- .dockerignore for build context optimization

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:01:39 +01:00