From 19c463051aebfdb0229cc13db8ed30ba669f4adf Mon Sep 17 00:00:00 2001 From: hsiegeln <37154749+hsiegeln@users.noreply.github.com> Date: Wed, 15 Apr 2026 20:47:14 +0200 Subject: [PATCH] docs: add design spec for externalizing docker compose templates Co-Authored-By: Claude Opus 4.6 (1M context) --- ...15-externalize-compose-templates-design.md | 164 ++++++++++++++++++ 1 file changed, 164 insertions(+) create mode 100644 docs/superpowers/specs/2026-04-15-externalize-compose-templates-design.md diff --git a/docs/superpowers/specs/2026-04-15-externalize-compose-templates-design.md b/docs/superpowers/specs/2026-04-15-externalize-compose-templates-design.md new file mode 100644 index 0000000..88794c2 --- /dev/null +++ b/docs/superpowers/specs/2026-04-15-externalize-compose-templates-design.md @@ -0,0 +1,164 @@ +# Externalize Docker Compose Templates + +**Date:** 2026-04-15 +**Status:** Approved + +## Problem + +The installer scripts (`install.sh` and `install.ps1`) generate docker-compose YAML inline via heredocs and string interpolation. This causes three problems: + +1. **Maintainability** -- ~450 lines of compose generation are duplicated across two scripts in two languages. Every compose change must be made twice. +2. **Readability** -- The compose structure is buried inside 1800-line scripts and difficult to review or edit. +3. **User customization** -- Users cannot tweak the compose after installation without re-running the installer or editing generated output that gets overwritten on next run. + +## Design + +Replace inline compose generation with static template files. The installer collects user input, writes `.env`, copies templates, and runs `docker compose up -d`. + +### File Layout + +``` +installer/ + templates/ + docker-compose.yml # Infra: traefik, postgres, clickhouse + docker-compose.server.yml # Standalone: server + server-ui + docker-compose.saas.yml # SaaS: logto + cameleer-saas + docker-compose.tls.yml # Overlay: custom cert volume on traefik + docker-compose.monitoring.yml # Overlay: override monitoring network to external + .env.example # Documented variable reference + install.sh + install.ps1 +``` + +### Compose File Responsibilities + +#### `docker-compose.yml` (infra base, always loaded) + +- **Services:** `cameleer-traefik`, `cameleer-postgres`, `cameleer-clickhouse` +- **Prometheus labels** on all services unconditionally (harmless when no scraper connects) +- **Logto console port** always mapped; bind address controlled by `${LOGTO_CONSOLE_BIND:-127.0.0.1}` (exposed = `0.0.0.0`, unexposed = localhost-only) +- **Networks:** `cameleer`, `cameleer-traefik`, `monitoring` (local noop bridge by default) +- **Volumes:** `cameleer-pgdata`, `cameleer-chdata`, `cameleer-certs` +- All services reference the `monitoring` network + +#### `docker-compose.server.yml` (standalone mode) + +- **Services:** `cameleer-server`, `cameleer-server-ui` +- **Additional volumes:** `jars` +- **Additional network:** `cameleer-apps` +- **Monitoring network:** defines local noop bridge, both services reference it + +#### `docker-compose.saas.yml` (SaaS mode) + +- **Services:** `cameleer-logto`, `cameleer-saas` +- **Additional volume:** `cameleer-bootstrapdata` +- **Monitoring network:** defines local noop bridge, both services reference it + +#### `docker-compose.tls.yml` (overlay) + +- Adds `./certs:/user-certs:ro` volume mount to `cameleer-traefik` + +#### `docker-compose.monitoring.yml` (overlay) + +- Overrides the `monitoring` network definition from local noop bridge to `external: true` with `name: ${MONITORING_NETWORK}` +- No per-service entries needed -- services already reference the `monitoring` network in their base files + +### Monitoring Network Pattern + +Each compose file defines the `monitoring` network as a local bridge with a throwaway name: + +```yaml +# In docker-compose.yml, docker-compose.server.yml, docker-compose.saas.yml +networks: + monitoring: + name: cameleer-monitoring-noop +``` + +Services reference this network unconditionally. Without the monitoring overlay, it is an inert local bridge. When `docker-compose.monitoring.yml` is included, Docker Compose merges the network definition, overriding it to external: + +```yaml +# docker-compose.monitoring.yml +networks: + monitoring: + external: true + name: ${MONITORING_NETWORK} +``` + +This avoids conditional YAML injection and keeps a single monitoring overlay file regardless of deployment mode. + +### Logto Console Port Handling + +Instead of conditionally injecting the port mapping, always include it with a bind address variable: + +```yaml +# In docker-compose.yml (traefik ports) +- "${LOGTO_CONSOLE_BIND:-127.0.0.1}:${LOGTO_CONSOLE_PORT:-3002}:3002" +``` + +The installer writes `LOGTO_CONSOLE_BIND=0.0.0.0` when the user chooses to expose the console, and omits it (defaulting to `127.0.0.1`) when not. + +### `.env.example` + +Documented reference of all variables, grouped by section: + +- **Version/images:** `VERSION`, `CAMELEER_SAAS_PROVISIONING_SERVERIMAGE`, `CAMELEER_SAAS_PROVISIONING_SERVERUIIMAGE` +- **Public access:** `PUBLIC_HOST`, `PUBLIC_PROTOCOL` +- **Ports:** `HTTP_PORT`, `HTTPS_PORT`, `LOGTO_CONSOLE_BIND`, `LOGTO_CONSOLE_PORT` +- **PostgreSQL:** `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB` +- **ClickHouse:** `CLICKHOUSE_PASSWORD` +- **Admin credentials (SaaS):** `SAAS_ADMIN_USER`, `SAAS_ADMIN_PASS` +- **Admin credentials (standalone):** `SERVER_ADMIN_USER`, `SERVER_ADMIN_PASS`, `BOOTSTRAP_TOKEN` +- **Docker:** `DOCKER_SOCKET`, `DOCKER_GID` +- **TLS (optional):** `CERT_FILE`, `KEY_FILE`, `CA_FILE` +- **Monitoring (optional):** `MONITORING_NETWORK` +- **Compose file assembly:** `COMPOSE_FILE` +- **TLS validation:** `NODE_TLS_REJECT` + +### `COMPOSE_FILE` Assembly + +The installer builds the `COMPOSE_FILE` value in `.env` based on user choices: + +| Mode | COMPOSE_FILE | +|---|---| +| Standalone | `docker-compose.yml:docker-compose.server.yml` | +| Standalone + TLS | `docker-compose.yml:docker-compose.server.yml:docker-compose.tls.yml` | +| Standalone + monitoring | `docker-compose.yml:docker-compose.server.yml:docker-compose.monitoring.yml` | +| SaaS | `docker-compose.yml:docker-compose.saas.yml` | +| SaaS + TLS | `docker-compose.yml:docker-compose.saas.yml:docker-compose.tls.yml` | +| SaaS + TLS + monitoring | `docker-compose.yml:docker-compose.saas.yml:docker-compose.tls.yml:docker-compose.monitoring.yml` | + +### Installer Script Changes + +**Removed:** + +- `generate_compose_file()` / `Generate-ComposeFile` (~250 lines each, SaaS mode) +- `generate_compose_file_standalone()` / `Generate-ComposeFileStandalone` (~200 lines each, standalone mode) +- All conditional YAML injection logic (logto console, TLS, monitoring, GID) + +**Added:** + +- Copy template files from `templates/` to install directory +- `COMPOSE_FILE` assembly logic in `.env` generation +- `LOGTO_CONSOLE_BIND` variable (replaces conditional port injection) +- `DOCKER_GID` written to `.env` (replaces inline `stat` in heredoc) + +**Unchanged:** + +- User prompting, argument parsing, config file reading +- Password generation, Docker GID detection +- `credentials.txt`, `INSTALL.md`, `cameleer.conf` generation +- `docker compose up -d` invocation + +### Password Fallback Safety + +All password variables in templates use `:?` (fail-if-unset) instead of `:-` (default) to prevent silent fallback to weak credentials: + +```yaml +SAAS_ADMIN_PASS: ${SAAS_ADMIN_PASS:?SAAS_ADMIN_PASS must be set in .env} +``` + +Username defaults (`:-admin`) are acceptable and retained. + +### Migration for Existing Installs + +Existing `installer/cameleer/` has a single generated `docker-compose.yml`. On re-install, the installer copies the template files and regenerates `.env`. The old monolithic compose file is replaced by the multi-file set. `cameleer.conf` preserves user choices, so re-running the installer reproduces the same configuration with the new file structure.