cameleer-saas/HOWTO.md

# Cameleer SaaS -- How to Install, Start & Bootstrap

## Quick Start (Development)

```bash
# 1. Clone
git clone https://gitea.siegeln.net/cameleer/cameleer-saas.git
cd cameleer-saas

# 2. Create environment file
cp .env.example .env

# 3. Generate Ed25519 key pair
mkdir -p keys
ssh-keygen -t ed25519 -f keys/ed25519 -N ""
mv keys/ed25519 keys/ed25519.key

# 4. Start the stack
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d

# 5. Wait for services to be ready (~30s)
docker compose logs -f cameleer-saas --since 10s
# Look for: "Started CameleerSaasApplication"

# 6. Verify
curl http://localhost:8080/actuator/health
# {"status":"UP"}
```

## Prerequisites

- Docker Desktop (Windows/Mac) or Docker Engine 24+ (Linux)
- Git
- `curl` or any HTTP client (for testing)

## Architecture

The platform runs as a Docker Compose stack:

| Service | Image | Port | Purpose |
|---------|-------|------|---------|
| **traefik-certs** | alpine:latest | — | Init container: generates self-signed cert or copies user-supplied cert |
| **traefik** | traefik:v3 | 80, 443, 3002 | Reverse proxy, TLS termination, routing |
| **postgres** | postgres:16-alpine | 5432* | Platform database + Logto database |
| **logto** | ghcr.io/logto-io/logto | 3001*, 3002* | Identity provider (OIDC) |
| **cameleer-saas** | cameleer-saas:latest | 8080* | SaaS API server + vendor UI |
| **clickhouse** | clickhouse-server:latest | 8123* | Trace/metrics/log storage |

*Ports exposed to host only with `docker-compose.dev.yml` overlay.

Per-tenant `cameleer-server` and `cameleer-server-ui` containers are provisioned dynamically by `DockerTenantProvisioner` — they are NOT part of the compose stack.

## Installation

### 1. Environment Configuration

```bash
cp .env.example .env
```

Edit `.env` and set at minimum:

```bash
# Change in production
POSTGRES_PASSWORD=<strong-password>

# Logto M2M credentials (auto-provisioned by bootstrap, or get from Logto admin console)
CAMELEER_SAAS_IDENTITY_M2MCLIENTID=
CAMELEER_SAAS_IDENTITY_M2MCLIENTSECRET=
```

### 2. Ed25519 Keys

The platform uses Ed25519 keys for license signing and machine token verification.

```bash
mkdir -p keys
ssh-keygen -t ed25519 -f keys/ed25519 -N ""
mv keys/ed25519 keys/ed25519.key
```

This creates `keys/ed25519.key` (private) and `keys/ed25519.pub` (public). The keys directory is mounted read-only into the cameleer-saas container.

If no key files are configured, the platform generates ephemeral keys on startup (suitable for development only -- keys change on every restart).

### 3. TLS Certificate (Optional)

By default, the `traefik-certs` init container generates a self-signed certificate for `PUBLIC_HOST`. To supply your own certificate at bootstrap time, set these env vars in `.env`:

```bash
CERT_FILE=/path/to/cert.pem    # PEM-encoded certificate
KEY_FILE=/path/to/key.pem      # PEM-encoded private key
CA_FILE=/path/to/ca.pem        # Optional: CA bundle (for private CA trust)
```

The init container validates that the key matches the certificate before accepting. If validation fails, the container exits with an error.

**Runtime certificate replacement** is available via the vendor UI at `/vendor/certificates`:
- Upload a new cert+key+CA bundle (staged, not yet active)
- Validate and activate (atomic swap, Traefik hot-reloads)
- Roll back to the previous certificate if needed
- Track which tenants need a restart to pick up CA bundle changes

### 4. Start the Stack

**Development** (ports exposed for direct access):
```bash
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d
```

**Production** (traffic routed through Traefik only):
```bash
docker compose up -d
```

### 5. Verify Services

```bash
# Health check
curl http://localhost:8080/actuator/health

# Check all containers are running
docker compose ps
```

## Bootstrapping

### First-Time Logto Setup

On first boot, Logto seeds its database automatically. Access the admin console to configure it:

1. Open http://localhost:3002 (Logto admin console)
2. Complete the initial setup wizard
3. Create a **Machine-to-Machine** application:
   - Go to Applications > Create Application > Machine-to-Machine
   - Note the **App ID** and **App Secret**
   - Assign the **Logto Management API** resource with all scopes
4. Update `.env`:
   ```
   CAMELEER_SAAS_IDENTITY_M2MCLIENTID=<app-id>
   CAMELEER_SAAS_IDENTITY_M2MCLIENTSECRET=<app-secret>
   ```
5. Restart cameleer-saas: `docker compose restart cameleer-saas`

### Create Your First Tenant

With a Logto user token (obtained via OIDC login flow):

```bash
TOKEN="<your-logto-jwt>"

# Create tenant
curl -X POST http://localhost:8080/api/tenants \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"name": "My Company", "slug": "my-company", "tier": "MID"}'

# A "default" environment is auto-created with the tenant
```

### Generate a License

```bash
TENANT_ID="<uuid-from-above>"

curl -X POST "http://localhost:8080/api/tenants/$TENANT_ID/license" \
  -H "Authorization: Bearer $TOKEN"
```

### Deploy a Camel Application

```bash
# List environments
curl "http://localhost:8080/api/tenants/$TENANT_ID/environments" \
  -H "Authorization: Bearer $TOKEN"

ENV_ID="<default-environment-uuid>"

# Upload JAR and create app
curl -X POST "http://localhost:8080/api/environments/$ENV_ID/apps" \
  -H "Authorization: Bearer $TOKEN" \
  -F 'metadata={"slug":"order-service","displayName":"Order Service"};type=application/json' \
  -F "file=@/path/to/your-camel-app.jar"

APP_ID="<app-uuid-from-response>"

# Deploy (async -- returns 202 with deployment ID)
curl -X POST "http://localhost:8080/api/apps/$APP_ID/deploy" \
  -H "Authorization: Bearer $TOKEN"

DEPLOYMENT_ID="<deployment-uuid>"

# Poll deployment status
curl "http://localhost:8080/api/apps/$APP_ID/deployments/$DEPLOYMENT_ID" \
  -H "Authorization: Bearer $TOKEN"
# Status transitions: BUILDING -> STARTING -> RUNNING (or FAILED)

# View container logs
curl "http://localhost:8080/api/apps/$APP_ID/logs?limit=50" \
  -H "Authorization: Bearer $TOKEN"

# Stop the app
curl -X POST "http://localhost:8080/api/apps/$APP_ID/stop" \
  -H "Authorization: Bearer $TOKEN"
```

### Enable Inbound HTTP Routing

If your Camel app exposes a REST endpoint, you can make it reachable from outside the stack:

```bash
# Set the port your app listens on (e.g., 8080 for Spring Boot)
curl -X PATCH "http://localhost:8080/api/environments/$ENV_ID/apps/$APP_ID/routing" \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"exposedPort": 8080}'
```

Your app is now reachable at `http://{app-slug}.{env-slug}.{tenant-slug}.{domain}` (e.g., `http://order-service.default.my-company.localhost`). Traefik routes traffic automatically.

To disable routing, set `exposedPort` to `null`.

### View the Observability Dashboard

The cameleer-server React SPA dashboard is available at:

```
http://localhost/dashboard
```

This shows execution traces, route topology graphs, metrics, and logs for all deployed apps. Authentication is required (Logto OIDC token via forward-auth).

### Check Agent & Observability Status

```bash
# Is the agent registered with cameleer-server?
curl "http://localhost:8080/api/apps/$APP_ID/agent-status" \
  -H "Authorization: Bearer $TOKEN"
# Returns: registered, state (ACTIVE/STALE/DEAD/UNKNOWN), routeIds

# Is the app producing observability data?
curl "http://localhost:8080/api/apps/$APP_ID/observability-status" \
  -H "Authorization: Bearer $TOKEN"
# Returns: hasTraces, lastTraceAt, traceCount24h
```

## API Reference

### Tenants
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/tenants` | Create tenant |
| GET | `/api/tenants/{id}` | Get tenant |
| GET | `/api/tenants/by-slug/{slug}` | Get tenant by slug |

### Licensing
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/tenants/{tid}/license` | Generate license |
| GET | `/api/tenants/{tid}/license` | Get active license |

### Environments
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/tenants/{tid}/environments` | Create environment |
| GET | `/api/tenants/{tid}/environments` | List environments |
| GET | `/api/tenants/{tid}/environments/{eid}` | Get environment |
| PATCH | `/api/tenants/{tid}/environments/{eid}` | Rename environment |
| DELETE | `/api/tenants/{tid}/environments/{eid}` | Delete environment |

### Apps
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/environments/{eid}/apps` | Create app + upload JAR |
| GET | `/api/environments/{eid}/apps` | List apps |
| GET | `/api/environments/{eid}/apps/{aid}` | Get app |
| PUT | `/api/environments/{eid}/apps/{aid}/jar` | Re-upload JAR |
| PATCH | `/api/environments/{eid}/apps/{aid}/routing` | Set/clear exposed port |
| DELETE | `/api/environments/{eid}/apps/{aid}` | Delete app |

### Deployments
| Method | Path | Description |
|--------|------|-------------|
| POST | `/api/apps/{aid}/deploy` | Deploy app (async, 202) |
| GET | `/api/apps/{aid}/deployments` | Deployment history |
| GET | `/api/apps/{aid}/deployments/{did}` | Get deployment status |
| POST | `/api/apps/{aid}/stop` | Stop current deployment |
| POST | `/api/apps/{aid}/restart` | Restart app |

### Logs
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/apps/{aid}/logs` | Query container logs |

Query params: `since`, `until` (ISO timestamps), `limit` (default 500), `stream` (stdout/stderr/both)

### Observability
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/apps/{aid}/agent-status` | Agent registration status |
| GET | `/api/apps/{aid}/observability-status` | Trace/metrics data health |

### Dashboard
| Path | Description |
|------|-------------|
| `/dashboard` | cameleer-server observability dashboard (forward-auth protected) |

### Vendor: Certificates (platform:admin)
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/vendor/certificates` | Overview (active, staged, archived, stale count) |
| POST | `/api/vendor/certificates/stage` | Upload cert+key+CA (multipart) |
| POST | `/api/vendor/certificates/activate` | Promote staged -> active |
| POST | `/api/vendor/certificates/restore` | Swap archived <-> active |
| DELETE | `/api/vendor/certificates/staged` | Discard staged cert |
| GET | `/api/vendor/certificates/stale-tenants` | Count tenants needing CA restart |

### Vendor: Tenants (platform:admin)
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/vendor/tenants` | List all tenants (includes fleet health: agentCount, environmentCount, agentLimit) |
| POST | `/api/vendor/tenants` | Create tenant (async provisioning) |
| GET | `/api/vendor/tenants/{id}` | Tenant detail + server state |
| POST | `/api/vendor/tenants/{id}/restart` | Restart server containers |
| POST | `/api/vendor/tenants/{id}/suspend` | Suspend tenant |
| POST | `/api/vendor/tenants/{id}/activate` | Activate tenant |
| DELETE | `/api/vendor/tenants/{id}` | Delete tenant |
| POST | `/api/vendor/tenants/{id}/license` | Renew license |

### Tenant Portal (org-scoped)
| Method | Path | Description |
|--------|------|-------------|
| GET | `/api/tenant/dashboard` | Tenant dashboard data |
| GET | `/api/tenant/license` | License details |
| POST | `/api/tenant/server/restart` | Restart server |
| GET | `/api/tenant/team` | List team members |
| POST | `/api/tenant/team/invite` | Invite team member |
| DELETE | `/api/tenant/team/{userId}` | Remove team member |
| GET | `/api/tenant/settings` | Tenant settings |
| GET | `/api/tenant/sso` | List SSO connectors |
| POST | `/api/tenant/sso` | Create SSO connector |
| GET | `/api/tenant/ca` | List tenant CA certificates |
| POST | `/api/tenant/ca` | Upload CA cert (staged) |
| POST | `/api/tenant/ca/{id}/activate` | Activate staged CA cert |
| DELETE | `/api/tenant/ca/{id}` | Remove CA cert |
| GET | `/api/tenant/audit` | Tenant audit log |

### Health
| Method | Path | Description |
|--------|------|-------------|
| GET | `/actuator/health` | Health check (public) |
| GET | `/api/health/secured` | Authenticated health check |

## Tier Limits

| Tier | Environments | Apps | Retention | Features |
|------|-------------|------|-----------|----------|
| LOW | 1 | 3 | 7 days | Topology |
| MID | 2 | 10 | 30 days | + Lineage, Correlation |
| HIGH | Unlimited | 50 | 90 days | + Debugger, Replay |
| BUSINESS | Unlimited | Unlimited | 365 days | All features |

## Frontend Development

The SaaS management UI is a React SPA in the `ui/` directory.

### Setup

```bash
cd ui
npm install
```

### Dev Server

```bash
cd ui
npm run dev
```

The Vite dev server starts on http://localhost:5173 and proxies `/api` to `http://localhost:8080` (the Spring Boot backend). Run the backend in another terminal with `mvn spring-boot:run` or via Docker Compose.

### Environment Variables

| Variable | Purpose | Default |
|----------|---------|---------|
| `VITE_LOGTO_ENDPOINT` | Logto OIDC endpoint | `http://localhost:3001` |
| `VITE_LOGTO_CLIENT_ID` | Logto application client ID | (empty) |

Create a `ui/.env.local` file for local overrides:
```bash
VITE_LOGTO_ENDPOINT=http://localhost:3001
VITE_LOGTO_CLIENT_ID=your-client-id
```

### Production Build

```bash
cd ui
npm run build
```

Output goes to `src/main/resources/static/` (configured in `vite.config.ts`). The subsequent `mvn package` bundles the SPA into the JAR. In Docker builds, the Dockerfile handles this automatically via a multi-stage build.

### SPA Routing

Spring Boot serves `index.html` for all non-API routes via `SpaController.java`. React Router handles client-side routing. The SPA lives at `/`, while the observability dashboard (cameleer-server) is at `/dashboard`.

## Development

### Running Tests

```bash
# Unit tests only (no Docker required)
mvn test -B -Dsurefire.excludes="**/*ControllerTest.java,**/AuditRepositoryTest.java,**/CameleerSaasApplicationTest.java"

# Integration tests (requires Docker Desktop)
mvn test -B -Dtest="EnvironmentControllerTest,AppControllerTest,DeploymentControllerTest"

# All tests
mvn verify -B
```

### Building Locally

```bash
# Build JAR
mvn clean package -DskipTests -B

# Build Docker image
docker build -t cameleer-saas:local .

# Use local image
VERSION=local docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d
```

## Troubleshooting

**Logto fails to start**: Check that PostgreSQL is healthy first. Logto needs the `logto` database created by `docker/init-databases.sh`. Run `docker compose logs logto` for details.

**cameleer-saas won't start**: Check `docker compose logs cameleer-saas`. Common issues:
- PostgreSQL not ready (wait for healthcheck)
- Flyway migration conflict (check for manual schema changes)

**Ephemeral key warnings**: `No Ed25519 key files configured -- generating ephemeral keys (dev mode)` is normal in development. For production, generate keys as described above.

**Container deployment fails**: Check that Docker socket is mounted (`/var/run/docker.sock`) and the `cameleer-runtime-base` image is available. Pull it with: `docker pull gitea.siegeln.net/cameleer/cameleer-runtime-base:latest`