# Phase 4: Observability Pipeline + Inbound Routing — Implementation Plan > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. **Goal:** Complete the deploy → hit endpoint → see traces loop. Serve the existing cameleer-server dashboard, add agent connectivity verification, enable optional inbound HTTP routing for customer apps, and wire up observability data health checks. **Architecture:** Wiring phase — cameleer-server already has full observability. Phase 4 adds Traefik routing for the dashboard + customer app endpoints, new API endpoints in cameleer-saas for agent-status and observability-status, and configures `CAMELEER_TENANT_ID` on the server. **Tech Stack:** Spring Boot 3.4.3, docker-java 3.4.1, ClickHouse JDBC, Traefik v3 labels, Spring RestClient --- ## File Structure ### New Files - `src/main/java/net/siegeln/cameleer/saas/observability/AgentStatusService.java` — Queries cameleer-server for agent registration - `src/main/java/net/siegeln/cameleer/saas/observability/AgentStatusController.java` — Agent status + observability status endpoints - `src/main/java/net/siegeln/cameleer/saas/observability/dto/AgentStatusResponse.java` — Response DTO - `src/main/java/net/siegeln/cameleer/saas/observability/dto/ObservabilityStatusResponse.java` — Response DTO - `src/main/java/net/siegeln/cameleer/saas/observability/dto/UpdateRoutingRequest.java` — Request DTO for PATCH routing - `src/main/java/net/siegeln/cameleer/saas/observability/ConnectivityHealthCheck.java` — Startup connectivity verification - `src/test/java/net/siegeln/cameleer/saas/observability/AgentStatusServiceTest.java` — Unit tests - `src/test/java/net/siegeln/cameleer/saas/observability/AgentStatusControllerTest.java` — Integration tests - `src/main/resources/db/migration/V010__add_exposed_port_to_apps.sql` — Migration ### Modified Files - `src/main/java/net/siegeln/cameleer/saas/runtime/StartContainerRequest.java` — Add `labels` field - `src/main/java/net/siegeln/cameleer/saas/runtime/DockerRuntimeOrchestrator.java` — Apply labels on container create - `src/main/java/net/siegeln/cameleer/saas/runtime/RuntimeConfig.java` — Add `domain` property - `src/main/java/net/siegeln/cameleer/saas/app/AppEntity.java` — Add `exposedPort` field - `src/main/java/net/siegeln/cameleer/saas/app/AppService.java` — Add `updateRouting` method - `src/main/java/net/siegeln/cameleer/saas/app/AppController.java` — Add PATCH routing endpoint - `src/main/java/net/siegeln/cameleer/saas/app/dto/AppResponse.java` — Add `exposedPort` + `routeUrl` fields - `src/main/java/net/siegeln/cameleer/saas/deployment/DeploymentService.java` — Build labels for Traefik routing - `src/main/resources/application.yml` — Add `domain` property - `docker-compose.yml` — Add dashboard Traefik route, `CAMELEER_TENANT_ID` - `.env.example` — Add `CAMELEER_TENANT_SLUG` - `HOWTO.md` — Update with observability + routing docs --- ## Task 1: Database Migration + Entity Changes **Files:** - Create: `src/main/resources/db/migration/V010__add_exposed_port_to_apps.sql` - Modify: `src/main/java/net/siegeln/cameleer/saas/app/AppEntity.java` - [ ] **Step 1: Create migration V010** ```sql ALTER TABLE apps ADD COLUMN exposed_port INTEGER; ``` - [ ] **Step 2: Add exposedPort field to AppEntity** Add after `previousDeploymentId` field: ```java @Column(name = "exposed_port") private Integer exposedPort; ``` Add getter and setter: ```java public Integer getExposedPort() { return exposedPort; } public void setExposedPort(Integer exposedPort) { this.exposedPort = exposedPort; } ``` - [ ] **Step 3: Verify compilation** Run: `mvn compile -B -q` - [ ] **Step 4: Commit** ```bash git add src/main/resources/db/migration/V010__add_exposed_port_to_apps.sql \ src/main/java/net/siegeln/cameleer/saas/app/AppEntity.java git commit -m "feat: add exposed_port column to apps table" ``` --- ## Task 2: StartContainerRequest Labels + DockerRuntimeOrchestrator **Files:** - Modify: `src/main/java/net/siegeln/cameleer/saas/runtime/StartContainerRequest.java` - Modify: `src/main/java/net/siegeln/cameleer/saas/runtime/DockerRuntimeOrchestrator.java` - [ ] **Step 1: Add labels field to StartContainerRequest** Replace the current record with: ```java package net.siegeln.cameleer.saas.runtime; import java.util.Map; public record StartContainerRequest( String imageRef, String containerName, String network, Map envVars, long memoryLimitBytes, int cpuShares, int healthCheckPort, Map labels ) {} ``` - [ ] **Step 2: Apply labels in DockerRuntimeOrchestrator.startContainer** In the `startContainer` method, after `.withHostConfig(hostConfig)` and before `.withHealthcheck(...)`, add: ```java .withLabels(request.labels() != null ? request.labels() : Map.of()) ``` - [ ] **Step 3: Fix all existing callers of StartContainerRequest** The `DeploymentService.executeDeploymentAsync` method creates a `StartContainerRequest`. Add `Map.of()` as the labels argument (empty labels for now — routing labels come in Task 5): Find the existing `new StartContainerRequest(...)` call and add `Map.of()` as the last argument. - [ ] **Step 4: Verify compilation and run unit tests** Run: `mvn test -B -Dsurefire.excludes="**/*ControllerTest.java,**/AuditRepositoryTest.java,**/CameleerSaasApplicationTest.java" -q` - [ ] **Step 5: Commit** ```bash git add src/main/java/net/siegeln/cameleer/saas/runtime/StartContainerRequest.java \ src/main/java/net/siegeln/cameleer/saas/runtime/DockerRuntimeOrchestrator.java \ src/main/java/net/siegeln/cameleer/saas/deployment/DeploymentService.java git commit -m "feat: add labels support to StartContainerRequest and DockerRuntimeOrchestrator" ``` --- ## Task 3: RuntimeConfig Domain + AppResponse + AppService Routing **Files:** - Modify: `src/main/java/net/siegeln/cameleer/saas/runtime/RuntimeConfig.java` - Modify: `src/main/java/net/siegeln/cameleer/saas/app/dto/AppResponse.java` - Modify: `src/main/java/net/siegeln/cameleer/saas/app/AppService.java` - Modify: `src/main/java/net/siegeln/cameleer/saas/app/AppController.java` - Create: `src/main/java/net/siegeln/cameleer/saas/observability/dto/UpdateRoutingRequest.java` - Modify: `src/main/resources/application.yml` - [ ] **Step 1: Add domain property to RuntimeConfig** Add field and getter: ```java @Value("${cameleer.runtime.domain:localhost}") private String domain; public String getDomain() { return domain; } ``` - [ ] **Step 2: Add domain to application.yml** In the `cameleer.runtime` section, add: ```yaml domain: ${DOMAIN:localhost} ``` - [ ] **Step 3: Update AppResponse to include exposedPort and routeUrl** Replace the record: ```java package net.siegeln.cameleer.saas.app.dto; import java.time.Instant; import java.util.UUID; public record AppResponse( UUID id, UUID environmentId, String slug, String displayName, String jarOriginalFilename, Long jarSizeBytes, String jarChecksum, Integer exposedPort, String routeUrl, UUID currentDeploymentId, UUID previousDeploymentId, Instant createdAt, Instant updatedAt ) {} ``` - [ ] **Step 4: Create UpdateRoutingRequest** ```java package net.siegeln.cameleer.saas.observability.dto; public record UpdateRoutingRequest( Integer exposedPort ) {} ``` - [ ] **Step 5: Add updateRouting method to AppService** ```java public AppEntity updateRouting(UUID appId, Integer exposedPort, UUID actorId) { var app = appRepository.findById(appId) .orElseThrow(() -> new IllegalArgumentException("App not found")); app.setExposedPort(exposedPort); return appRepository.save(app); } ``` - [ ] **Step 6: Update AppController — add PATCH routing endpoint and update toResponse** Add the endpoint: ```java @PatchMapping("/{appId}/routing") public ResponseEntity updateRouting( @PathVariable UUID environmentId, @PathVariable UUID appId, @RequestBody UpdateRoutingRequest request, Authentication authentication) { try { var actorId = resolveActorId(authentication); var app = appService.updateRouting(appId, request.exposedPort(), actorId); var env = environmentService.getById(app.getEnvironmentId()).orElse(null); return ResponseEntity.ok(toResponse(app, env)); } catch (IllegalArgumentException e) { return ResponseEntity.notFound().build(); } } ``` This requires adding `EnvironmentService` and `RuntimeConfig` as constructor dependencies to `AppController`. Update the constructor. Update `toResponse` to accept the environment and compute the route URL: ```java private AppResponse toResponse(AppEntity app, EnvironmentEntity env) { String routeUrl = null; if (app.getExposedPort() != null && env != null) { var tenant = tenantRepository.findById(env.getTenantId()).orElse(null); if (tenant != null) { routeUrl = "http://" + app.getSlug() + "." + env.getSlug() + "." + tenant.getSlug() + "." + runtimeConfig.getDomain(); } } return new AppResponse( app.getId(), app.getEnvironmentId(), app.getSlug(), app.getDisplayName(), app.getJarOriginalFilename(), app.getJarSizeBytes(), app.getJarChecksum(), app.getExposedPort(), routeUrl, app.getCurrentDeploymentId(), app.getPreviousDeploymentId(), app.getCreatedAt(), app.getUpdatedAt()); } ``` This requires adding `TenantRepository` as a constructor dependency too. Update the existing `toResponse(AppEntity)` calls in other methods to pass the environment — look up the environment from the `environmentId` path variable or from `environmentService`. For the list/get/create endpoints that already have `environmentId` in the path, look up the environment once and pass it. - [ ] **Step 7: Verify compilation** Run: `mvn compile -B -q` - [ ] **Step 8: Commit** ```bash git add src/main/java/net/siegeln/cameleer/saas/runtime/RuntimeConfig.java \ src/main/java/net/siegeln/cameleer/saas/app/dto/AppResponse.java \ src/main/java/net/siegeln/cameleer/saas/app/AppService.java \ src/main/java/net/siegeln/cameleer/saas/app/AppController.java \ src/main/java/net/siegeln/cameleer/saas/observability/dto/UpdateRoutingRequest.java \ src/main/resources/application.yml git commit -m "feat: add exposed port routing and route URL to app API" ``` --- ## Task 4: Agent Status + Observability Status Endpoints (TDD) **Files:** - Create: `src/main/java/net/siegeln/cameleer/saas/observability/dto/AgentStatusResponse.java` - Create: `src/main/java/net/siegeln/cameleer/saas/observability/dto/ObservabilityStatusResponse.java` - Create: `src/main/java/net/siegeln/cameleer/saas/observability/AgentStatusService.java` - Create: `src/main/java/net/siegeln/cameleer/saas/observability/AgentStatusController.java` - Create: `src/test/java/net/siegeln/cameleer/saas/observability/AgentStatusServiceTest.java` - [ ] **Step 1: Create DTOs** `AgentStatusResponse.java`: ```java package net.siegeln.cameleer.saas.observability.dto; import java.time.Instant; import java.util.List; public record AgentStatusResponse( boolean registered, String state, Instant lastHeartbeat, List routeIds, String applicationId, String environmentId ) {} ``` `ObservabilityStatusResponse.java`: ```java package net.siegeln.cameleer.saas.observability.dto; import java.time.Instant; public record ObservabilityStatusResponse( boolean hasTraces, boolean hasMetrics, boolean hasDiagrams, Instant lastTraceAt, long traceCount24h ) {} ``` - [ ] **Step 2: Write failing tests** ```java package net.siegeln.cameleer.saas.observability; import net.siegeln.cameleer.saas.app.AppEntity; import net.siegeln.cameleer.saas.app.AppRepository; import net.siegeln.cameleer.saas.environment.EnvironmentEntity; import net.siegeln.cameleer.saas.environment.EnvironmentRepository; import net.siegeln.cameleer.saas.runtime.RuntimeConfig; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.extension.ExtendWith; import org.mockito.Mock; import org.mockito.junit.jupiter.MockitoExtension; import java.util.Optional; import java.util.UUID; import static org.junit.jupiter.api.Assertions.*; import static org.mockito.Mockito.*; @ExtendWith(MockitoExtension.class) class AgentStatusServiceTest { @Mock private AppRepository appRepository; @Mock private EnvironmentRepository environmentRepository; @Mock private RuntimeConfig runtimeConfig; private AgentStatusService agentStatusService; @BeforeEach void setUp() { when(runtimeConfig.getCameleerServerEndpoint()).thenReturn("http://cameleer-server:8081"); agentStatusService = new AgentStatusService(appRepository, environmentRepository, runtimeConfig); } @Test void getAgentStatus_appNotFound_shouldThrow() { when(appRepository.findById(any())).thenReturn(Optional.empty()); assertThrows(IllegalArgumentException.class, () -> agentStatusService.getAgentStatus(UUID.randomUUID())); } @Test void getAgentStatus_shouldReturnUnknownWhenServerUnreachable() { var appId = UUID.randomUUID(); var envId = UUID.randomUUID(); var app = new AppEntity(); app.setId(appId); app.setEnvironmentId(envId); app.setSlug("my-app"); when(appRepository.findById(appId)).thenReturn(Optional.of(app)); var env = new EnvironmentEntity(); env.setId(envId); env.setSlug("default"); when(environmentRepository.findById(envId)).thenReturn(Optional.of(env)); var result = agentStatusService.getAgentStatus(appId); assertNotNull(result); assertFalse(result.registered()); assertEquals("UNKNOWN", result.state()); } } ``` - [ ] **Step 3: Implement AgentStatusService** ```java package net.siegeln.cameleer.saas.observability; import net.siegeln.cameleer.saas.app.AppRepository; import net.siegeln.cameleer.saas.environment.EnvironmentRepository; import net.siegeln.cameleer.saas.observability.dto.AgentStatusResponse; import net.siegeln.cameleer.saas.observability.dto.ObservabilityStatusResponse; import net.siegeln.cameleer.saas.runtime.RuntimeConfig; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.stereotype.Service; import org.springframework.web.client.RestClient; import javax.sql.DataSource; import java.sql.Timestamp; import java.time.Instant; import java.util.List; import java.util.UUID; @Service public class AgentStatusService { private static final Logger log = LoggerFactory.getLogger(AgentStatusService.class); private final AppRepository appRepository; private final EnvironmentRepository environmentRepository; private final RuntimeConfig runtimeConfig; private final RestClient restClient; @Autowired(required = false) @Qualifier("clickHouseDataSource") private DataSource clickHouseDataSource; public AgentStatusService(AppRepository appRepository, EnvironmentRepository environmentRepository, RuntimeConfig runtimeConfig) { this.appRepository = appRepository; this.environmentRepository = environmentRepository; this.runtimeConfig = runtimeConfig; this.restClient = RestClient.builder() .baseUrl(runtimeConfig.getCameleerServerEndpoint()) .build(); } public AgentStatusResponse getAgentStatus(UUID appId) { var app = appRepository.findById(appId) .orElseThrow(() -> new IllegalArgumentException("App not found")); var env = environmentRepository.findById(app.getEnvironmentId()) .orElseThrow(() -> new IllegalStateException("Environment not found")); try { var response = restClient.get() .uri("/api/v1/agents") .header("Authorization", "Bearer " + runtimeConfig.getBootstrapToken()) .retrieve() .body(List.class); if (response != null) { for (var agentObj : response) { if (agentObj instanceof java.util.Map agent) { var agentAppId = String.valueOf(agent.get("applicationId")); var agentEnvId = String.valueOf(agent.get("environmentId")); if (app.getSlug().equals(agentAppId) && env.getSlug().equals(agentEnvId)) { var state = String.valueOf(agent.getOrDefault("state", "UNKNOWN")); var routeIds = agent.get("routeIds"); @SuppressWarnings("unchecked") var routes = routeIds instanceof List r ? (List) r : List.of(); return new AgentStatusResponse(true, state, null, routes, agentAppId, agentEnvId); } } } } return new AgentStatusResponse(false, "NOT_REGISTERED", null, List.of(), app.getSlug(), env.getSlug()); } catch (Exception e) { log.warn("Failed to query agent status from cameleer-server: {}", e.getMessage()); return new AgentStatusResponse(false, "UNKNOWN", null, List.of(), app.getSlug(), env.getSlug()); } } public ObservabilityStatusResponse getObservabilityStatus(UUID appId) { var app = appRepository.findById(appId) .orElseThrow(() -> new IllegalArgumentException("App not found")); var env = environmentRepository.findById(app.getEnvironmentId()) .orElseThrow(() -> new IllegalStateException("Environment not found")); if (clickHouseDataSource == null) { return new ObservabilityStatusResponse(false, false, false, null, 0); } try (var conn = clickHouseDataSource.getConnection(); var ps = conn.prepareStatement(""" SELECT count() as trace_count, max(start_time) as last_trace FROM executions WHERE application_id = ? AND environment = ? AND start_time > now() - INTERVAL 24 HOUR """)) { ps.setString(1, app.getSlug()); ps.setString(2, env.getSlug()); try (var rs = ps.executeQuery()) { if (rs.next()) { var count = rs.getLong("trace_count"); var lastTrace = rs.getTimestamp("last_trace"); return new ObservabilityStatusResponse( count > 0, false, false, lastTrace != null ? lastTrace.toInstant() : null, count); } } } catch (Exception e) { log.warn("Failed to query observability status from ClickHouse: {}", e.getMessage()); } return new ObservabilityStatusResponse(false, false, false, null, 0); } } ``` - [ ] **Step 4: Create AgentStatusController** ```java package net.siegeln.cameleer.saas.observability; import net.siegeln.cameleer.saas.observability.dto.AgentStatusResponse; import net.siegeln.cameleer.saas.observability.dto.ObservabilityStatusResponse; import org.springframework.http.ResponseEntity; import org.springframework.web.bind.annotation.*; import java.util.UUID; @RestController @RequestMapping("/api/apps/{appId}") public class AgentStatusController { private final AgentStatusService agentStatusService; public AgentStatusController(AgentStatusService agentStatusService) { this.agentStatusService = agentStatusService; } @GetMapping("/agent-status") public ResponseEntity getAgentStatus(@PathVariable UUID appId) { try { var status = agentStatusService.getAgentStatus(appId); return ResponseEntity.ok(status); } catch (IllegalArgumentException e) { return ResponseEntity.notFound().build(); } } @GetMapping("/observability-status") public ResponseEntity getObservabilityStatus(@PathVariable UUID appId) { try { var status = agentStatusService.getObservabilityStatus(appId); return ResponseEntity.ok(status); } catch (IllegalArgumentException e) { return ResponseEntity.notFound().build(); } } } ``` - [ ] **Step 5: Run tests** Run: `mvn test -pl . -Dtest=AgentStatusServiceTest -B` Expected: 2 tests PASS - [ ] **Step 6: Commit** ```bash git add src/main/java/net/siegeln/cameleer/saas/observability/ \ src/test/java/net/siegeln/cameleer/saas/observability/ git commit -m "feat: add agent status and observability status endpoints" ``` --- ## Task 5: Traefik Routing Labels in DeploymentService **Files:** - Modify: `src/main/java/net/siegeln/cameleer/saas/deployment/DeploymentService.java` - [ ] **Step 1: Build Traefik labels when app has exposedPort** In `executeDeploymentAsync`, after building the `envVars` map and before creating `startRequest`, add label computation: ```java // Build Traefik labels for inbound routing var labels = new java.util.HashMap(); if (app.getExposedPort() != null) { labels.put("traefik.enable", "true"); labels.put("traefik.http.routers." + containerName + ".rule", "Host(`" + app.getSlug() + "." + env.getSlug() + "." + tenant.getSlug() + "." + runtimeConfig.getDomain() + "`)"); labels.put("traefik.http.services." + containerName + ".loadbalancer.server.port", String.valueOf(app.getExposedPort())); } ``` Then pass `labels` to the `StartContainerRequest` constructor (replacing the `Map.of()` added in Task 2). Note: The `tenant` variable is already looked up earlier in the method for container naming. - [ ] **Step 2: Run unit tests** Run: `mvn test -pl . -Dtest=DeploymentServiceTest -B` Expected: All tests PASS - [ ] **Step 3: Commit** ```bash git add src/main/java/net/siegeln/cameleer/saas/deployment/DeploymentService.java git commit -m "feat: add Traefik routing labels for customer apps with exposed ports" ``` --- ## Task 6: Connectivity Health Check **Files:** - Create: `src/main/java/net/siegeln/cameleer/saas/observability/ConnectivityHealthCheck.java` - [ ] **Step 1: Create startup connectivity check** ```java package net.siegeln.cameleer.saas.observability; import net.siegeln.cameleer.saas.runtime.RuntimeConfig; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.boot.context.event.ApplicationReadyEvent; import org.springframework.context.event.EventListener; import org.springframework.stereotype.Component; import org.springframework.web.client.RestClient; @Component public class ConnectivityHealthCheck { private static final Logger log = LoggerFactory.getLogger(ConnectivityHealthCheck.class); private final RuntimeConfig runtimeConfig; public ConnectivityHealthCheck(RuntimeConfig runtimeConfig) { this.runtimeConfig = runtimeConfig; } @EventListener(ApplicationReadyEvent.class) public void verifyConnectivity() { checkCameleerServer(); } private void checkCameleerServer() { try { var client = RestClient.builder() .baseUrl(runtimeConfig.getCameleerServerEndpoint()) .build(); var response = client.get() .uri("/actuator/health") .retrieve() .toBodilessEntity(); if (response.getStatusCode().is2xxSuccessful()) { log.info("cameleer-server connectivity: OK ({})", runtimeConfig.getCameleerServerEndpoint()); } else { log.warn("cameleer-server connectivity: HTTP {} ({})", response.getStatusCode(), runtimeConfig.getCameleerServerEndpoint()); } } catch (Exception e) { log.warn("cameleer-server connectivity: FAILED ({}) - {}", runtimeConfig.getCameleerServerEndpoint(), e.getMessage()); } } } ``` - [ ] **Step 2: Verify compilation** Run: `mvn compile -B -q` - [ ] **Step 3: Commit** ```bash git add src/main/java/net/siegeln/cameleer/saas/observability/ConnectivityHealthCheck.java git commit -m "feat: add cameleer-server startup connectivity check" ``` --- ## Task 7: Docker Compose + .env + CI Updates **Files:** - Modify: `docker-compose.yml` - Modify: `.env.example` - Modify: `.gitea/workflows/ci.yml` - [ ] **Step 1: Update docker-compose.yml — add dashboard route and CAMELEER_TENANT_ID** In the `cameleer-server` service: Add to environment section: ```yaml CAMELEER_TENANT_ID: ${CAMELEER_TENANT_SLUG:-default} ``` Add new Traefik labels (after existing ones): ```yaml - traefik.http.routers.dashboard.rule=PathPrefix(`/dashboard`) - traefik.http.routers.dashboard.middlewares=forward-auth,dashboard-strip - traefik.http.middlewares.dashboard-strip.stripprefix.prefixes=/dashboard - traefik.http.services.dashboard.loadbalancer.server.port=8080 ``` - [ ] **Step 2: Update .env.example** Add: ``` CAMELEER_TENANT_SLUG=default ``` - [ ] **Step 3: Update CI excludes** In `.gitea/workflows/ci.yml`, add `**/AgentStatusControllerTest.java` to the Surefire excludes (if integration test exists). - [ ] **Step 4: Run all unit tests** Run: `mvn test -B -Dsurefire.excludes="**/*ControllerTest.java,**/AuditRepositoryTest.java,**/CameleerSaasApplicationTest.java" -q` - [ ] **Step 5: Commit** ```bash git add docker-compose.yml .env.example .gitea/workflows/ci.yml git commit -m "feat: add dashboard Traefik route and CAMELEER_TENANT_ID config" ``` --- ## Task 8: Update HOWTO.md **Files:** - Modify: `HOWTO.md` - [ ] **Step 1: Add observability and routing sections** After the "Deploy a Camel Application" section, add: **Observability Dashboard section** — explains how to access the dashboard at `/dashboard`, what data is visible. **Inbound HTTP Routing section** — explains how to set `exposedPort` on an app and what URL to use. **Agent Status section** — explains the agent-status and observability-status endpoints. Update the API Reference table with the new endpoints: - `GET /api/apps/{aid}/agent-status` - `GET /api/apps/{aid}/observability-status` - `PATCH /api/environments/{eid}/apps/{aid}/routing` Update the .env table to include `CAMELEER_TENANT_SLUG`. - [ ] **Step 2: Commit** ```bash git add HOWTO.md git commit -m "docs: update HOWTO with observability dashboard, routing, and agent status" ``` --- ## Summary of Spec Coverage | Spec Requirement | Task | |---|---| | Serve cameleer-server dashboard via Traefik | Task 7 (dashboard Traefik labels) | | CAMELEER_TENANT_ID configuration | Task 7 (docker-compose env) | | Agent connectivity verification endpoint | Task 4 (AgentStatusService + Controller) | | Observability data health endpoint | Task 4 (ObservabilityStatusResponse) | | Inbound HTTP routing (exposedPort + Traefik labels) | Tasks 1, 2, 3, 5 | | StartContainerRequest labels support | Task 2 | | AppResponse with routeUrl | Task 3 | | PATCH routing API | Task 3 | | Startup connectivity check | Task 6 | | Docker Compose changes | Task 7 | | .env.example updates | Task 7 | | HOWTO.md updates | Task 8 | | V010 migration | Task 1 |