Files
cameleer-server/.planning/phases/01-ingestion-pipeline-api-foundation/01-03-PLAN.md
2026-03-11 11:33:32 +01:00

9.6 KiB

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
phase plan type wave depends_on files_modified autonomous requirements must_haves
01-ingestion-pipeline-api-foundation 03 execute 2
01-01
cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java
cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java
cameleer3-server-app/src/main/java/com/cameleer3/server/app/Cameleer3ServerApplication.java
cameleer3-server-app/src/test/java/com/cameleer3/server/app/interceptor/ProtocolVersionIT.java
cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/HealthControllerIT.java
cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/OpenApiIT.java
cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/ForwardCompatIT.java
true
API-01
API-02
API-03
API-04
API-05
INGST-06
truths artifacts key_links
GET /api/v1/health returns 200 with health status
GET /api/v1/swagger-ui returns Swagger UI page
GET /api/v1/api-docs returns OpenAPI JSON spec listing all endpoints
Requests without X-Cameleer-Protocol-Version header to /api/v1/data/* return 400
Requests with wrong protocol version return 400
Health and swagger endpoints work without protocol version header
Unknown JSON fields in request body do not cause deserialization errors
ClickHouse tables have TTL clause for 30-day retention
path provides min_lines
cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java Validates X-Cameleer-Protocol-Version:1 header on data endpoints 20
path provides min_lines
cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java Registers interceptor with path patterns 10
from to via pattern
WebConfig.java ProtocolVersionInterceptor.java addInterceptors registration addInterceptors.*ProtocolVersionInterceptor
from to via pattern
application.yml Actuator health endpoint management.endpoints config management.endpoints
from to via pattern
application.yml springdoc springdoc.api-docs.path and swagger-ui.path springdoc
Implement the API foundation: health endpoint, OpenAPI documentation, protocol version header validation, forward compatibility, and TTL verification.

Purpose: Completes the API scaffolding so all endpoints follow the protocol v1 contract. Health endpoint enables monitoring. OpenAPI enables discoverability. Protocol version interceptor enforces compatibility. TTL verification confirms data retention. Output: Working health, Swagger UI, protocol header enforcement, and verified TTL retention.

<execution_context> @C:/Users/Hendrik/.claude/get-shit-done/workflows/execute-plan.md @C:/Users/Hendrik/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/01-ingestion-pipeline-api-foundation/01-RESEARCH.md @.planning/phases/01-ingestion-pipeline-api-foundation/01-01-SUMMARY.md Task 1: Protocol version interceptor, WebConfig, and Spring Boot application class cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java, cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java, cameleer3-server-app/src/main/java/com/cameleer3/server/app/Cameleer3ServerApplication.java - Request to /api/v1/data/* without X-Cameleer-Protocol-Version header returns 400 - Request to /api/v1/data/* with X-Cameleer-Protocol-Version:2 returns 400 - Request to /api/v1/data/* with X-Cameleer-Protocol-Version:1 passes through - Request to /api/v1/health without the header succeeds (excluded from interceptor) - Request to /api/v1/swagger-ui without the header succeeds (excluded) - Request to /api/v1/api-docs without the header succeeds (excluded) 1. Create ProtocolVersionInterceptor implementing HandlerInterceptor: - preHandle: read X-Cameleer-Protocol-Version header - If null or not "1": set response status 400, write JSON error body {"error": "Missing or unsupported X-Cameleer-Protocol-Version header"}, return false - If "1": return true
2. Create WebConfig implementing WebMvcConfigurer:
   - @Configuration
   - Inject ProtocolVersionInterceptor (declare it as @Component or @Bean)
   - Override addInterceptors: register interceptor with pathPatterns "/api/v1/data/**" and "/api/v1/agents/**"
   - Explicitly EXCLUDE: "/api/v1/health", "/api/v1/api-docs/**", "/api/v1/swagger-ui/**", "/api/v1/swagger-ui.html"

3. Create or update Cameleer3ServerApplication:
   - @SpringBootApplication in package com.cameleer3.server.app
   - @EnableScheduling (needed for ClickHouseFlushScheduler from Plan 02)
   - @EnableConfigurationProperties(IngestionConfig.class)
   - Main method with SpringApplication.run()
   - Ensure package scanning covers com.cameleer3.server.app and com.cameleer3.server.core
mvn clean compile -pl cameleer3-server-app -q 2>&1 | tail -5 ProtocolVersionInterceptor validates header on data/agent paths. Health, swagger, and api-docs paths excluded. Application class enables scheduling and config properties. Task 2: Health, OpenAPI, protocol version, forward compat, and TTL integration tests cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/HealthControllerIT.java, cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/OpenApiIT.java, cameleer3-server-app/src/test/java/com/cameleer3/server/app/interceptor/ProtocolVersionIT.java, cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/ForwardCompatIT.java - GET /api/v1/health returns 200 with JSON containing status field - GET /api/v1/api-docs returns 200 with OpenAPI JSON containing paths - GET /api/v1/swagger-ui/index.html returns 200 (Swagger UI page) - POST /api/v1/data/executions without protocol header returns 400 - POST /api/v1/data/executions with X-Cameleer-Protocol-Version:1 does NOT return 400 (may return 202 or other based on body) - POST /api/v1/data/executions with X-Cameleer-Protocol-Version:2 returns 400 - GET /api/v1/health without protocol header returns 200 (not intercepted) - POST /api/v1/data/executions with extra unknown JSON fields returns 202 (not 400/422) - ClickHouse route_executions table SHOW CREATE TABLE includes TTL 1. Create HealthControllerIT (extends AbstractClickHouseIT): - Test: GET /api/v1/health -> 200 with body containing "status" - Test: No X-Cameleer-Protocol-Version header required for health
2. Create OpenApiIT (extends AbstractClickHouseIT):
   - Test: GET /api/v1/api-docs -> 200, body contains "openapi" and "paths"
   - Test: GET /api/v1/swagger-ui/index.html -> 200 or 302 (redirect to UI)
   - Test: api-docs lists /api/v1/data/executions path

3. Create ProtocolVersionIT (extends AbstractClickHouseIT):
   - Test: POST /api/v1/data/executions without header -> 400 with error message
   - Test: POST /api/v1/data/executions with header value "2" -> 400
   - Test: POST /api/v1/data/executions with header value "1" and valid body -> 202 (not 400)
   - Test: GET /api/v1/health without header -> 200 (excluded from interceptor)
   - Test: GET /api/v1/api-docs without header -> 200 (excluded from interceptor)

4. Create ForwardCompatIT (extends AbstractClickHouseIT):
   - Test: POST /api/v1/data/executions with valid RouteExecution JSON plus extra unknown fields (e.g., "futureField": "value") -> 202 (Jackson does not fail on unknown properties)
   - This validates API-05 requirement explicitly.

5. TTL verification (add to HealthControllerIT or separate test):
   - Query ClickHouse: SHOW CREATE TABLE route_executions
   - Assert result contains "TTL start_time + toIntervalDay(30)"
   - Query: SHOW CREATE TABLE agent_metrics
   - Assert result contains TTL clause

Note: All tests that POST to data endpoints must include X-Cameleer-Protocol-Version:1 header.
mvn test -pl cameleer3-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q 2>&1 | tail -15 Health returns 200. OpenAPI docs are available and list endpoints. Protocol version header enforced on data paths, not on health/docs. Unknown JSON fields accepted. TTL confirmed in ClickHouse DDL. - `mvn test -pl cameleer3-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q` -- all tests pass - GET /api/v1/health returns 200 - GET /api/v1/api-docs returns OpenAPI spec - Missing protocol header returns 400 on data endpoints - Health works without protocol header

<success_criteria> All four integration test classes green. Health endpoint accessible. OpenAPI docs list all ingestion endpoints. Protocol version header enforced on data/agent paths but not health/docs. Forward compatibility confirmed. TTL verified in ClickHouse schema. </success_criteria>

After completion, create `.planning/phases/01-ingestion-pipeline-api-foundation/01-03-SUMMARY.md`