fix(01): revise plans based on checker feedback

- Remove AbstractClickHouseIT and application-test.yml from Plan 01-01, move to Plan 01-03 Task 1 (reduces 01-01 from 15 to 13 files) - Change Plan 01-02 Task 1 tdd="true" to tdd="false" (compile-only verify) - Add DiagramRepository and MetricsRepository flush key_links to Plan 01-02 - Update VALIDATION.md: INGST-06 TTL maps to HealthControllerIT#ttlConfigured* instead of non-existent ClickHouseTtlIT Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:40:20 +01:00
parent e808b567cd
commit da09e5fda7
4 changed files with 67 additions and 74 deletions
--- a/.planning/phases/01-ingestion-pipeline-api-foundation/01-03-PLAN.md
+++ b/.planning/phases/01-ingestion-pipeline-api-foundation/01-03-PLAN.md
@@ -8,6 +8,8 @@ files_modified:
  - cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java
  - cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java
  - cameleer3-server-app/src/main/java/com/cameleer3/server/app/Cameleer3ServerApplication.java
+  - cameleer3-server-app/src/test/resources/application-test.yml
+  - cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractClickHouseIT.java
  - cameleer3-server-app/src/test/java/com/cameleer3/server/app/interceptor/ProtocolVersionIT.java
  - cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/HealthControllerIT.java
  - cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/OpenApiIT.java
@@ -38,6 +40,9 @@ must_haves:
    - path: "cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java"
      provides: "Registers interceptor with path patterns"
      min_lines: 10
+    - path: "cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractClickHouseIT.java"
+      provides: "Shared Testcontainers base class for integration tests"
+      min_lines: 20
  key_links:
    - from: "WebConfig.java"
      to: "ProtocolVersionInterceptor.java"
@@ -54,10 +59,10 @@ must_haves:
 ---

 <objective>
-Implement the API foundation: health endpoint, OpenAPI documentation, protocol version header validation, forward compatibility, and TTL verification.
+Implement the API foundation: test infrastructure, health endpoint, OpenAPI documentation, protocol version header validation, forward compatibility, and TTL verification.

-Purpose: Completes the API scaffolding so all endpoints follow the protocol v1 contract. Health endpoint enables monitoring. OpenAPI enables discoverability. Protocol version interceptor enforces compatibility. TTL verification confirms data retention.
-Output: Working health, Swagger UI, protocol header enforcement, and verified TTL retention.
+Purpose: Establishes the Testcontainers base class used by all integration tests across plans. Completes the API scaffolding so all endpoints follow the protocol v1 contract. Health endpoint enables monitoring. OpenAPI enables discoverability. Protocol version interceptor enforces compatibility. TTL verification confirms data retention.
+Output: AbstractClickHouseIT base class, working health, Swagger UI, protocol header enforcement, and verified TTL retention.
 </objective>

 <execution_context>
@@ -75,34 +80,40 @@ Output: Working health, Swagger UI, protocol header enforcement, and verified TT

 <tasks>

-<task type="auto" tdd="true">
-  <name>Task 1: Protocol version interceptor, WebConfig, and Spring Boot application class</name>
+<task type="auto">
+  <name>Task 1: Test infrastructure, protocol version interceptor, WebConfig, and Spring Boot application class</name>
  <files>
+    cameleer3-server-app/src/test/resources/application-test.yml,
+    cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractClickHouseIT.java,
    cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java,
    cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java,
    cameleer3-server-app/src/main/java/com/cameleer3/server/app/Cameleer3ServerApplication.java
  </files>
-  <behavior>
-    - Request to /api/v1/data/* without X-Cameleer-Protocol-Version header returns 400
-    - Request to /api/v1/data/* with X-Cameleer-Protocol-Version:2 returns 400
-    - Request to /api/v1/data/* with X-Cameleer-Protocol-Version:1 passes through
-    - Request to /api/v1/health without the header succeeds (excluded from interceptor)
-    - Request to /api/v1/swagger-ui without the header succeeds (excluded)
-    - Request to /api/v1/api-docs without the header succeeds (excluded)
-  </behavior>
  <action>
-    1. Create ProtocolVersionInterceptor implementing HandlerInterceptor:
+    1. Create application-test.yml for test profile:
+       - Placeholder datasource config (overridden by Testcontainers in AbstractClickHouseIT)
+       - ingestion: small buffer for tests (capacity=100, batch-size=10, flush-interval-ms=100)
+
+    2. Create AbstractClickHouseIT base class:
+       - @Testcontainers + @Container with ClickHouseContainer("clickhouse/clickhouse-server:25.3")
+       - @DynamicPropertySource to override spring.datasource.url/username/password
+       - @SpringBootTest
+       - @ActiveProfiles("test")
+       - @BeforeAll: read clickhouse/init/01-schema.sql and execute it against the container via JDBC
+       - Expose protected JdbcTemplate for subclasses
+
+    3. Create ProtocolVersionInterceptor implementing HandlerInterceptor:
       - preHandle: read X-Cameleer-Protocol-Version header
       - If null or not "1": set response status 400, write JSON error body {"error": "Missing or unsupported X-Cameleer-Protocol-Version header"}, return false
       - If "1": return true

-    2. Create WebConfig implementing WebMvcConfigurer:
+    4. Create WebConfig implementing WebMvcConfigurer:
       - @Configuration
       - Inject ProtocolVersionInterceptor (declare it as @Component or @Bean)
       - Override addInterceptors: register interceptor with pathPatterns "/api/v1/data/**" and "/api/v1/agents/**"
       - Explicitly EXCLUDE: "/api/v1/health", "/api/v1/api-docs/**", "/api/v1/swagger-ui/**", "/api/v1/swagger-ui.html"

-    3. Create or update Cameleer3ServerApplication:
+    5. Create or update Cameleer3ServerApplication:
       - @SpringBootApplication in package com.cameleer3.server.app
       - @EnableScheduling (needed for ClickHouseFlushScheduler from Plan 02)
       - @EnableConfigurationProperties(IngestionConfig.class)
@@ -112,7 +123,7 @@ Output: Working health, Swagger UI, protocol header enforcement, and verified TT
  <verify>
    <automated>mvn clean compile -pl cameleer3-server-app -q 2>&1 | tail -5</automated>
  </verify>
-  <done>ProtocolVersionInterceptor validates header on data/agent paths. Health, swagger, and api-docs paths excluded. Application class enables scheduling and config properties.</done>
+  <done>AbstractClickHouseIT base class ready for integration tests. ProtocolVersionInterceptor validates header on data/agent paths. Health, swagger, and api-docs paths excluded. Application class enables scheduling and config properties.</done>
 </task>

 <task type="auto" tdd="true">
@@ -133,6 +144,7 @@ Output: Working health, Swagger UI, protocol header enforcement, and verified TT
    - GET /api/v1/health without protocol header returns 200 (not intercepted)
    - POST /api/v1/data/executions with extra unknown JSON fields returns 202 (not 400/422)
    - ClickHouse route_executions table SHOW CREATE TABLE includes TTL
+    - ClickHouse agent_metrics table SHOW CREATE TABLE includes TTL
  </behavior>
  <action>
    1. Create HealthControllerIT (extends AbstractClickHouseIT):
@@ -155,18 +167,20 @@ Output: Working health, Swagger UI, protocol header enforcement, and verified TT
       - Test: POST /api/v1/data/executions with valid RouteExecution JSON plus extra unknown fields (e.g., "futureField": "value") -> 202 (Jackson does not fail on unknown properties)
       - This validates API-05 requirement explicitly.

-    5. TTL verification (add to HealthControllerIT or separate test):
-       - Query ClickHouse: SHOW CREATE TABLE route_executions
-       - Assert result contains "TTL start_time + toIntervalDay(30)"
-       - Query: SHOW CREATE TABLE agent_metrics
-       - Assert result contains TTL clause
+    5. TTL verification (add to HealthControllerIT as dedicated test methods):
+       - Test method: ttlConfiguredOnRouteExecutions
+         Query ClickHouse: SHOW CREATE TABLE route_executions
+         Assert result contains "TTL start_time + toIntervalDay(30)"
+       - Test method: ttlConfiguredOnAgentMetrics
+         Query ClickHouse: SHOW CREATE TABLE agent_metrics
+         Assert result contains TTL clause

    Note: All tests that POST to data endpoints must include X-Cameleer-Protocol-Version:1 header.
  </action>
  <verify>
    <automated>mvn test -pl cameleer3-server-app -Dtest="HealthControllerIT,OpenApiIT,ProtocolVersionIT,ForwardCompatIT" -q 2>&1 | tail -15</automated>
  </verify>
-  <done>Health returns 200. OpenAPI docs are available and list endpoints. Protocol version header enforced on data paths, not on health/docs. Unknown JSON fields accepted. TTL confirmed in ClickHouse DDL.</done>
+  <done>Health returns 200. OpenAPI docs are available and list endpoints. Protocol version header enforced on data paths, not on health/docs. Unknown JSON fields accepted. TTL confirmed in ClickHouse DDL via HealthControllerIT test methods.</done>
 </task>

 </tasks>
@@ -180,7 +194,7 @@ Output: Working health, Swagger UI, protocol header enforcement, and verified TT
 </verification>

 <success_criteria>
-All four integration test classes green. Health endpoint accessible. OpenAPI docs list all ingestion endpoints. Protocol version header enforced on data/agent paths but not health/docs. Forward compatibility confirmed. TTL verified in ClickHouse schema.
+All four integration test classes green. AbstractClickHouseIT base class works with Testcontainers. Health endpoint accessible. OpenAPI docs list all ingestion endpoints. Protocol version header enforced on data/agent paths but not health/docs. Forward compatibility confirmed. TTL verified in ClickHouse schema via HealthControllerIT.
 </success_criteria>

 <output>