docs(01-03): complete API foundation plan

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:05:02 +01:00
parent 2d3fde3766
commit ff0af0ef2f
4 changed files with 182 additions and 17 deletions
--- a/.planning/phases/01-ingestion-pipeline-api-foundation/01-03-SUMMARY.md
+++ b/.planning/phases/01-ingestion-pipeline-api-foundation/01-03-SUMMARY.md
@@ -0,0 +1,161 @@
+---
+phase: 01-ingestion-pipeline-api-foundation
+plan: 03
+subsystem: api
+tags: [spring-boot, testcontainers, openapi, interceptor, clickhouse, integration-tests]
+
+requires:
+  - phase: 01-01
+    provides: ClickHouse schema, application.yml, dependencies
+
+provides:
+  - AbstractClickHouseIT base class for all integration tests
+  - ProtocolVersionInterceptor enforcing X-Cameleer-Protocol-Version:1 on data/agent paths
+  - WebConfig with interceptor registration and path exclusions
+  - Cameleer3ServerApplication with @EnableScheduling and component scanning
+  - 12 passing integration tests (health, OpenAPI, protocol version, forward compat, TTL)
+
+affects: [01-02, 02-search, 03-agent-registry]
+
+tech-stack:
+  added: [testcontainers 2.0.3, docker-java 3.7.0]
+  patterns: [shared static Testcontainers ClickHouse container, HandlerInterceptor for protocol validation, WebMvcConfigurer for path-based interceptor registration]
+
+key-files:
+  created:
+    - cameleer3-server-app/src/main/java/com/cameleer3/server/app/Cameleer3ServerApplication.java
+    - cameleer3-server-app/src/main/java/com/cameleer3/server/app/interceptor/ProtocolVersionInterceptor.java
+    - cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/WebConfig.java
+    - cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractClickHouseIT.java
+    - cameleer3-server-app/src/test/resources/application-test.yml
+    - cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/HealthControllerIT.java
+    - cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/OpenApiIT.java
+    - cameleer3-server-app/src/test/java/com/cameleer3/server/app/interceptor/ProtocolVersionIT.java
+    - cameleer3-server-app/src/test/java/com/cameleer3/server/app/controller/ForwardCompatIT.java
+  modified:
+    - cameleer3-server-app/pom.xml
+    - pom.xml
+    - clickhouse/init/01-schema.sql
+
+key-decisions:
+  - "Upgraded testcontainers from 1.20.5 to 2.0.3 for Docker Desktop 29.x compatibility (docker-java 3.7.0)"
+  - "Removed junit-jupiter dependency; manual container lifecycle via static initializer instead"
+  - "Changed error_message/error_stacktrace from Nullable(String) to String DEFAULT '' for tokenbf_v1 skip index compatibility"
+  - "TTL expressions use toDateTime() cast for DateTime64 columns in ClickHouse 25.3"
+
+patterns-established:
+  - "AbstractClickHouseIT: static container shared across test classes, @DynamicPropertySource for datasource, @BeforeAll for schema init"
+  - "ProtocolVersionInterceptor: all data/agent endpoints require X-Cameleer-Protocol-Version:1 header"
+  - "Path exclusions: health, api-docs, swagger-ui bypass protocol version check"
+
+requirements-completed: [API-01, API-02, API-03, API-04, API-05, INGST-06]
+
+duration: 10min
+completed: 2026-03-11
+---
+
+# Phase 1 Plan 03: API Foundation Summary
+
+**Protocol version interceptor, health/OpenAPI endpoints, Testcontainers IT base class, and 12 green integration tests covering TTL, forward compat, and interceptor exclusions**
+
+## Performance
+
+- **Duration:** 10 min
+- **Started:** 2026-03-11T10:52:21Z
+- **Completed:** 2026-03-11T11:03:08Z
+- **Tasks:** 2
+- **Files modified:** 12
+
+## Accomplishments
+- ProtocolVersionInterceptor validates X-Cameleer-Protocol-Version:1 on /api/v1/data/** and /api/v1/agents/** paths, returning 400 JSON error for missing or wrong version
+- AbstractClickHouseIT base class with Testcontainers ClickHouse 25.3, shared static container, schema init from 01-schema.sql
+- 12 integration tests: health endpoint (2), OpenAPI docs (2), protocol version enforcement (5), forward compatibility (1), TTL verification (2)
+- Cameleer3ServerApplication with @EnableScheduling, @EnableConfigurationProperties, and dual package scanning
+
+## Task Commits
+
+Each task was committed atomically:
+
+1. **Task 1: Test infrastructure, protocol version interceptor, WebConfig, and app bootstrap** - `b8a4739` (feat)
+2. **Task 2: Integration tests for health, OpenAPI, protocol version, forward compat, and TTL** - `2d3fde3` (test)
+
+## Files Created/Modified
+- `cameleer3-server-app/src/main/java/.../Cameleer3ServerApplication.java` - Spring Boot entry point with scheduling and config properties
+- `cameleer3-server-app/src/main/java/.../interceptor/ProtocolVersionInterceptor.java` - Validates protocol version header on data/agent paths
+- `cameleer3-server-app/src/main/java/.../config/WebConfig.java` - Registers interceptor with path patterns and exclusions
+- `cameleer3-server-app/src/test/java/.../AbstractClickHouseIT.java` - Shared Testcontainers base class for ITs
+- `cameleer3-server-app/src/test/resources/application-test.yml` - Test profile with small buffer config
+- `cameleer3-server-app/src/test/java/.../controller/HealthControllerIT.java` - Health endpoint and TTL tests
+- `cameleer3-server-app/src/test/java/.../controller/OpenApiIT.java` - OpenAPI and Swagger UI tests
+- `cameleer3-server-app/src/test/java/.../interceptor/ProtocolVersionIT.java` - Protocol header enforcement tests
+- `cameleer3-server-app/src/test/java/.../controller/ForwardCompatIT.java` - Unknown JSON fields test
+- `pom.xml` - Override testcontainers.version to 2.0.3
+- `cameleer3-server-app/pom.xml` - Remove junit-jupiter, upgrade testcontainers-clickhouse to 2.0.3
+- `clickhouse/init/01-schema.sql` - Fix TTL expressions and error column types
+
+## Decisions Made
+- Upgraded Testcontainers to 2.0.3 because docker-java 3.4.1 (from TC 1.20.5) is incompatible with Docker Desktop 29.x API 1.52
+- Removed junit-jupiter module (doesn't exist in TC 2.x); manage container lifecycle via static initializer block instead
+- Changed error_message and error_stacktrace from Nullable(String) to String DEFAULT '' because ClickHouse tokenbf_v1 skip indexes require non-nullable String columns
+- Added toDateTime() cast in TTL expressions because ClickHouse 25.3 requires DateTime (not DateTime64) in TTL column references
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Rule 3 - Blocking] Fixed testcontainers junit-jupiter dependency not available in 2.0.x**
+- **Found during:** Task 2 (compilation)
+- **Issue:** org.testcontainers:junit-jupiter:2.0.2 does not exist in Maven Central
+- **Fix:** Removed junit-jupiter dependency, upgraded to TC 2.0.3, managed container lifecycle manually via static initializer
+- **Files modified:** cameleer3-server-app/pom.xml, pom.xml, AbstractClickHouseIT.java
+- **Verification:** All tests compile and pass
+- **Committed in:** 2d3fde3
+
+**2. [Rule 3 - Blocking] Fixed Docker Desktop 29.x incompatibility with Testcontainers**
+- **Found during:** Task 2 (test execution)
+- **Issue:** docker-java 3.4.1 (from TC 1.20.5) sends unversioned API calls to Docker API 1.52, receiving 400 errors
+- **Fix:** Override testcontainers.version to 2.0.3 in parent POM (brings docker-java 3.7.0 with proper API version negotiation)
+- **Files modified:** pom.xml
+- **Verification:** ClickHouse container starts successfully
+- **Committed in:** 2d3fde3
+
+**3. [Rule 1 - Bug] Fixed ClickHouse TTL expression for DateTime64 columns**
+- **Found during:** Task 2 (schema init in tests)
+- **Issue:** ClickHouse 25.3 requires TTL expressions to resolve to DateTime, not DateTime64(3, 'UTC')
+- **Fix:** Changed `TTL start_time + INTERVAL 30 DAY` to `TTL toDateTime(start_time) + toIntervalDay(30)`
+- **Files modified:** clickhouse/init/01-schema.sql
+- **Verification:** Schema creates without errors in ClickHouse 25.3 container
+- **Committed in:** 2d3fde3
+
+**4. [Rule 1 - Bug] Fixed tokenbf_v1 index on Nullable column**
+- **Found during:** Task 2 (schema init in tests)
+- **Issue:** ClickHouse 25.3 does not allow tokenbf_v1 skip indexes on Nullable(String) columns
+- **Fix:** Changed error_message and error_stacktrace from Nullable(String) to String DEFAULT ''
+- **Files modified:** clickhouse/init/01-schema.sql
+- **Verification:** Schema creates without errors, all 12 tests pass
+- **Committed in:** 2d3fde3
+
+---
+
+**Total deviations:** 4 auto-fixed (2 blocking, 2 bugs)
+**Impact on plan:** All fixes necessary for test execution on current Docker Desktop and ClickHouse versions. No scope creep.
+
+## Issues Encountered
+- Surefire runs from module directory, not project root; fixed schema path lookup in AbstractClickHouseIT to check both locations
+
+## User Setup Required
+None - no external service configuration required.
+
+## Next Phase Readiness
+- AbstractClickHouseIT base class ready for all future integration tests
+- Protocol version interceptor active for data/agent endpoints
+- API foundation complete, ready for Plan 02 (REST controllers, ClickHouse repositories, flush scheduler)
+- Health endpoint at /api/v1/health, OpenAPI at /api/v1/api-docs, Swagger UI at /api/v1/swagger-ui
+
+## Self-Check: PASSED
+
+All 9 created files verified present. Both task commits verified in git log.
+
+---
+*Phase: 01-ingestion-pipeline-api-foundation*
+*Completed: 2026-03-11*