cameleer-server/.planning/phases/01-ingestion-pipeline-api-foundation/01-03-SUMMARY.md

---
phase: 01-ingestion-pipeline-api-foundation
plan: 03
subsystem: api
tags: [spring-boot, testcontainers, openapi, interceptor, clickhouse, integration-tests]

requires:
  - phase: 01-01
    provides: ClickHouse schema, application.yml, dependencies

provides:
  - AbstractClickHouseIT base class for all integration tests
  - ProtocolVersionInterceptor enforcing X-Cameleer-Protocol-Version:1 on data/agent paths
  - WebConfig with interceptor registration and path exclusions
  - CameleerServerApplication with @EnableScheduling and component scanning
  - 12 passing integration tests (health, OpenAPI, protocol version, forward compat, TTL)

affects: [01-02, 02-search, 03-agent-registry]

tech-stack:
  added: [testcontainers 2.0.3, docker-java 3.7.0]
  patterns: [shared static Testcontainers ClickHouse container, HandlerInterceptor for protocol validation, WebMvcConfigurer for path-based interceptor registration]

key-files:
  created:
    - cameleer-server-app/src/main/java/com/cameleer/server/app/CameleerServerApplication.java
    - cameleer-server-app/src/main/java/com/cameleer/server/app/interceptor/ProtocolVersionInterceptor.java
    - cameleer-server-app/src/main/java/com/cameleer/server/app/config/WebConfig.java
    - cameleer-server-app/src/test/java/com/cameleer/server/app/AbstractClickHouseIT.java
    - cameleer-server-app/src/test/resources/application-test.yml
    - cameleer-server-app/src/test/java/com/cameleer/server/app/controller/HealthControllerIT.java
    - cameleer-server-app/src/test/java/com/cameleer/server/app/controller/OpenApiIT.java
    - cameleer-server-app/src/test/java/com/cameleer/server/app/interceptor/ProtocolVersionIT.java
    - cameleer-server-app/src/test/java/com/cameleer/server/app/controller/ForwardCompatIT.java
  modified:
    - cameleer-server-app/pom.xml
    - pom.xml
    - clickhouse/init/01-schema.sql

key-decisions:
  - "Upgraded testcontainers from 1.20.5 to 2.0.3 for Docker Desktop 29.x compatibility (docker-java 3.7.0)"
  - "Removed junit-jupiter dependency; manual container lifecycle via static initializer instead"
  - "Changed error_message/error_stacktrace from Nullable(String) to String DEFAULT '' for tokenbf_v1 skip index compatibility"
  - "TTL expressions use toDateTime() cast for DateTime64 columns in ClickHouse 25.3"

patterns-established:
  - "AbstractClickHouseIT: static container shared across test classes, @DynamicPropertySource for datasource, @BeforeAll for schema init"
  - "ProtocolVersionInterceptor: all data/agent endpoints require X-Cameleer-Protocol-Version:1 header"
  - "Path exclusions: health, api-docs, swagger-ui bypass protocol version check"

requirements-completed: [API-01, API-02, API-03, API-04, API-05, INGST-06]

duration: 10min
completed: 2026-03-11
---

# Phase 1 Plan 03: API Foundation Summary

**Protocol version interceptor, health/OpenAPI endpoints, Testcontainers IT base class, and 12 green integration tests covering TTL, forward compat, and interceptor exclusions**

## Performance

- **Duration:** 10 min
- **Started:** 2026-03-11T10:52:21Z
- **Completed:** 2026-03-11T11:03:08Z
- **Tasks:** 2
- **Files modified:** 12

## Accomplishments
- ProtocolVersionInterceptor validates X-Cameleer-Protocol-Version:1 on /api/v1/data/** and /api/v1/agents/** paths, returning 400 JSON error for missing or wrong version
- AbstractClickHouseIT base class with Testcontainers ClickHouse 25.3, shared static container, schema init from 01-schema.sql
- 12 integration tests: health endpoint (2), OpenAPI docs (2), protocol version enforcement (5), forward compatibility (1), TTL verification (2)
- CameleerServerApplication with @EnableScheduling, @EnableConfigurationProperties, and dual package scanning

## Task Commits

Each task was committed atomically:

1. **Task 1: Test infrastructure, protocol version interceptor, WebConfig, and app bootstrap** - `b8a4739` (feat)
2. **Task 2: Integration tests for health, OpenAPI, protocol version, forward compat, and TTL** - `2d3fde3` (test)

## Files Created/Modified
- `cameleer-server-app/src/main/java/.../CameleerServerApplication.java` - Spring Boot entry point with scheduling and config properties
- `cameleer-server-app/src/main/java/.../interceptor/ProtocolVersionInterceptor.java` - Validates protocol version header on data/agent paths
- `cameleer-server-app/src/main/java/.../config/WebConfig.java` - Registers interceptor with path patterns and exclusions
- `cameleer-server-app/src/test/java/.../AbstractClickHouseIT.java` - Shared Testcontainers base class for ITs
- `cameleer-server-app/src/test/resources/application-test.yml` - Test profile with small buffer config
- `cameleer-server-app/src/test/java/.../controller/HealthControllerIT.java` - Health endpoint and TTL tests
- `cameleer-server-app/src/test/java/.../controller/OpenApiIT.java` - OpenAPI and Swagger UI tests
- `cameleer-server-app/src/test/java/.../interceptor/ProtocolVersionIT.java` - Protocol header enforcement tests
- `cameleer-server-app/src/test/java/.../controller/ForwardCompatIT.java` - Unknown JSON fields test
- `pom.xml` - Override testcontainers.version to 2.0.3
- `cameleer-server-app/pom.xml` - Remove junit-jupiter, upgrade testcontainers-clickhouse to 2.0.3
- `clickhouse/init/01-schema.sql` - Fix TTL expressions and error column types

## Decisions Made
- Upgraded Testcontainers to 2.0.3 because docker-java 3.4.1 (from TC 1.20.5) is incompatible with Docker Desktop 29.x API 1.52
- Removed junit-jupiter module (doesn't exist in TC 2.x); manage container lifecycle via static initializer block instead
- Changed error_message and error_stacktrace from Nullable(String) to String DEFAULT '' because ClickHouse tokenbf_v1 skip indexes require non-nullable String columns
- Added toDateTime() cast in TTL expressions because ClickHouse 25.3 requires DateTime (not DateTime64) in TTL column references

## Deviations from Plan

### Auto-fixed Issues

**1. [Rule 3 - Blocking] Fixed testcontainers junit-jupiter dependency not available in 2.0.x**
- **Found during:** Task 2 (compilation)
- **Issue:** org.testcontainers:junit-jupiter:2.0.2 does not exist in Maven Central
- **Fix:** Removed junit-jupiter dependency, upgraded to TC 2.0.3, managed container lifecycle manually via static initializer
- **Files modified:** cameleer-server-app/pom.xml, pom.xml, AbstractClickHouseIT.java
- **Verification:** All tests compile and pass
- **Committed in:** 2d3fde3

**2. [Rule 3 - Blocking] Fixed Docker Desktop 29.x incompatibility with Testcontainers**
- **Found during:** Task 2 (test execution)
- **Issue:** docker-java 3.4.1 (from TC 1.20.5) sends unversioned API calls to Docker API 1.52, receiving 400 errors
- **Fix:** Override testcontainers.version to 2.0.3 in parent POM (brings docker-java 3.7.0 with proper API version negotiation)
- **Files modified:** pom.xml
- **Verification:** ClickHouse container starts successfully
- **Committed in:** 2d3fde3

**3. [Rule 1 - Bug] Fixed ClickHouse TTL expression for DateTime64 columns**
- **Found during:** Task 2 (schema init in tests)
- **Issue:** ClickHouse 25.3 requires TTL expressions to resolve to DateTime, not DateTime64(3, 'UTC')
- **Fix:** Changed `TTL start_time + INTERVAL 30 DAY` to `TTL toDateTime(start_time) + toIntervalDay(30)`
- **Files modified:** clickhouse/init/01-schema.sql
- **Verification:** Schema creates without errors in ClickHouse 25.3 container
- **Committed in:** 2d3fde3

**4. [Rule 1 - Bug] Fixed tokenbf_v1 index on Nullable column**
- **Found during:** Task 2 (schema init in tests)
- **Issue:** ClickHouse 25.3 does not allow tokenbf_v1 skip indexes on Nullable(String) columns
- **Fix:** Changed error_message and error_stacktrace from Nullable(String) to String DEFAULT ''
- **Files modified:** clickhouse/init/01-schema.sql
- **Verification:** Schema creates without errors, all 12 tests pass
- **Committed in:** 2d3fde3

---

**Total deviations:** 4 auto-fixed (2 blocking, 2 bugs)
**Impact on plan:** All fixes necessary for test execution on current Docker Desktop and ClickHouse versions. No scope creep.

## Issues Encountered
- Surefire runs from module directory, not project root; fixed schema path lookup in AbstractClickHouseIT to check both locations

## User Setup Required
None - no external service configuration required.

## Next Phase Readiness
- AbstractClickHouseIT base class ready for all future integration tests
- Protocol version interceptor active for data/agent endpoints
- API foundation complete, ready for Plan 02 (REST controllers, ClickHouse repositories, flush scheduler)
- Health endpoint at /api/v1/health, OpenAPI at /api/v1/api-docs, Swagger UI at /api/v1/swagger-ui

## Self-Check: PASSED

All 9 created files verified present. Both task commits verified in git log.

---
*Phase: 01-ingestion-pipeline-api-foundation*
*Completed: 2026-03-11*