Commit Graph

27 Commits

Author SHA1 Message Date
hsiegeln
9dffa9ea81 Move schema initialization from ClickHouse init scripts to server startup
All checks were successful
CI / build (push) Successful in 49s
CI / docker (push) Successful in 43s
CI / deploy (push) Successful in 15s
Server now applies schema via @PostConstruct using classpath SQL files.
All statements use IF NOT EXISTS/IF NOT EXISTS so it's idempotent and
safe to run on every startup. Removes ConfigMap and init script mount
from K8s manifest since ClickHouse no longer needs to manage the schema.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-12 19:59:33 +01:00
hsiegeln
539b85f307 test(04-02): adapt all ITs for JWT auth and add 4 security integration tests
- Replace TestSecurityConfig permit-all with real SecurityConfig active in tests
- Create TestSecurityHelper for JWT-authenticated test requests
- Update 15 existing ITs to use JWT Bearer auth and bootstrap token headers
- Add SecurityFilterIT: protected/public endpoint access control (6 tests)
- Add BootstrapTokenIT: registration requires valid bootstrap token (4 tests)
- Add RegistrationSecurityIT: registration returns tokens + public key (3 tests)
- Add JwtRefreshIT: refresh flow with valid/invalid/mismatched tokens (5 tests)
- Add /error to SecurityConfig permitAll for proper error page forwarding
- Exclude register and refresh paths from ProtocolVersionInterceptor
- All 91 tests pass (18 new security + 73 existing)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:38:28 +01:00
hsiegeln
0215fd96ae feat(04-03): implement SSE payload signing with Ed25519
- SsePayloadSigner signs JSON payloads and adds signature field before SSE delivery
- SseConnectionManager signs all command payloads via SsePayloadSigner before sendEvent
- Signed payload parsed to JsonNode for correct SseEmitter serialization
- Integration tests use bootstrap token + JWT auth (adapts to Plan 02 security layer)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:29:54 +01:00
hsiegeln
387e2e66b2 feat(04-02): wire Spring Security filter chain with JWT auth, bootstrap registration, and refresh endpoint
- JwtAuthenticationFilter extracts JWT from Authorization header or query param, validates via JwtService
- SecurityConfig creates stateless SecurityFilterChain with public/protected endpoint split
- AgentRegistrationController requires bootstrap token, returns accessToken + refreshToken + serverPublicKey
- New POST /agents/{id}/refresh endpoint issues new access JWT from valid refresh token

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:13:53 +01:00
hsiegeln
b3b4e62d34 test(04-03): add failing tests for SSE payload signing
- SsePayloadSignerTest: 7 unit tests for sign/verify roundtrip and edge cases
- SseSigningIT: 2 integration tests for end-to-end SSE signature verification

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:13:44 +01:00
hsiegeln
ac9e8ae4dd feat(04-01): implement security service foundation
- JwtServiceImpl: HMAC-SHA256 via Nimbus JOSE+JWT with ephemeral 256-bit secret
- Ed25519SigningServiceImpl: JDK 17 KeyPairGenerator with ephemeral keypair
- BootstrapTokenValidator: constant-time comparison with dual-token rotation
- SecurityBeanConfig: bean wiring with fail-fast validation for CAMELEER_AUTH_TOKEN
- SecurityProperties: config binding for token expiry and bootstrap tokens
- TestSecurityConfig: permit-all filter chain to keep existing tests green
- application.yml: security config with env var mapping
- All 18 security unit tests pass, all 71 tests pass in full verify

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 20:08:30 +01:00
hsiegeln
51a02700dd test(04-01): add failing tests for security services
- JwtService: 7 tests for access/refresh token creation and validation
- Ed25519SigningService: 5 tests for keypair, signing, verification
- BootstrapTokenValidator: 6 tests for token matching and rotation
- Core interfaces and stub implementations (all throw UnsupportedOperationException)
- Added nimbus-jose-jwt and spring-boot-starter-security dependencies

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:58:59 +01:00
hsiegeln
a1909baad6 test(03-02): integration tests for SSE and command endpoints
- AgentSseControllerIT: connect, 404 unknown, config-update/deep-trace/replay delivery, ping keepalive, Last-Event-ID
- AgentCommandControllerIT: single/group/broadcast commands, ack, ack-unknown, command-to-unregistered
- Test config with 1s ping interval for faster SSE keepalive testing
- All 71 tests pass with mvn clean verify

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 19:16:25 +01:00
hsiegeln
5746886a0b feat(03-02): SSE connection manager, SSE endpoint, and command controller
- SseConnectionManager with per-agent SseEmitter, ping keepalive, event delivery
- AgentSseController GET /{id}/events SSE endpoint with Last-Event-ID support
- AgentCommandController with single/group/broadcast command targeting + ack
- WebConfig excludes SSE events path from protocol version interceptor

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 18:45:47 +01:00
hsiegeln
0372be2334 feat(03-01): add agent registration controller, config, lifecycle monitor
- AgentRegistryConfig: heartbeat, stale, dead, ping, command expiry settings
- AgentRegistryBeanConfig: wires AgentRegistryService as Spring bean
- AgentLifecycleMonitor: @Scheduled lifecycle check + command expiry sweep
- AgentRegistrationController: POST /register, POST /{id}/heartbeat, GET /agents
- Updated Cameleer3ServerApplication with AgentRegistryConfig
- Updated application.yml with agent-registry section and async timeout
- 7 integration tests: register, re-register, heartbeat, list, filter, invalid status

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 18:40:57 +01:00
hsiegeln
34c831040a feat(02-04): populate diagram_content_hash during ingestion and fix test stability
- Inject DiagramRepository into ClickHouseExecutionRepository for hash lookup
- Replace empty string placeholder with actual SHA-256 diagram hash in insertBatch
- Add Surefire/Failsafe forkCount=1 reuseForks=false for classloader isolation
- Add failsafe-plugin integration-test/verify goals for IT execution
- Create DiagramLinkingIT with positive (hash populated) and negative (empty fallback) cases
- Fix flaky awaitility assertions with ignoreExceptions for EmptyResultDataAccess
- Increase IngestionSchemaIT timeouts to 30s for reliable batch flush waits
- Adjust SearchControllerIT pagination assertion to match correct seed data count

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 17:36:33 +01:00
hsiegeln
ea9d81213f test(02-04): add failing DiagramLinkingIT for diagram hash linking
- Test 1: diagram_content_hash populated with SHA-256 when RouteGraph exists
- Test 2: diagram_content_hash empty when no RouteGraph exists for route

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:51:34 +01:00
hsiegeln
079dce5daf fix(02-03): make search tests resilient to shared ClickHouse data
- Use correlationId scoping and >= assertions for status/duration tests
- Prevents false failures when other test classes seed data in same container

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:32:50 +01:00
hsiegeln
0615a9851d feat(02-03): detail controller, tree reconstruction, processor snapshot endpoint
- Implement findRawById and findProcessorSnapshot in ClickHouseExecutionRepository
- DetailController with GET /executions/{id} returning nested processor tree
- GET /executions/{id}/processors/{index}/snapshot for per-processor exchange data
- 5 unit tests for tree reconstruction (linear, branching, multiple roots, empty)
- 6 integration tests for detail endpoint, snapshot, and 404 handling
- Added assertj and mockito test dependencies to core module

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:29:53 +01:00
hsiegeln
82a190c8e2 feat(02-03): ClickHouse search engine, search controller, and 13 integration tests
- ClickHouseSearchEngine with dynamic WHERE clause building and LIKE escape
- SearchController with GET (basic filters) and POST (advanced JSON body)
- SearchBeanConfig wiring SearchEngine, SearchService, DetailService beans
- 13 integration tests covering all filter types, combinations, pagination, empty results

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:23:20 +01:00
hsiegeln
c1bc32d50a feat(02-02): implement ELK diagram renderer with SVG/JSON content negotiation
- ElkDiagramRenderer: ELK layered layout (top-to-bottom) with JFreeSVG rendering
- Color-coded nodes: blue endpoints, green processors, red errors, purple EIPs, cyan cross-route
- Compound node support for CHOICE/SPLIT/TRY_CATCH swimlane groups
- DiagramRenderController: GET /api/v1/diagrams/{hash}/render with Accept header negotiation
- DiagramBeanConfig for Spring wiring
- 11 unit tests (layout, SVG structure, colors, compound nodes)
- 4 integration tests (SVG, JSON, 404, default format)
- Added xtext xbase lib dependency for ELK compatibility

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:17:13 +01:00
hsiegeln
f6ff279a60 feat(02-01): extend ingestion to populate Phase 2 columns with integration tests
- Refactored flattenProcessors to flattenWithMetadata (FlatProcessor record with depth/parentIndex)
- INSERT now populates 12 new columns: exchange_bodies, exchange_headers, processor_depths,
  processor_parent_indexes, processor_error_messages, processor_error_stacktraces,
  processor_input_bodies, processor_output_bodies, processor_input_headers,
  processor_output_headers, processor_diagram_node_ids, diagram_content_hash
- Exchange bodies/headers concatenated from all processor snapshots + route-level snapshots
- Null ExchangeSnapshot handled gracefully (empty string defaults)
- Headers serialized to JSON via Jackson ObjectMapper
- IngestionSchemaIT verifies 3-level tree metadata, body concatenation, null snapshot handling
- DiagramRenderer/DiagramLayout stubs created to fix pre-existing compilation error (Rule 3)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:15:41 +01:00
hsiegeln
c0922430c4 test(02-01): add failing IngestionSchemaIT for new column population
- Tests processor tree metadata (depths, parent indexes)
- Tests exchange body concatenation for search
- Tests null snapshot graceful handling
- AbstractClickHouseIT loads 02-search-columns.sql
- DiagramRenderer/DiagramLayout stubs to fix pre-existing compilation error

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:09:45 +01:00
hsiegeln
6df74505be feat(02-02): add ELK/JFreeSVG dependencies and core diagram rendering interfaces
- Add Eclipse ELK core + layered algorithm and JFreeSVG dependencies to app module
- Create DiagramRenderer interface in core with renderSvg and layoutJson methods
- Create DiagramLayout, PositionedNode, PositionedEdge records for layout data

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:04:32 +01:00
hsiegeln
8fe65f083c feat(01-02): implement ingestion REST controllers with backpressure
- ExecutionController: POST /api/v1/data/executions (single or array)
- DiagramController: POST /api/v1/data/diagrams (single or array)
- MetricsController: POST /api/v1/data/metrics (array)
- All return 202 Accepted or 503 with Retry-After when buffer full
- Fix duplicate IngestionConfig bean (remove @Configuration, use @EnableConfigurationProperties)
- Fix BackpressureIT timing by using batch POST and 60s flush interval
- All 11 integration tests green

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:13:27 +01:00
hsiegeln
d55ebc1f57 test(01-02): add failing integration tests for ingestion endpoints
- ExecutionControllerIT: single/array POST, flush verification, unknown fields
- DiagramControllerIT: single/array POST, flush verification
- MetricsControllerIT: POST metrics, flush verification
- BackpressureIT: buffer-full returns 503, buffered data not lost

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:10:49 +01:00
hsiegeln
17a18cf6da feat(01-02): add IngestionService, ClickHouse repositories, and flush scheduler
- IngestionService routes data to WriteBuffer instances (core module, plain class)
- ClickHouseExecutionRepository: batch insert with parallel processor arrays
- ClickHouseDiagramRepository: JSON storage with SHA-256 content-hash dedup
- ClickHouseMetricsRepository: batch insert for agent_metrics table
- ClickHouseFlushScheduler: scheduled drain with SmartLifecycle shutdown flush
- IngestionBeanConfig: wires WriteBuffer and IngestionService beans

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:08:36 +01:00
hsiegeln
2d3fde3766 test(01-03): add integration tests for health, OpenAPI, protocol version, forward compat, and TTL
- HealthControllerIT: health returns 200, no protocol header needed, TTL verified
- OpenApiIT: api-docs returns OpenAPI spec, swagger UI accessible
- ProtocolVersionIT: missing/wrong header returns 400, correct header passes, excluded paths work
- ForwardCompatIT: unknown JSON fields do not cause deserialization errors
- Fix testcontainers version to 2.0.3 (docker-java 3.7.0 for Docker Desktop 29.x compat)
- Fix ClickHouse schema: TTL with toDateTime() cast, non-nullable error columns for tokenbf_v1

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 12:03:00 +01:00
hsiegeln
b8a4739f72 feat(01-03): add test infrastructure, protocol version interceptor, and app bootstrap
- AbstractClickHouseIT base class with Testcontainers ClickHouse and schema init
- ProtocolVersionInterceptor validates X-Cameleer-Protocol-Version:1 on data/agent paths
- WebConfig registers interceptor with path patterns, excludes health/docs
- Cameleer3ServerApplication with @EnableScheduling and component scanning
- application-test.yml with small buffer config for tests

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:53:31 +01:00
hsiegeln
cc1c082adb feat(01-01): add WriteBuffer, repository interfaces, and config classes
- WriteBuffer<T> with offer/offerBatch/drain and backpressure (all tests green)
- ExecutionRepository, DiagramRepository, MetricsRepository interfaces
- MetricsSnapshot record for agent metrics data
- IngestionConfig for buffer-capacity/batch-size/flush-interval-ms properties
- ClickHouseConfig exposing JdbcTemplate bean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:49:25 +01:00
hsiegeln
96c52b88d1 feat(01-01): add ClickHouse dependencies, Docker Compose, schema, and app config
- Add clickhouse-jdbc, springdoc-openapi, actuator, testcontainers deps
- Add slf4j-api to core module
- Create Docker Compose with ClickHouse service on ports 8123/9000
- Create ClickHouse DDL: route_executions, route_diagrams, agent_metrics
- Configure application.yml with datasource, ingestion buffer, springdoc

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 11:47:20 +01:00
hsiegeln
db17f02fcc Scaffold cameleer3-server project structure
Some checks failed
CI / build (push) Failing after 3s
Multi-module Maven project (server-core + server-app) with Spring Boot 3.4.3,
Gitea CI workflow, and dependency on cameleer3-common from Gitea Maven registry.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 10:06:17 +01:00