cameleer/cameleer-server
1
0
Fork 0
Code Issues 41 Pull Requests Actions Packages Projects Releases Wiki Activity

chore: rename cameleer3 to cameleer
Some checks failed
CI / cleanup-branch (push) Has been skipped
Details
CI / build (push) Failing after 18s
Details
CI / docker (push) Has been skipped
Details
CI / deploy (push) Has been skipped
Details
CI / deploy-feature (push) Has been skipped
Details

Browse Source 
Rename Java packages from com.cameleer3 to com.cameleer, module
directories from cameleer3-* to cameleer-*, and all references
throughout workflows, Dockerfiles, docs, migrations, and pom.xml.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
hsiegeln
2026-04-15 15:28:42 +02:00
parent 1077293343
commit cb3ebfea7c
569 changed files with 4356 additions and 3245 deletions
Download Patch File Download Diff File Expand all files Collapse all files

56
.planning/phases/01-ingestion-pipeline-api-foundation/01-RESEARCH.md
View File

@@ -6,7 +6,7 @@
## Summary
Phase 1 establishes the data pipeline and API skeleton for Cameleer3 Server. Agents POST execution data, diagrams, and metrics to REST endpoints; the server buffers these in memory and batch-flushes to ClickHouse. The ClickHouse schema design is the most critical and least reversible decision in this phase -- ORDER BY and partitioning cannot be changed without table recreation.
Phase 1 establishes the data pipeline and API skeleton for Cameleer Server. Agents POST execution data, diagrams, and metrics to REST endpoints; the server buffers these in memory and batch-flushes to ClickHouse. The ClickHouse schema design is the most critical and least reversible decision in this phase -- ORDER BY and partitioning cannot be changed without table recreation.
The ClickHouse Java ecosystem has undergone significant changes. The recommended approach is **clickhouse-jdbc v0.9.7** (JDBC V2 driver) with Spring Boot's JdbcTemplate for batch inserts. An alternative is the standalone **client-v2** artifact which offers a POJO-based insert API, but JDBC integration with Spring Boot is more conventional and better documented. ClickHouse now has a native full-text index (TYPE text, GA as of March 2026) that supersedes the older tokenbf_v1 bloom filter approach -- this is relevant for Phase 2 but should be accounted for in schema design now.
@@ -17,7 +17,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
| ID | Description | Research Support |
|----|-------------|-----------------|
| INGST-01 (#1) | Accept RouteExecution via POST /api/v1/data/executions, return 202 | REST controller + async write buffer pattern; Jackson deserialization of cameleer3-common models |
| INGST-01 (#1) | Accept RouteExecution via POST /api/v1/data/executions, return 202 | REST controller + async write buffer pattern; Jackson deserialization of cameleer-common models |
| INGST-02 (#2) | Accept RouteGraph via POST /api/v1/data/diagrams, return 202 | Same pattern; separate ClickHouse table for diagrams with content-hash dedup |
| INGST-03 (#3) | Accept metrics via POST /api/v1/data/metrics, return 202 | Same pattern; separate ClickHouse table for metrics |
| INGST-04 (#4) | In-memory batch buffer with configurable flush interval/size | ArrayBlockingQueue + @Scheduled flush; configurable via application.yml |
@@ -60,7 +60,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
| ArrayBlockingQueue | LMAX Disruptor | Disruptor is faster under extreme contention but adds complexity; ABQ is sufficient for this throughput |
| Spring JdbcTemplate | Raw JDBC PreparedStatement | JdbcTemplate provides cleaner error handling and resource management; no meaningful overhead |
**Installation (add to cameleer3-server-app/pom.xml):**
**Installation (add to cameleer-server-app/pom.xml):**
```xml
<!-- ClickHouse JDBC V2 -->
<dependency>
@@ -103,7 +103,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
</dependency>
```
**Add to cameleer3-server-core/pom.xml:**
**Add to cameleer-server-core/pom.xml:**
```xml
<!-- SLF4J for logging (no Spring dependency) -->
<dependency>
@@ -117,7 +117,7 @@ The ClickHouse Java ecosystem has undergone significant changes. The recommended
### Recommended Project Structure
```
cameleer3-server-core/src/main/java/com/cameleer3/server/core/
cameleer-server-core/src/main/java/com/cameleer/server/core/
ingestion/
WriteBuffer.java # Bounded queue + flush logic
IngestionService.java # Accepts data, routes to buffer
@@ -126,9 +126,9 @@ cameleer3-server-core/src/main/java/com/cameleer3/server/core/
DiagramRepository.java # Interface: store/retrieve diagrams
MetricsRepository.java # Interface: store metrics
model/
(extend/complement cameleer3-common models as needed)
(extend/complement cameleer-common models as needed)
cameleer3-server-app/src/main/java/com/cameleer3/server/app/
cameleer-server-app/src/main/java/com/cameleer/server/app/
config/
ClickHouseConfig.java # DataSource + JdbcTemplate bean
IngestionConfig.java # Buffer size, flush interval from YAML
@@ -424,7 +424,7 @@ services:
environment:
CLICKHOUSE_USER: cameleer
CLICKHOUSE_PASSWORD: cameleer_dev
CLICKHOUSE_DB: cameleer3
CLICKHOUSE_DB: cameleer
ulimits:
nofile:
soft: 262144
@@ -442,7 +442,7 @@ server:
spring:
datasource:
url: jdbc:ch://localhost:8123/cameleer3
url: jdbc:ch://localhost:8123/cameleer
username: cameleer
password: cameleer_dev
driver-class-name: com.clickhouse.jdbc.ClickHouseDriver
@@ -493,10 +493,10 @@ management:
## Open Questions
1. **Exact cameleer3-common model structure**
1. **Exact cameleer-common model structure**
- What we know: Models include RouteExecution, ProcessorExecution, ExchangeSnapshot, RouteGraph, RouteNode, RouteEdge
- What's unclear: Exact field names, types, nesting structure -- needed to design ClickHouse schema precisely
- Recommendation: Read cameleer3-common source code before implementing schema. Schema must match the wire format.
- Recommendation: Read cameleer-common source code before implementing schema. Schema must match the wire format.
2. **ClickHouse JDBC V2 + HikariCP compatibility**
- What we know: clickhouse-jdbc 0.9.7 implements JDBC spec; HikariCP is Spring Boot default
@@ -515,36 +515,36 @@ management:
| Property | Value |
|----------|-------|
| Framework | JUnit 5 (Spring Boot managed) + Testcontainers 2.0.2 |
| Config file | cameleer3-server-app/src/test/resources/application-test.yml (Wave 0) |
| Quick run command | `mvn test -pl cameleer3-server-core -Dtest=WriteBufferTest -q` |
| Config file | cameleer-server-app/src/test/resources/application-test.yml (Wave 0) |
| Quick run command | `mvn test -pl cameleer-server-core -Dtest=WriteBufferTest -q` |
| Full suite command | `mvn verify` |
### Phase Requirements -> Test Map
| Req ID | Behavior | Test Type | Automated Command | File Exists? |
|--------|----------|-----------|-------------------|-------------|
| INGST-01 | POST /api/v1/data/executions returns 202, data in ClickHouse | integration | `mvn test -pl cameleer3-server-app -Dtest=ExecutionControllerIT -q` | Wave 0 |
| INGST-02 | POST /api/v1/data/diagrams returns 202 | integration | `mvn test -pl cameleer3-server-app -Dtest=DiagramControllerIT -q` | Wave 0 |
| INGST-03 | POST /api/v1/data/metrics returns 202 | integration | `mvn test -pl cameleer3-server-app -Dtest=MetricsControllerIT -q` | Wave 0 |
| INGST-04 | Buffer flushes at interval/size | unit | `mvn test -pl cameleer3-server-core -Dtest=WriteBufferTest -q` | Wave 0 |
| INGST-05 | 503 when buffer full | unit+integration | `mvn test -pl cameleer3-server-app -Dtest=BackpressureIT -q` | Wave 0 |
| INGST-06 | TTL removes old data | integration | `mvn test -pl cameleer3-server-app -Dtest=ClickHouseTtlIT -q` | Wave 0 |
| INGST-01 | POST /api/v1/data/executions returns 202, data in ClickHouse | integration | `mvn test -pl cameleer-server-app -Dtest=ExecutionControllerIT -q` | Wave 0 |
| INGST-02 | POST /api/v1/data/diagrams returns 202 | integration | `mvn test -pl cameleer-server-app -Dtest=DiagramControllerIT -q` | Wave 0 |
| INGST-03 | POST /api/v1/data/metrics returns 202 | integration | `mvn test -pl cameleer-server-app -Dtest=MetricsControllerIT -q` | Wave 0 |
| INGST-04 | Buffer flushes at interval/size | unit | `mvn test -pl cameleer-server-core -Dtest=WriteBufferTest -q` | Wave 0 |
| INGST-05 | 503 when buffer full | unit+integration | `mvn test -pl cameleer-server-app -Dtest=BackpressureIT -q` | Wave 0 |
| INGST-06 | TTL removes old data | integration | `mvn test -pl cameleer-server-app -Dtest=ClickHouseTtlIT -q` | Wave 0 |
| API-01 | Endpoints under /api/v1/ | integration | Covered by controller ITs | Wave 0 |
| API-02 | OpenAPI docs available | integration | `mvn test -pl cameleer3-server-app -Dtest=OpenApiIT -q` | Wave 0 |
| API-03 | GET /api/v1/health responds | integration | `mvn test -pl cameleer3-server-app -Dtest=HealthControllerIT -q` | Wave 0 |
| API-04 | Protocol version header validated | integration | `mvn test -pl cameleer3-server-app -Dtest=ProtocolVersionIT -q` | Wave 0 |
| API-05 | Unknown JSON fields accepted | unit | `mvn test -pl cameleer3-server-app -Dtest=ForwardCompatIT -q` | Wave 0 |
| API-02 | OpenAPI docs available | integration | `mvn test -pl cameleer-server-app -Dtest=OpenApiIT -q` | Wave 0 |
| API-03 | GET /api/v1/health responds | integration | `mvn test -pl cameleer-server-app -Dtest=HealthControllerIT -q` | Wave 0 |
| API-04 | Protocol version header validated | integration | `mvn test -pl cameleer-server-app -Dtest=ProtocolVersionIT -q` | Wave 0 |
| API-05 | Unknown JSON fields accepted | unit | `mvn test -pl cameleer-server-app -Dtest=ForwardCompatIT -q` | Wave 0 |
### Sampling Rate
- **Per task commit:** `mvn test -pl cameleer3-server-core -q` (unit tests, fast)
- **Per task commit:** `mvn test -pl cameleer-server-core -q` (unit tests, fast)
- **Per wave merge:** `mvn verify` (full suite with Testcontainers integration tests)
- **Phase gate:** Full suite green before verification
### Wave 0 Gaps
- [ ] `cameleer3-server-app/src/test/resources/application-test.yml` -- test ClickHouse config
- [ ] `cameleer3-server-core/src/test/java/.../WriteBufferTest.java` -- buffer unit tests
- [ ] `cameleer3-server-app/src/test/java/.../AbstractClickHouseIT.java` -- shared Testcontainers base class
- [ ] `cameleer3-server-app/src/test/java/.../ExecutionControllerIT.java` -- ingestion integration test
- [ ] `cameleer-server-app/src/test/resources/application-test.yml` -- test ClickHouse config
- [ ] `cameleer-server-core/src/test/java/.../WriteBufferTest.java` -- buffer unit tests
- [ ] `cameleer-server-app/src/test/java/.../AbstractClickHouseIT.java` -- shared Testcontainers base class
- [ ] `cameleer-server-app/src/test/java/.../ExecutionControllerIT.java` -- ingestion integration test
- [ ] Docker available on test machine for Testcontainers
## Sources