chore: rename cameleer3 to cameleer
Rename Java packages from com.cameleer3 to com.cameleer, module directories from cameleer3-* to cameleer-*, and all references throughout workflows, Dockerfiles, docs, migrations, and pom.xml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
hsiegeln
@@ -16,7 +16,7 @@
|
||||
|
||||
### New files
|
||||
|
||||
**Core module** (`cameleer3-server-core/src/main/java/com/cameleer3/server/core/`):
|
||||
**Core module** (`cameleer-server-core/src/main/java/com/cameleer/server/core/`):
|
||||
- `storage/ExecutionStore.java` — new interface replacing ExecutionRepository
|
||||
- `storage/StatsStore.java` — new interface for stats from continuous aggregates
|
||||
- `storage/SearchIndex.java` — new interface for OpenSearch operations
|
||||
@@ -28,7 +28,7 @@
|
||||
- `indexing/SearchIndexer.java` — debounced event listener for OpenSearch indexing
|
||||
- `indexing/ExecutionUpdatedEvent.java` — event published after execution write
|
||||
|
||||
**App module** (`cameleer3-server-app/src/main/java/com/cameleer3/server/app/`):
|
||||
**App module** (`cameleer-server-app/src/main/java/com/cameleer/server/app/`):
|
||||
- `storage/PostgresExecutionStore.java` — ExecutionStore impl with upsert
|
||||
- `storage/PostgresStatsStore.java` — StatsStore impl querying continuous aggregates
|
||||
- `storage/PostgresDiagramStore.java` — DiagramStore impl
|
||||
@@ -41,7 +41,7 @@
|
||||
- `ingestion/MetricsFlushScheduler.java` — scheduled metrics buffer flush (replaces ClickHouseFlushScheduler, metrics only)
|
||||
- `retention/RetentionScheduler.java` — scheduled job for drop_chunks and OpenSearch index deletion
|
||||
|
||||
**Flyway migrations** (`cameleer3-server-app/src/main/resources/db/migration/`):
|
||||
**Flyway migrations** (`cameleer-server-app/src/main/resources/db/migration/`):
|
||||
- `V1__extensions.sql` — CREATE EXTENSION timescaledb, timescaledb_toolkit
|
||||
- `V2__executions.sql` — executions hypertable
|
||||
- `V3__processor_executions.sql` — processor_executions hypertable
|
||||
@@ -53,7 +53,7 @@
|
||||
|
||||
Note: Retention is NOT in a Flyway migration (Flyway migrations are immutable once applied). No V9 file. Retention is handled by `RetentionScheduler` at runtime with configurable intervals.
|
||||
|
||||
**Test files** (`cameleer3-server-app/src/test/java/com/cameleer3/server/app/`):
|
||||
**Test files** (`cameleer-server-app/src/test/java/com/cameleer/server/app/`):
|
||||
- `AbstractPostgresIT.java` — replaces AbstractClickHouseIT (TimescaleDB Testcontainer)
|
||||
- `storage/PostgresExecutionStoreIT.java` — upsert, dedup, chunked arrival tests
|
||||
- `storage/PostgresStatsStoreIT.java` — continuous aggregate query tests
|
||||
@@ -64,35 +64,35 @@ Note: Retention is NOT in a Flyway migration (Flyway migrations are immutable on
|
||||
### Files to modify
|
||||
|
||||
- `pom.xml` (root) — no changes needed
|
||||
- `cameleer3-server-app/pom.xml` — swap clickhouse-jdbc for postgresql + opensearch-java + flyway
|
||||
- `cameleer3-server-core/.../core/search/SearchService.java` — split: search delegates to SearchIndex, stats/timeseries to StatsStore
|
||||
- `cameleer3-server-core/.../core/detail/DetailService.java` — use ExecutionStore instead of ExecutionRepository
|
||||
- `cameleer3-server-core/.../core/detail/RawExecutionRow.java` — remove (replaced by normalized model)
|
||||
- `cameleer3-server-core/.../core/ingestion/IngestionService.java` — synchronous execution/diagram writes, keep buffer for metrics
|
||||
- `cameleer3-server-app/.../app/config/SearchBeanConfig.java` — wire StatsStore into SearchService
|
||||
- `cameleer3-server-app/.../app/config/IngestionBeanConfig.java` — update bean wiring
|
||||
- `cameleer3-server-app/src/main/resources/application.yml` — PostgreSQL + OpenSearch config
|
||||
- `cameleer3-server-app/src/test/resources/application-test.yml` — test config
|
||||
- `cameleer-server-app/pom.xml` — swap clickhouse-jdbc for postgresql + opensearch-java + flyway
|
||||
- `cameleer-server-core/.../core/search/SearchService.java` — split: search delegates to SearchIndex, stats/timeseries to StatsStore
|
||||
- `cameleer-server-core/.../core/detail/DetailService.java` — use ExecutionStore instead of ExecutionRepository
|
||||
- `cameleer-server-core/.../core/detail/RawExecutionRow.java` — remove (replaced by normalized model)
|
||||
- `cameleer-server-core/.../core/ingestion/IngestionService.java` — synchronous execution/diagram writes, keep buffer for metrics
|
||||
- `cameleer-server-app/.../app/config/SearchBeanConfig.java` — wire StatsStore into SearchService
|
||||
- `cameleer-server-app/.../app/config/IngestionBeanConfig.java` — update bean wiring
|
||||
- `cameleer-server-app/src/main/resources/application.yml` — PostgreSQL + OpenSearch config
|
||||
- `cameleer-server-app/src/test/resources/application-test.yml` — test config
|
||||
|
||||
### Files to delete
|
||||
|
||||
- `cameleer3-server-app/.../app/storage/ClickHouseExecutionRepository.java`
|
||||
- `cameleer3-server-app/.../app/storage/ClickHouseDiagramRepository.java`
|
||||
- `cameleer3-server-app/.../app/storage/ClickHouseMetricsRepository.java`
|
||||
- `cameleer3-server-app/.../app/storage/ClickHouseUserRepository.java`
|
||||
- `cameleer3-server-app/.../app/storage/ClickHouseOidcConfigRepository.java`
|
||||
- `cameleer3-server-app/.../app/search/ClickHouseSearchEngine.java`
|
||||
- `cameleer3-server-app/.../app/ingestion/ClickHouseFlushScheduler.java`
|
||||
- `cameleer3-server-app/.../app/config/ClickHouseConfig.java`
|
||||
- `cameleer3-server-core/.../core/storage/ExecutionRepository.java`
|
||||
- `cameleer3-server-core/.../core/storage/DiagramRepository.java`
|
||||
- `cameleer3-server-core/.../core/storage/MetricsRepository.java`
|
||||
- `cameleer3-server-core/.../core/search/SearchEngine.java`
|
||||
- `cameleer3-server-core/.../core/detail/RawExecutionRow.java`
|
||||
- `cameleer-server-app/.../app/storage/ClickHouseExecutionRepository.java`
|
||||
- `cameleer-server-app/.../app/storage/ClickHouseDiagramRepository.java`
|
||||
- `cameleer-server-app/.../app/storage/ClickHouseMetricsRepository.java`
|
||||
- `cameleer-server-app/.../app/storage/ClickHouseUserRepository.java`
|
||||
- `cameleer-server-app/.../app/storage/ClickHouseOidcConfigRepository.java`
|
||||
- `cameleer-server-app/.../app/search/ClickHouseSearchEngine.java`
|
||||
- `cameleer-server-app/.../app/ingestion/ClickHouseFlushScheduler.java`
|
||||
- `cameleer-server-app/.../app/config/ClickHouseConfig.java`
|
||||
- `cameleer-server-core/.../core/storage/ExecutionRepository.java`
|
||||
- `cameleer-server-core/.../core/storage/DiagramRepository.java`
|
||||
- `cameleer-server-core/.../core/storage/MetricsRepository.java`
|
||||
- `cameleer-server-core/.../core/search/SearchEngine.java`
|
||||
- `cameleer-server-core/.../core/detail/RawExecutionRow.java`
|
||||
|
||||
Note: `UserRepository` and `OidcConfigRepository` interfaces in `core.security` are **kept** — the new Postgres implementations implement these existing interfaces. No rename needed since their contracts are unchanged.
|
||||
- `cameleer3-server-app/src/main/resources/clickhouse/*.sql` (all 8 files)
|
||||
- `cameleer3-server-app/src/test/.../app/AbstractClickHouseIT.java`
|
||||
- `cameleer-server-app/src/main/resources/clickhouse/*.sql` (all 8 files)
|
||||
- `cameleer-server-app/src/test/.../app/AbstractClickHouseIT.java`
|
||||
|
||||
---
|
||||
|
||||
@@ -101,11 +101,11 @@ Note: `UserRepository` and `OidcConfigRepository` interfaces in `core.security`
|
||||
### Task 1: Update Maven dependencies
|
||||
|
||||
**Files:**
|
||||
- Modify: `cameleer3-server-app/pom.xml`
|
||||
- Modify: `cameleer-server-app/pom.xml`
|
||||
|
||||
- [ ] **Step 1: Replace ClickHouse JDBC with PostgreSQL driver + Flyway + OpenSearch client**
|
||||
|
||||
In `cameleer3-server-app/pom.xml`, replace the ClickHouse dependency and add new ones:
|
||||
In `cameleer-server-app/pom.xml`, replace the ClickHouse dependency and add new ones:
|
||||
|
||||
Remove:
|
||||
```xml
|
||||
@@ -165,22 +165,22 @@ Note: `postgresql` driver and `flyway-core` versions are managed by Spring Boot
|
||||
- [ ] **Step 2: Commit** (compilation will fail until ClickHouse code is deleted in Task 16 — this is expected)
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/pom.xml
|
||||
git add cameleer-server-app/pom.xml
|
||||
git commit -m "chore: swap ClickHouse deps for PostgreSQL, Flyway, OpenSearch"
|
||||
```
|
||||
|
||||
### Task 2: Write Flyway migrations
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V1__extensions.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V2__executions.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V3__processor_executions.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V4__agent_metrics.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V5__route_diagrams.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V6__users.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V7__oidc_config.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V8__continuous_aggregates.sql`
|
||||
- Create: `cameleer3-server-app/src/main/resources/db/migration/V9__retention_policies.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V1__extensions.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V2__executions.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V3__processor_executions.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V4__agent_metrics.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V5__route_diagrams.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V6__users.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V7__oidc_config.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V8__continuous_aggregates.sql`
|
||||
- Create: `cameleer-server-app/src/main/resources/db/migration/V9__retention_policies.sql`
|
||||
|
||||
- [ ] **Step 1: Create V1__extensions.sql**
|
||||
|
||||
@@ -405,19 +405,19 @@ SELECT add_continuous_aggregate_policy('stats_1m_processor',
|
||||
- [ ] **Step 9: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/resources/db/migration/
|
||||
git add cameleer-server-app/src/main/resources/db/migration/
|
||||
git commit -m "feat: add Flyway migrations for PostgreSQL/TimescaleDB schema"
|
||||
```
|
||||
|
||||
### Task 3: Create test base class with TimescaleDB Testcontainer
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractPostgresIT.java`
|
||||
- Create: `cameleer-server-app/src/test/java/com/cameleer/server/app/AbstractPostgresIT.java`
|
||||
|
||||
- [ ] **Step 1: Write AbstractPostgresIT**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app;
|
||||
package com.cameleer.server.app;
|
||||
|
||||
import org.springframework.boot.test.context.SpringBootTest;
|
||||
import org.springframework.test.context.DynamicPropertyRegistry;
|
||||
@@ -433,7 +433,7 @@ public abstract class AbstractPostgresIT {
|
||||
@Container
|
||||
static final PostgreSQLContainer<?> postgres =
|
||||
new PostgreSQLContainer<>("timescale/timescaledb:latest-pg16")
|
||||
.withDatabaseName("cameleer3")
|
||||
.withDatabaseName("cameleer")
|
||||
.withUsername("cameleer")
|
||||
.withPassword("test");
|
||||
|
||||
@@ -450,12 +450,12 @@ public abstract class AbstractPostgresIT {
|
||||
|
||||
- [ ] **Step 2: Write a smoke test to verify migrations run**
|
||||
|
||||
Create `cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/FlywayMigrationIT.java`:
|
||||
Create `cameleer-server-app/src/test/java/com/cameleer/server/app/storage/FlywayMigrationIT.java`:
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.app.AbstractPostgresIT;
|
||||
import com.cameleer.server.app.AbstractPostgresIT;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
@@ -493,22 +493,22 @@ class FlywayMigrationIT extends AbstractPostgresIT {
|
||||
|
||||
- [ ] **Step 3: Verify test passes** (this test will not compile until Task 16 deletes ClickHouse code. Run it after Task 16 is complete. Listed here for logical grouping.)
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest=FlywayMigrationIT -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest=FlywayMigrationIT -q`
|
||||
Expected: PASS — all migrations apply, tables and continuous aggregates exist
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractPostgresIT.java
|
||||
git add cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/FlywayMigrationIT.java
|
||||
git add cameleer-server-app/src/test/java/com/cameleer/server/app/AbstractPostgresIT.java
|
||||
git add cameleer-server-app/src/test/java/com/cameleer/server/app/storage/FlywayMigrationIT.java
|
||||
git commit -m "test: add TimescaleDB test base class and Flyway migration smoke test"
|
||||
```
|
||||
|
||||
### Task 4: Update application.yml for PostgreSQL + OpenSearch
|
||||
|
||||
**Files:**
|
||||
- Modify: `cameleer3-server-app/src/main/resources/application.yml`
|
||||
- Modify: `cameleer3-server-app/src/test/resources/application-test.yml`
|
||||
- Modify: `cameleer-server-app/src/main/resources/application.yml`
|
||||
- Modify: `cameleer-server-app/src/test/resources/application-test.yml`
|
||||
|
||||
- [ ] **Step 1: Update application.yml datasource section**
|
||||
|
||||
@@ -516,7 +516,7 @@ Replace:
|
||||
```yaml
|
||||
spring:
|
||||
datasource:
|
||||
url: jdbc:ch://localhost:8123/cameleer3
|
||||
url: jdbc:ch://localhost:8123/cameleer
|
||||
username: cameleer
|
||||
password: cameleer_dev
|
||||
driver-class-name: com.clickhouse.jdbc.ClickHouseDriver
|
||||
@@ -526,7 +526,7 @@ With:
|
||||
```yaml
|
||||
spring:
|
||||
datasource:
|
||||
url: jdbc:postgresql://localhost:5432/cameleer3
|
||||
url: jdbc:postgresql://localhost:5432/cameleer
|
||||
username: cameleer
|
||||
password: ${CAMELEER_DB_PASSWORD:cameleer_dev}
|
||||
driver-class-name: org.postgresql.Driver
|
||||
@@ -565,8 +565,8 @@ opensearch:
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/resources/application.yml
|
||||
git add cameleer3-server-app/src/test/resources/application-test.yml
|
||||
git add cameleer-server-app/src/main/resources/application.yml
|
||||
git add cameleer-server-app/src/test/resources/application-test.yml
|
||||
git commit -m "config: switch datasource to PostgreSQL, add OpenSearch and Flyway config"
|
||||
```
|
||||
|
||||
@@ -577,18 +577,18 @@ git commit -m "config: switch datasource to PostgreSQL, add OpenSearch and Flywa
|
||||
### Task 5: Create new storage interfaces in core module
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/ExecutionStore.java`
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/StatsStore.java`
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/SearchIndex.java`
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/DiagramStore.java`
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/MetricsStore.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ExecutionStore.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/StatsStore.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/SearchIndex.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/DiagramStore.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/MetricsStore.java`
|
||||
|
||||
- [ ] **Step 1: Create ExecutionStore interface**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.storage;
|
||||
package com.cameleer.server.core.storage;
|
||||
|
||||
import com.cameleer3.server.core.detail.ProcessorNode;
|
||||
import com.cameleer.server.core.detail.ProcessorNode;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
@@ -629,10 +629,10 @@ public interface ExecutionStore {
|
||||
Supports all 4 aggregation levels: global, per-app, per-route, per-processor.
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.storage;
|
||||
package com.cameleer.server.core.storage;
|
||||
|
||||
import com.cameleer3.server.core.search.ExecutionStats;
|
||||
import com.cameleer3.server.core.search.StatsTimeseries;
|
||||
import com.cameleer.server.core.search.ExecutionStats;
|
||||
import com.cameleer.server.core.search.StatsTimeseries;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
@@ -670,12 +670,12 @@ public interface StatsStore {
|
||||
- [ ] **Step 3: Create SearchIndex interface**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.storage;
|
||||
package com.cameleer.server.core.storage;
|
||||
|
||||
import com.cameleer3.server.core.search.ExecutionSummary;
|
||||
import com.cameleer3.server.core.search.SearchRequest;
|
||||
import com.cameleer3.server.core.search.SearchResult;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer.server.core.search.ExecutionSummary;
|
||||
import com.cameleer.server.core.search.SearchRequest;
|
||||
import com.cameleer.server.core.search.SearchResult;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument;
|
||||
|
||||
public interface SearchIndex {
|
||||
|
||||
@@ -692,10 +692,10 @@ public interface SearchIndex {
|
||||
- [ ] **Step 4: Create DiagramStore interface**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.storage;
|
||||
package com.cameleer.server.core.storage;
|
||||
|
||||
import com.cameleer3.common.graph.RouteGraph;
|
||||
import com.cameleer3.server.core.ingestion.TaggedDiagram;
|
||||
import com.cameleer.common.graph.RouteGraph;
|
||||
import com.cameleer.server.core.ingestion.TaggedDiagram;
|
||||
|
||||
import java.util.List;
|
||||
import java.util.Optional;
|
||||
@@ -715,9 +715,9 @@ public interface DiagramStore {
|
||||
- [ ] **Step 5: Create MetricsStore interface**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.storage;
|
||||
package com.cameleer.server.core.storage;
|
||||
|
||||
import com.cameleer3.server.core.storage.model.MetricsSnapshot;
|
||||
import com.cameleer.server.core.storage.model.MetricsSnapshot;
|
||||
|
||||
import java.util.List;
|
||||
|
||||
@@ -730,20 +730,20 @@ public interface MetricsStore {
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/storage/
|
||||
git commit -m "feat: add new storage interfaces for PostgreSQL/OpenSearch backends"
|
||||
```
|
||||
|
||||
### Task 6: Create ExecutionDocument model and indexing event
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/model/ExecutionDocument.java`
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/indexing/ExecutionUpdatedEvent.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/storage/model/ExecutionDocument.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/indexing/ExecutionUpdatedEvent.java`
|
||||
|
||||
- [ ] **Step 1: Create ExecutionDocument**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.storage.model;
|
||||
package com.cameleer.server.core.storage.model;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
@@ -767,7 +767,7 @@ public record ExecutionDocument(
|
||||
- [ ] **Step 2: Create ExecutionUpdatedEvent**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.indexing;
|
||||
package com.cameleer.server.core.indexing;
|
||||
|
||||
import java.time.Instant;
|
||||
|
||||
@@ -777,25 +777,25 @@ public record ExecutionUpdatedEvent(String executionId, Instant startTime) {}
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/model/ExecutionDocument.java
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/indexing/
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/storage/model/ExecutionDocument.java
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/indexing/
|
||||
git commit -m "feat: add ExecutionDocument model and ExecutionUpdatedEvent"
|
||||
```
|
||||
|
||||
### Task 7: Update SearchService to use StatsStore for stats/timeseries
|
||||
|
||||
**Files:**
|
||||
- Modify: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/search/SearchService.java`
|
||||
- Modify: `cameleer-server-core/src/main/java/com/cameleer/server/core/search/SearchService.java`
|
||||
|
||||
- [ ] **Step 1: Refactor SearchService to accept SearchIndex + StatsStore**
|
||||
|
||||
Replace the single `SearchEngine` dependency with two dependencies:
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.search;
|
||||
package com.cameleer.server.core.search;
|
||||
|
||||
import com.cameleer3.server.core.storage.SearchIndex;
|
||||
import com.cameleer3.server.core.storage.StatsStore;
|
||||
import com.cameleer.server.core.storage.SearchIndex;
|
||||
import com.cameleer.server.core.storage.StatsStore;
|
||||
|
||||
import java.time.Instant;
|
||||
import java.util.List;
|
||||
@@ -840,24 +840,24 @@ public class SearchService {
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/search/SearchService.java
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/search/SearchService.java
|
||||
git commit -m "refactor: SearchService uses SearchIndex + StatsStore instead of SearchEngine"
|
||||
```
|
||||
|
||||
### Task 8: Update DetailService to use ExecutionStore
|
||||
|
||||
**Files:**
|
||||
- Modify: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/detail/DetailService.java`
|
||||
- Modify: `cameleer-server-core/src/main/java/com/cameleer/server/core/detail/DetailService.java`
|
||||
|
||||
- [ ] **Step 1: Rewrite DetailService to use ExecutionStore**
|
||||
|
||||
The tree reconstruction from parallel arrays is no longer needed. Processors are now individual records with `parentProcessorId` for tree structure.
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.detail;
|
||||
package com.cameleer.server.core.detail;
|
||||
|
||||
import com.cameleer3.server.core.storage.ExecutionStore;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import com.cameleer.server.core.storage.ExecutionStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
|
||||
import java.util.*;
|
||||
|
||||
@@ -920,30 +920,30 @@ public class DetailService {
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/detail/DetailService.java
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/detail/DetailService.java
|
||||
git commit -m "refactor: DetailService uses ExecutionStore, tree built from parentProcessorId"
|
||||
```
|
||||
|
||||
### Task 9: Update IngestionService for synchronous writes
|
||||
|
||||
**Files:**
|
||||
- Modify: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/ingestion/IngestionService.java`
|
||||
- Modify: `cameleer-server-core/src/main/java/com/cameleer/server/core/ingestion/IngestionService.java`
|
||||
|
||||
- [ ] **Step 1: Rewrite IngestionService**
|
||||
|
||||
Executions and diagrams become synchronous writes. Metrics keep the write buffer. Add event publishing for OpenSearch indexing.
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.ingestion;
|
||||
package com.cameleer.server.core.ingestion;
|
||||
|
||||
import com.cameleer3.common.model.ProcessorExecution;
|
||||
import com.cameleer3.common.model.RouteExecution;
|
||||
import com.cameleer3.server.core.indexing.ExecutionUpdatedEvent;
|
||||
import com.cameleer3.server.core.storage.DiagramStore;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import com.cameleer3.server.core.storage.model.MetricsSnapshot;
|
||||
import com.cameleer.common.model.ProcessorExecution;
|
||||
import com.cameleer.common.model.RouteExecution;
|
||||
import com.cameleer.server.core.indexing.ExecutionUpdatedEvent;
|
||||
import com.cameleer.server.core.storage.DiagramStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import com.cameleer.server.core.storage.model.MetricsSnapshot;
|
||||
|
||||
import java.util.ArrayList;
|
||||
import java.util.List;
|
||||
@@ -1055,7 +1055,7 @@ public class IngestionService {
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/ingestion/IngestionService.java
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/ingestion/IngestionService.java
|
||||
git commit -m "refactor: IngestionService uses synchronous ExecutionStore writes with event publishing"
|
||||
```
|
||||
|
||||
@@ -1066,18 +1066,18 @@ git commit -m "refactor: IngestionService uses synchronous ExecutionStore writes
|
||||
### Task 10: Implement PostgresExecutionStore
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresExecutionStore.java`
|
||||
- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/PostgresExecutionStoreIT.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresExecutionStore.java`
|
||||
- Create: `cameleer-server-app/src/test/java/com/cameleer/server/app/storage/PostgresExecutionStoreIT.java`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.app.AbstractPostgresIT;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import com.cameleer.server.app.AbstractPostgresIT;
|
||||
import com.cameleer.server.core.storage.ExecutionStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
|
||||
@@ -1159,15 +1159,15 @@ class PostgresExecutionStoreIT extends AbstractPostgresIT {
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest=PostgresExecutionStoreIT -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest=PostgresExecutionStoreIT -q`
|
||||
Expected: FAIL — `ExecutionStore` bean not found
|
||||
|
||||
- [ ] **Step 3: Implement PostgresExecutionStore**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.core.storage.ExecutionStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.jdbc.core.RowMapper;
|
||||
import org.springframework.stereotype.Repository;
|
||||
@@ -1300,34 +1300,34 @@ public class PostgresExecutionStore implements ExecutionStore {
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest=PostgresExecutionStoreIT -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest=PostgresExecutionStoreIT -q`
|
||||
Expected: PASS — all 3 tests green
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresExecutionStore.java
|
||||
git add cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/PostgresExecutionStoreIT.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresExecutionStore.java
|
||||
git add cameleer-server-app/src/test/java/com/cameleer/server/app/storage/PostgresExecutionStoreIT.java
|
||||
git commit -m "feat: implement PostgresExecutionStore with upsert and dedup"
|
||||
```
|
||||
|
||||
### Task 11: Implement PostgresStatsStore
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresStatsStore.java`
|
||||
- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/PostgresStatsStoreIT.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresStatsStore.java`
|
||||
- Create: `cameleer-server-app/src/test/java/com/cameleer/server/app/storage/PostgresStatsStoreIT.java`
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.app.AbstractPostgresIT;
|
||||
import com.cameleer3.server.core.search.ExecutionStats;
|
||||
import com.cameleer3.server.core.search.StatsTimeseries;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer3.server.core.storage.StatsStore;
|
||||
import com.cameleer.server.app.AbstractPostgresIT;
|
||||
import com.cameleer.server.core.search.ExecutionStats;
|
||||
import com.cameleer.server.core.search.StatsTimeseries;
|
||||
import com.cameleer.server.core.storage.ExecutionStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer.server.core.storage.StatsStore;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
@@ -1385,18 +1385,18 @@ class PostgresStatsStoreIT extends AbstractPostgresIT {
|
||||
|
||||
- [ ] **Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest=PostgresStatsStoreIT -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest=PostgresStatsStoreIT -q`
|
||||
Expected: FAIL — `StatsStore` bean not found
|
||||
|
||||
- [ ] **Step 3: Implement PostgresStatsStore**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.core.search.ExecutionStats;
|
||||
import com.cameleer3.server.core.search.StatsTimeseries;
|
||||
import com.cameleer3.server.core.search.StatsTimeseries.TimeseriesBucket;
|
||||
import com.cameleer3.server.core.storage.StatsStore;
|
||||
import com.cameleer.server.core.search.ExecutionStats;
|
||||
import com.cameleer.server.core.search.StatsTimeseries;
|
||||
import com.cameleer.server.core.search.StatsTimeseries.TimeseriesBucket;
|
||||
import com.cameleer.server.core.storage.StatsStore;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.stereotype.Repository;
|
||||
|
||||
@@ -1582,24 +1582,24 @@ public class PostgresStatsStore implements StatsStore {
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest=PostgresStatsStoreIT -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest=PostgresStatsStoreIT -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresStatsStore.java
|
||||
git add cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/PostgresStatsStoreIT.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresStatsStore.java
|
||||
git add cameleer-server-app/src/test/java/com/cameleer/server/app/storage/PostgresStatsStoreIT.java
|
||||
git commit -m "feat: implement PostgresStatsStore querying continuous aggregates"
|
||||
```
|
||||
|
||||
### Task 12: Implement PostgresDiagramStore, PostgresUserRepository, PostgresOidcConfigRepository, PostgresMetricsStore
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresDiagramStore.java`
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresUserRepository.java`
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresOidcConfigRepository.java`
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/PostgresMetricsStore.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresDiagramStore.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresUserRepository.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresOidcConfigRepository.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/storage/PostgresMetricsStore.java`
|
||||
|
||||
- [ ] **Step 1: Write failing test for PostgresDiagramStore**
|
||||
|
||||
@@ -1620,10 +1620,10 @@ Straightforward CRUD with `ON CONFLICT (content_hash) DO NOTHING`. Port the SHA-
|
||||
Implements `UserRepository` interface (existing interface in `core.security`, unchanged).
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.core.security.UserInfo;
|
||||
import com.cameleer3.server.core.security.UserRepository;
|
||||
import com.cameleer.server.core.security.UserInfo;
|
||||
import com.cameleer.server.core.security.UserRepository;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.stereotype.Repository;
|
||||
|
||||
@@ -1696,10 +1696,10 @@ public class PostgresUserRepository implements UserRepository {
|
||||
Implements `OidcConfigRepository` interface (existing interface in `core.security`).
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.core.security.OidcConfig;
|
||||
import com.cameleer3.server.core.security.OidcConfigRepository;
|
||||
import com.cameleer.server.core.security.OidcConfig;
|
||||
import com.cameleer.server.core.security.OidcConfigRepository;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
import org.springframework.stereotype.Repository;
|
||||
|
||||
@@ -1760,10 +1760,10 @@ public class PostgresOidcConfigRepository implements OidcConfigRepository {
|
||||
- [ ] **Step 6: Implement PostgresMetricsStore**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.storage;
|
||||
package com.cameleer.server.app.storage;
|
||||
|
||||
import com.cameleer3.server.core.storage.MetricsStore;
|
||||
import com.cameleer3.server.core.storage.model.MetricsSnapshot;
|
||||
import com.cameleer.server.core.storage.MetricsStore;
|
||||
import com.cameleer.server.core.storage.model.MetricsSnapshot;
|
||||
import com.fasterxml.jackson.core.JsonProcessingException;
|
||||
import com.fasterxml.jackson.databind.ObjectMapper;
|
||||
import org.springframework.jdbc.core.JdbcTemplate;
|
||||
@@ -1806,13 +1806,13 @@ public class PostgresMetricsStore implements MetricsStore {
|
||||
|
||||
- [ ] **Step 7: Run all store tests, verify pass**
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest="Postgres*IT" -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest="Postgres*IT" -q`
|
||||
|
||||
- [ ] **Step 8: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/Postgres*.java
|
||||
git add cameleer3-server-app/src/test/java/com/cameleer3/server/app/storage/Postgres*.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/storage/Postgres*.java
|
||||
git add cameleer-server-app/src/test/java/com/cameleer/server/app/storage/Postgres*.java
|
||||
git commit -m "feat: implement PostgresDiagramStore, PostgresUserRepository, PostgresOidcConfigRepository, PostgresMetricsStore"
|
||||
```
|
||||
|
||||
@@ -1823,22 +1823,22 @@ git commit -m "feat: implement PostgresDiagramStore, PostgresUserRepository, Pos
|
||||
### Task 13: Implement OpenSearchIndex
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/OpenSearchConfig.java`
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/search/OpenSearchIndex.java`
|
||||
- Create: `cameleer3-server-app/src/test/java/com/cameleer3/server/app/search/OpenSearchIndexIT.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/config/OpenSearchConfig.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/search/OpenSearchIndex.java`
|
||||
- Create: `cameleer-server-app/src/test/java/com/cameleer/server/app/search/OpenSearchIndexIT.java`
|
||||
|
||||
- [ ] **Step 1: Write failing test**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.search;
|
||||
package com.cameleer.server.app.search;
|
||||
|
||||
import com.cameleer3.server.app.AbstractPostgresIT;
|
||||
import com.cameleer3.server.core.search.ExecutionSummary;
|
||||
import com.cameleer3.server.core.search.SearchRequest;
|
||||
import com.cameleer3.server.core.search.SearchResult;
|
||||
import com.cameleer3.server.core.storage.SearchIndex;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument.ProcessorDoc;
|
||||
import com.cameleer.server.app.AbstractPostgresIT;
|
||||
import com.cameleer.server.core.search.ExecutionSummary;
|
||||
import com.cameleer.server.core.search.SearchRequest;
|
||||
import com.cameleer.server.core.search.SearchResult;
|
||||
import com.cameleer.server.core.storage.SearchIndex;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument.ProcessorDoc;
|
||||
import org.junit.jupiter.api.Test;
|
||||
import org.opensearch.testcontainers.OpensearchContainer;
|
||||
import org.springframework.beans.factory.annotation.Autowired;
|
||||
@@ -1922,7 +1922,7 @@ class OpenSearchIndexIT extends AbstractPostgresIT {
|
||||
- [ ] **Step 2: Create OpenSearchConfig**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.config;
|
||||
package com.cameleer.server.app.config;
|
||||
|
||||
import org.apache.hc.client5.http.impl.nio.PoolingAsyncClientConnectionManagerBuilder;
|
||||
import org.apache.hc.core5.http.HttpHost;
|
||||
@@ -1950,14 +1950,14 @@ public class OpenSearchConfig {
|
||||
- [ ] **Step 3: Implement OpenSearchIndex**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.search;
|
||||
package com.cameleer.server.app.search;
|
||||
|
||||
import com.cameleer3.server.core.search.ExecutionSummary;
|
||||
import com.cameleer3.server.core.search.SearchRequest;
|
||||
import com.cameleer3.server.core.search.SearchResult;
|
||||
import com.cameleer3.server.core.storage.SearchIndex;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument.ProcessorDoc;
|
||||
import com.cameleer.server.core.search.ExecutionSummary;
|
||||
import com.cameleer.server.core.search.SearchRequest;
|
||||
import com.cameleer.server.core.search.SearchResult;
|
||||
import com.cameleer.server.core.storage.SearchIndex;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument.ProcessorDoc;
|
||||
import jakarta.annotation.PostConstruct;
|
||||
import org.opensearch.client.opensearch.OpenSearchClient;
|
||||
import org.opensearch.client.opensearch._types.FieldValue;
|
||||
@@ -2264,34 +2264,34 @@ public class OpenSearchIndex implements SearchIndex {
|
||||
|
||||
- [ ] **Step 4: Run tests to verify they pass**
|
||||
|
||||
Run: `mvn test -pl cameleer3-server-app -Dtest=OpenSearchIndexIT -q`
|
||||
Run: `mvn test -pl cameleer-server-app -Dtest=OpenSearchIndexIT -q`
|
||||
Expected: PASS
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/OpenSearchConfig.java
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/search/OpenSearchIndex.java
|
||||
git add cameleer3-server-app/src/test/java/com/cameleer3/server/app/search/OpenSearchIndexIT.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/config/OpenSearchConfig.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/search/OpenSearchIndex.java
|
||||
git add cameleer-server-app/src/test/java/com/cameleer/server/app/search/OpenSearchIndexIT.java
|
||||
git commit -m "feat: implement OpenSearchIndex with full-text and wildcard search"
|
||||
```
|
||||
|
||||
### Task 14: Implement SearchIndexer (debounced event-driven indexer)
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-core/src/main/java/com/cameleer3/server/core/indexing/SearchIndexer.java`
|
||||
- Create: `cameleer-server-core/src/main/java/com/cameleer/server/core/indexing/SearchIndexer.java`
|
||||
|
||||
- [ ] **Step 1: Implement SearchIndexer**
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.core.indexing;
|
||||
package com.cameleer.server.core.indexing;
|
||||
|
||||
import com.cameleer3.server.core.storage.ExecutionStore;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer3.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import com.cameleer3.server.core.storage.SearchIndex;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer3.server.core.storage.model.ExecutionDocument.ProcessorDoc;
|
||||
import com.cameleer.server.core.storage.ExecutionStore;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ExecutionRecord;
|
||||
import com.cameleer.server.core.storage.ExecutionStore.ProcessorRecord;
|
||||
import com.cameleer.server.core.storage.SearchIndex;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument;
|
||||
import com.cameleer.server.core.storage.model.ExecutionDocument.ProcessorDoc;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
|
||||
@@ -2368,7 +2368,7 @@ public class SearchIndexer {
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-core/src/main/java/com/cameleer3/server/core/indexing/SearchIndexer.java
|
||||
git add cameleer-server-core/src/main/java/com/cameleer/server/core/indexing/SearchIndexer.java
|
||||
git commit -m "feat: implement debounced SearchIndexer for async OpenSearch indexing"
|
||||
```
|
||||
|
||||
@@ -2379,25 +2379,25 @@ git commit -m "feat: implement debounced SearchIndexer for async OpenSearch inde
|
||||
### Task 15: Create bean configuration and wire everything
|
||||
|
||||
**Files:**
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/StorageBeanConfig.java`
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/ingestion/MetricsFlushScheduler.java`
|
||||
- Create: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/retention/RetentionScheduler.java`
|
||||
- Modify: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/SearchBeanConfig.java`
|
||||
- Modify: `cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/IngestionBeanConfig.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/config/StorageBeanConfig.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/ingestion/MetricsFlushScheduler.java`
|
||||
- Create: `cameleer-server-app/src/main/java/com/cameleer/server/app/retention/RetentionScheduler.java`
|
||||
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/config/SearchBeanConfig.java`
|
||||
- Modify: `cameleer-server-app/src/main/java/com/cameleer/server/app/config/IngestionBeanConfig.java`
|
||||
|
||||
- [ ] **Step 1: Create StorageBeanConfig**
|
||||
|
||||
Wire `DetailService`, `SearchIndexer`, `IngestionService` with new store beans:
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.config;
|
||||
package com.cameleer.server.app.config;
|
||||
|
||||
import com.cameleer3.server.core.detail.DetailService;
|
||||
import com.cameleer3.server.core.indexing.SearchIndexer;
|
||||
import com.cameleer3.server.core.ingestion.IngestionService;
|
||||
import com.cameleer3.server.core.ingestion.WriteBuffer;
|
||||
import com.cameleer3.server.core.storage.*;
|
||||
import com.cameleer3.server.core.storage.model.MetricsSnapshot;
|
||||
import com.cameleer.server.core.detail.DetailService;
|
||||
import com.cameleer.server.core.indexing.SearchIndexer;
|
||||
import com.cameleer.server.core.ingestion.IngestionService;
|
||||
import com.cameleer.server.core.ingestion.WriteBuffer;
|
||||
import com.cameleer.server.core.storage.*;
|
||||
import com.cameleer.server.core.storage.model.MetricsSnapshot;
|
||||
import org.springframework.beans.factory.annotation.Value;
|
||||
import org.springframework.context.annotation.Bean;
|
||||
import org.springframework.context.annotation.Configuration;
|
||||
@@ -2453,15 +2453,15 @@ public WriteBuffer<MetricsSnapshot> metricsBuffer(IngestionConfig config) {
|
||||
|
||||
- [ ] **Step 4: Create MetricsFlushScheduler**
|
||||
|
||||
Create `cameleer3-server-app/src/main/java/com/cameleer3/server/app/ingestion/MetricsFlushScheduler.java`:
|
||||
Create `cameleer-server-app/src/main/java/com/cameleer/server/app/ingestion/MetricsFlushScheduler.java`:
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.ingestion;
|
||||
package com.cameleer.server.app.ingestion;
|
||||
|
||||
import com.cameleer3.server.app.config.IngestionConfig;
|
||||
import com.cameleer3.server.core.ingestion.WriteBuffer;
|
||||
import com.cameleer3.server.core.storage.MetricsStore;
|
||||
import com.cameleer3.server.core.storage.model.MetricsSnapshot;
|
||||
import com.cameleer.server.app.config.IngestionConfig;
|
||||
import com.cameleer.server.core.ingestion.WriteBuffer;
|
||||
import com.cameleer.server.core.storage.MetricsStore;
|
||||
import com.cameleer.server.core.storage.model.MetricsSnapshot;
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
import org.springframework.context.SmartLifecycle;
|
||||
@@ -2519,10 +2519,10 @@ public class MetricsFlushScheduler implements SmartLifecycle {
|
||||
|
||||
- [ ] **Step 5: Create RetentionScheduler**
|
||||
|
||||
Create `cameleer3-server-app/src/main/java/com/cameleer3/server/app/retention/RetentionScheduler.java`:
|
||||
Create `cameleer-server-app/src/main/java/com/cameleer/server/app/retention/RetentionScheduler.java`:
|
||||
|
||||
```java
|
||||
package com.cameleer3.server.app.retention;
|
||||
package com.cameleer.server.app.retention;
|
||||
|
||||
import org.slf4j.Logger;
|
||||
import org.slf4j.LoggerFactory;
|
||||
@@ -2575,11 +2575,11 @@ public class RetentionScheduler {
|
||||
- [ ] **Step 6: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/StorageBeanConfig.java
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/SearchBeanConfig.java
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/IngestionBeanConfig.java
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/ingestion/MetricsFlushScheduler.java
|
||||
git add cameleer3-server-app/src/main/java/com/cameleer3/server/app/retention/RetentionScheduler.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/config/StorageBeanConfig.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/config/SearchBeanConfig.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/config/IngestionBeanConfig.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/ingestion/MetricsFlushScheduler.java
|
||||
git add cameleer-server-app/src/main/java/com/cameleer/server/app/retention/RetentionScheduler.java
|
||||
git commit -m "feat: wire new storage beans, add MetricsFlushScheduler and RetentionScheduler"
|
||||
```
|
||||
|
||||
@@ -2591,10 +2591,10 @@ git commit -m "feat: wire new storage beans, add MetricsFlushScheduler and Reten
|
||||
- [ ] **Step 1: Delete ClickHouse implementations**
|
||||
|
||||
```bash
|
||||
rm cameleer3-server-app/src/main/java/com/cameleer3/server/app/storage/ClickHouse*.java
|
||||
rm cameleer3-server-app/src/main/java/com/cameleer3/server/app/search/ClickHouseSearchEngine.java
|
||||
rm cameleer3-server-app/src/main/java/com/cameleer3/server/app/ingestion/ClickHouseFlushScheduler.java
|
||||
rm cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/ClickHouseConfig.java
|
||||
rm cameleer-server-app/src/main/java/com/cameleer/server/app/storage/ClickHouse*.java
|
||||
rm cameleer-server-app/src/main/java/com/cameleer/server/app/search/ClickHouseSearchEngine.java
|
||||
rm cameleer-server-app/src/main/java/com/cameleer/server/app/ingestion/ClickHouseFlushScheduler.java
|
||||
rm cameleer-server-app/src/main/java/com/cameleer/server/app/config/ClickHouseConfig.java
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Delete old core interfaces replaced by new ones**
|
||||
@@ -2602,23 +2602,23 @@ rm cameleer3-server-app/src/main/java/com/cameleer3/server/app/config/ClickHouse
|
||||
`UserRepository` and `OidcConfigRepository` in `core.security` are **kept** — the new Postgres implementations implement them. Only interfaces replaced by new storage interfaces are deleted.
|
||||
|
||||
```bash
|
||||
rm cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/ExecutionRepository.java
|
||||
rm cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/DiagramRepository.java
|
||||
rm cameleer3-server-core/src/main/java/com/cameleer3/server/core/storage/MetricsRepository.java
|
||||
rm cameleer3-server-core/src/main/java/com/cameleer3/server/core/search/SearchEngine.java
|
||||
rm cameleer3-server-core/src/main/java/com/cameleer3/server/core/detail/RawExecutionRow.java
|
||||
rm cameleer-server-core/src/main/java/com/cameleer/server/core/storage/ExecutionRepository.java
|
||||
rm cameleer-server-core/src/main/java/com/cameleer/server/core/storage/DiagramRepository.java
|
||||
rm cameleer-server-core/src/main/java/com/cameleer/server/core/storage/MetricsRepository.java
|
||||
rm cameleer-server-core/src/main/java/com/cameleer/server/core/search/SearchEngine.java
|
||||
rm cameleer-server-core/src/main/java/com/cameleer/server/core/detail/RawExecutionRow.java
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Delete ClickHouse SQL migrations**
|
||||
|
||||
```bash
|
||||
rm -r cameleer3-server-app/src/main/resources/clickhouse/
|
||||
rm -r cameleer-server-app/src/main/resources/clickhouse/
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Delete old test base class**
|
||||
|
||||
```bash
|
||||
rm cameleer3-server-app/src/test/java/com/cameleer3/server/app/AbstractClickHouseIT.java
|
||||
rm cameleer-server-app/src/test/java/com/cameleer/server/app/AbstractClickHouseIT.java
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Fix compilation errors in specific files**
|
||||
@@ -2651,7 +2651,7 @@ git commit -m "refactor: remove all ClickHouse code, old interfaces, and SQL mig
|
||||
### Task 17: Update existing integration tests
|
||||
|
||||
**Files:**
|
||||
- Modify: all IT files under `cameleer3-server-app/src/test/`
|
||||
- Modify: all IT files under `cameleer-server-app/src/test/`
|
||||
|
||||
- [ ] **Step 1: Update all ITs to extend AbstractPostgresIT**
|
||||
|
||||
@@ -2689,7 +2689,7 @@ git commit -m "test: migrate all integration tests from ClickHouse to PostgreSQL
|
||||
|
||||
- [ ] **Step 1: Update Dockerfile**
|
||||
|
||||
No JDBC driver changes needed in Dockerfile (drivers are in the fat JAR). Just verify the `REGISTRY_TOKEN` build arg still works for `cameleer3-common` resolution.
|
||||
No JDBC driver changes needed in Dockerfile (drivers are in the fat JAR). Just verify the `REGISTRY_TOKEN` build arg still works for `cameleer-common` resolution.
|
||||
|
||||
- [ ] **Step 2: Update K8s manifests**
|
||||
|
||||
@@ -2717,7 +2717,7 @@ Changes needed:
|
||||
- Docker build: no ClickHouse references in the image (it's a fat JAR, driver is bundled)
|
||||
- Deploy step: update K8s secret names from `clickhouse-credentials` to `postgres-credentials`
|
||||
- Deploy step: add OpenSearch deployment manifests
|
||||
- Verify `REGISTRY_TOKEN` build arg still works for `cameleer3-common`
|
||||
- Verify `REGISTRY_TOKEN` build arg still works for `cameleer-common`
|
||||
- Integration tests still skipped in CI (`-DskipITs`) — Testcontainers needs Docker-in-Docker
|
||||
|
||||
- [ ] **Step 2: Commit**
|
||||
@@ -2738,7 +2738,7 @@ Run: `mvn spring-boot:run` (briefly, to generate spec), then fetch `/api/v1/api-
|
||||
- [ ] **Step 2: Commit**
|
||||
|
||||
```bash
|
||||
git add cameleer3-server-app/src/main/resources/static/openapi.json
|
||||
git add cameleer-server-app/src/main/resources/static/openapi.json
|
||||
git commit -m "docs: regenerate openapi.json after storage layer refactor"
|
||||
```
|
||||
|
||||
|
||||
Reference in New Issue
Block a user