Files
cameleer-server/.planning/phases/03-agent-registry-sse-push/03-02-PLAN.md

252 lines
15 KiB
Markdown
Raw Normal View History

---
phase: 03-agent-registry-sse-push
plan: 02
type: execute
wave: 2
depends_on: ["03-01"]
files_modified:
- cameleer-server-app/src/main/java/com/cameleer/server/app/agent/SseConnectionManager.java
- cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentSseController.java
- cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentCommandController.java
- cameleer-server-app/src/main/java/com/cameleer/server/app/config/WebConfig.java
- cameleer-server-app/src/test/java/com/cameleer/server/app/controller/AgentSseControllerIT.java
- cameleer-server-app/src/test/java/com/cameleer/server/app/controller/AgentCommandControllerIT.java
autonomous: true
requirements:
- AGNT-04
- AGNT-05
- AGNT-06
- AGNT-07
must_haves:
truths:
- "Registered agent can open SSE stream at GET /api/v1/agents/{id}/events and receive events"
- "Server pushes config-update events to a specific agent's SSE stream via POST /api/v1/agents/{id}/commands"
- "Server pushes deep-trace commands to a specific agent's SSE stream with correlationId in payload"
- "Server pushes replay commands to a specific agent's SSE stream"
- "Server can target commands to all agents in a group via POST /api/v1/agents/groups/{group}/commands"
- "Server can broadcast commands to all live agents via POST /api/v1/agents/commands"
- "SSE stream receives ping keepalive comments every 15 seconds"
- "SSE events include event ID for Last-Event-ID reconnection support (no replay of missed events)"
- "Agent can acknowledge command receipt via POST /api/v1/agents/{id}/commands/{commandId}/ack"
artifacts:
- path: "cameleer-server-app/src/main/java/com/cameleer/server/app/agent/SseConnectionManager.java"
provides: "Per-agent SseEmitter management, event sending, ping keepalive"
- path: "cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentSseController.java"
provides: "GET /{id}/events SSE endpoint"
- path: "cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentCommandController.java"
provides: "POST command endpoints (single, group, broadcast) + ack endpoint"
key_links:
- from: "AgentCommandController"
to: "SseConnectionManager"
via: "sendEvent for command delivery"
pattern: "connectionManager\\.sendEvent"
- from: "AgentCommandController"
to: "AgentRegistryService"
via: "addCommand + findByState/findByGroup"
pattern: "registryService\\.addCommand"
- from: "SseConnectionManager"
to: "AgentEventListener"
via: "implements interface, receives command notifications"
pattern: "implements AgentEventListener"
- from: "AgentSseController"
to: "SseConnectionManager"
via: "connect() returns SseEmitter"
pattern: "connectionManager\\.connect"
---
<objective>
Build SSE connection management and command push infrastructure for real-time agent communication.
Purpose: The server needs to push config-update, deep-trace, and replay commands to connected agents in real time via Server-Sent Events. This completes the bidirectional communication channel (agents POST data to server, server pushes commands via SSE).
Output: SseConnectionManager, SSE endpoint, command controller (single/group/broadcast targeting), command acknowledgement, ping keepalive, Last-Event-ID support, integration tests.
</objective>
<execution_context>
@C:/Users/Hendrik/.claude/get-shit-done/workflows/execute-plan.md
@C:/Users/Hendrik/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/PROJECT.md
@.planning/ROADMAP.md
@.planning/STATE.md
@.planning/phases/03-agent-registry-sse-push/03-CONTEXT.md
@.planning/phases/03-agent-registry-sse-push/03-RESEARCH.md
@.planning/phases/03-agent-registry-sse-push/03-01-SUMMARY.md
@cameleer-server-app/src/main/java/com/cameleer/server/app/config/WebConfig.java
@cameleer-server-app/src/main/resources/application.yml
@cameleer-server-app/src/test/java/com/cameleer/server/app/AbstractClickHouseIT.java
<interfaces>
<!-- From Plan 01 (must exist before this plan executes) -->
From cameleer-server-core/.../agent/AgentInfo.java:
```java
// Record or class with fields:
// id, name, group, version, routeIds, capabilities, state, registeredAt, lastHeartbeat, staleTransitionTime
// Methods: withState(), withLastHeartbeat(), etc.
```
From cameleer-server-core/.../agent/AgentState.java:
```java
public enum AgentState { LIVE, STALE, DEAD }
```
From cameleer-server-core/.../agent/CommandType.java:
```java
public enum CommandType { CONFIG_UPDATE, DEEP_TRACE, REPLAY }
```
From cameleer-server-core/.../agent/CommandStatus.java:
```java
public enum CommandStatus { PENDING, DELIVERED, ACKNOWLEDGED, EXPIRED }
```
From cameleer-server-core/.../agent/AgentCommand.java:
```java
// Record: id (UUID string), type (CommandType), payload (String JSON), targetAgentId, createdAt, status
// Method: withStatus()
```
From cameleer-server-core/.../agent/AgentEventListener.java:
```java
public interface AgentEventListener {
void onCommandReady(String agentId, AgentCommand command);
}
```
From cameleer-server-core/.../agent/AgentRegistryService.java:
```java
// Key methods:
// register(id, name, group, version, routeIds, capabilities) -> AgentInfo
// heartbeat(id) -> boolean
// findById(id) -> AgentInfo
// findAll() -> List<AgentInfo>
// findByState(state) -> List<AgentInfo>
// addCommand(agentId, type, payload) -> AgentCommand
// acknowledgeCommand(agentId, commandId) -> boolean
// markDelivered(agentId, commandId) -> void
// setEventListener(listener) -> void
```
From cameleer-server-app/.../config/AgentRegistryConfig.java:
```java
// @ConfigurationProperties(prefix = "agent-registry")
// getPingIntervalMs(), getCommandExpiryMs(), etc.
```
</interfaces>
</context>
<tasks>
<task type="auto">
<name>Task 1: SseConnectionManager, SSE controller, and command controller</name>
<files>
cameleer-server-app/src/main/java/com/cameleer/server/app/agent/SseConnectionManager.java,
cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentSseController.java,
cameleer-server-app/src/main/java/com/cameleer/server/app/controller/AgentCommandController.java,
cameleer-server-app/src/main/java/com/cameleer/server/app/config/AgentRegistryBeanConfig.java,
cameleer-server-app/src/main/java/com/cameleer/server/app/config/WebConfig.java
</files>
<action>
Build the SSE infrastructure and command delivery system:
1. **SseConnectionManager** (@Component, implements AgentEventListener):
- ConcurrentHashMap<String, SseEmitter> emitters for per-agent connections
- Inject AgentRegistryConfig for ping interval, inject AgentRegistryService (call setEventListener(this) in @PostConstruct)
- `connect(String agentId)`: Create SseEmitter(Long.MAX_VALUE). Register onCompletion/onTimeout/onError callbacks that remove the emitter ONLY if the current map value is the same instance (reference equality via == check to avoid Pitfall 3 from research). Replace existing emitter with put(), complete() old one if exists. Return new emitter.
- `sendEvent(String agentId, String eventId, String eventType, Object data)`: Get emitter from map, send SseEmitter.event().id(eventId).name(eventType).data(data, MediaType.APPLICATION_JSON). Catch IOException, remove emitter, return false. Return true on success.
- `sendPingToAll()`: Iterate emitters, send comment("ping") to each. Remove on IOException.
- `isConnected(String agentId)`: Check if emitter exists in map.
- `onCommandReady(String agentId, AgentCommand command)`: Attempt sendEvent with command.id() as eventId, command.type().name().toLowerCase().replace('_', '-') as event name (config-update, deep-trace, replay), command.payload() as data. If successful, call registryService.markDelivered(agentId, command.id()). If agent not connected, command stays PENDING (caller can re-send or it expires).
- @Scheduled(fixedDelayString = "${agent-registry.ping-interval-ms:15000}") pingAll(): calls sendPingToAll()
2. **Update AgentRegistryBeanConfig**: After creating AgentRegistryService bean, the SseConnectionManager (auto-scanned as @Component) will call setEventListener in @PostConstruct. No change needed in bean config if SseConnectionManager handles it. BUT -- to avoid circular dependency, SseConnectionManager should inject AgentRegistryService and call setEventListener(this) in @PostConstruct.
3. **AgentSseController** (@RestController, @RequestMapping("/api/v1/agents")):
- Inject SseConnectionManager, AgentRegistryService
- `GET /{id}/events` (produces TEXT_EVENT_STREAM_VALUE): Check agent exists via registryService.findById(id). If null, return 404 (throw ResponseStatusException). Read Last-Event-ID header (optional) -- log it at debug level but do NOT replay missed events (per locked decision). Call connectionManager.connect(id), return the SseEmitter.
- Add @Tag(name = "Agent SSE") and @Operation annotations.
4. **AgentCommandController** (@RestController, @RequestMapping("/api/v1/agents")):
- Inject AgentRegistryService, SseConnectionManager, ObjectMapper
- `POST /{id}/commands`: Accept raw String body. Parse JSON: { "type": "config-update|deep-trace|replay", "payload": {...} }. Map type string to CommandType enum (config-update -> CONFIG_UPDATE, deep-trace -> DEEP_TRACE, replay -> REPLAY). Call registryService.addCommand(id, type, payloadJsonString). The AgentEventListener.onCommandReady in SseConnectionManager handles delivery. Return 202 with { commandId, status: "PENDING" or "DELIVERED" depending on whether agent is connected }.
- `POST /groups/{group}/commands`: Same body parsing. Find all LIVE agents in group via registryService.findAll() filtered by group. For each, call registryService.addCommand(). Return 202 with { commandIds: [...], targetCount: N }.
- `POST /commands`: Broadcast to all LIVE agents. Same pattern as group but uses registryService.findByState(LIVE). Return 202 with count.
- `POST /{id}/commands/{commandId}/ack`: Call registryService.acknowledgeCommand(id, commandId). Return 200 if true, 404 if false.
- Add @Tag(name = "Agent Commands") and @Operation annotations.
5. **Update WebConfig**: The SSE endpoint GET /api/v1/agents/{id}/events is already covered by the interceptor pattern "/api/v1/agents/**". Agents send the protocol version header on all requests (per research recommendation), so no exclusion needed. However, if the SSE GET causes issues because browsers/clients may not easily add custom headers to EventSource, add the SSE events path to excludePathPatterns: `/api/v1/agents/*/events`. This is a practical consideration -- add the exclusion to be safe.
</action>
<verify>
<automated>mvn compile -pl cameleer-server-core,cameleer-server-app</automated>
</verify>
<done>SseConnectionManager, AgentSseController, and AgentCommandController compile. SSE endpoint returns SseEmitter. Command endpoints accept type/payload and deliver via SSE. Ping keepalive scheduled. WebConfig updated if needed.</done>
</task>
<task type="auto">
<name>Task 2: Integration tests for SSE, commands, and full flow</name>
<files>
cameleer-server-app/src/test/java/com/cameleer/server/app/controller/AgentSseControllerIT.java,
cameleer-server-app/src/test/java/com/cameleer/server/app/controller/AgentCommandControllerIT.java
</files>
<action>
Write integration tests covering SSE connection, command delivery, ping, and acknowledgement:
**SSE Test Strategy** (from RESEARCH.md): Testing SSE with TestRestTemplate is non-trivial. Use one of these approaches:
- Option A (preferred): Use raw HttpURLConnection or java.net.http.HttpClient to open the SSE stream in a separate thread, read lines, and assert event format.
- Option B: Use Spring WebClient (from spring-boot-starter-webflux test dependency) -- BUT do not add webflux as a main dependency, only as test scope if needed.
- Option C: Test at the service layer by calling SseConnectionManager.connect() directly, then sendEvent(), and reading from the SseEmitter via a custom handler.
Recommend Option A (HttpClient) for true end-to-end testing without adding dependencies.
1. **AgentSseControllerIT** (extends AbstractClickHouseIT):
- Test SSE connect for registered agent: Register agent, open GET /{id}/events with Accept: text/event-stream. Assert 200 and content-type is text/event-stream.
- Test SSE connect for unknown agent: GET /unknown-id/events, assert 404.
- Test config-update delivery: Register agent, open SSE stream (background thread), POST /{id}/commands with {"type":"config-update","payload":{"key":"value"}}. Use Awaitility to assert SSE stream received event with name "config-update" and correct data.
- Test deep-trace delivery: Same pattern with {"type":"deep-trace","payload":{"correlationId":"test-123"}}.
- Test replay delivery: Same pattern with {"type":"replay","payload":{"exchangeId":"ex-456"}}.
- Test ping keepalive: Open SSE stream, wait for ping comment (may need to set ping interval low in test config or use Awaitility with timeout). Assert ":ping" comment received.
- Test Last-Event-ID header: Open SSE with Last-Event-ID header set. Assert connection succeeds (no replay, just acknowledges).
- All POST requests include X-Cameleer-Protocol-Version:1 header. SSE GET may need the header excluded in WebConfig (test will reveal if this is an issue).
- Use Awaitility with ignoreExceptions() for async assertions (established pattern).
2. **AgentCommandControllerIT** (extends AbstractClickHouseIT):
- Test single agent command: Register agent, POST /{id}/commands, assert 202 with commandId.
- Test group command: Register 2 agents in same group, POST /groups/{group}/commands, assert 202 with targetCount=2.
- Test broadcast command: Register 3 agents, POST /commands, assert 202 with count of LIVE agents.
- Test command ack: Send command, POST /{id}/commands/{commandId}/ack, assert 200.
- Test ack unknown command: POST /{id}/commands/unknown-id/ack, assert 404.
- Test command to unregistered agent: POST /nonexistent/commands, assert 404.
**Test configuration**: If ping interval needs to be shorter for tests, add to test application.yml or use @TestPropertySource with agent-registry.ping-interval-ms=1000.
</action>
<verify>
<automated>mvn test -pl cameleer-server-core,cameleer-server-app -Dtest="Agent*"</automated>
</verify>
<done>All SSE integration tests pass: connect/disconnect, config-update/deep-trace/replay delivery via SSE, ping keepalive received, Last-Event-ID accepted, command targeting (single/group/broadcast), command acknowledgement. mvn clean verify passes with all existing tests still green.</done>
</task>
</tasks>
<verification>
mvn clean verify -- full suite green (all Phase 1, 2, and 3 tests pass)
</verification>
<success_criteria>
- SSE endpoint returns working event stream for registered agents
- config-update, deep-trace, and replay commands delivered via SSE in real time
- Group and broadcast targeting works correctly
- Ping keepalive sent every 15 seconds
- Last-Event-ID header accepted (no replay, per decision)
- Command acknowledgement endpoint works
- All integration tests pass
- Full mvn clean verify passes
</success_criteria>
<output>
After completion, create `.planning/phases/03-agent-registry-sse-push/03-02-SUMMARY.md`
</output>