cameleer/cameleer-server
1
0
Fork 0
Code Issues 42 Pull Requests Actions Packages Projects Releases Wiki Activity

INGST-04: Batch-buffered writes to ClickHouse #4

New Issue
Closed
opened 2026-03-11 11:13:12 +01:00 by claude · 1 comment
claude commented 2026-03-11 11:13:12 +01:00
Owner
Copy Link

Ingestion uses an in-memory batch buffer with configurable flush interval and batch size for ClickHouse writes. ClickHouse requires batch inserts (1K-10K rows) — row-by-row inserts cause "too many parts" errors.

Category: Data Ingestion
REQ-ID: INGST-04

Ingestion uses an in-memory batch buffer with configurable flush interval and batch size for ClickHouse writes. ClickHouse requires batch inserts (1K-10K rows) — row-by-row inserts cause "too many parts" errors. **Category:** Data Ingestion **REQ-ID:** INGST-04
claude commented 2026-03-11 21:21:04 +01:00
Author
Owner
Copy Link

Implemented in Phase 1. WriteBuffer accumulates records in-memory and ClickHouseFlushScheduler flushes to ClickHouse on a configurable interval (default 1000ms) or when batch size (default 5000) is reached. Key files: WriteBuffer.java, ClickHouseFlushScheduler.java.

Implemented in Phase 1. `WriteBuffer` accumulates records in-memory and `ClickHouseFlushScheduler` flushes to ClickHouse on a configurable interval (default 1000ms) or when batch size (default 5000) is reached. Key files: `WriteBuffer.java`, `ClickHouseFlushScheduler.java`.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
Accessibility (a11y) improvements
 bug
Something isn't working correctly
 cleanup
Code cleanup, dead code removal, refactoring
 epic
Epic: groups related issues
 feature
New functionality
 pmf
Product-market fit: critical for first market offer
 route-diagram
Route diagram page and execution overlay
 security
Authentication, authorization, RBAC
 ui
Frontend / React UI
 ux
UX/usability improvements
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
claude
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: cameleer/cameleer-server#4
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?