Skip to main content

Module batch_insert

Module batch_insert 

Source
Expand description

Issue #582 — Analytics slice 4: BatchInsertEndpoint HTTP support.

Owns the config + in-memory idempotency cache that the POST /collections/:name/batch handler uses to:

  • reject batches exceeding red.batch.max_rows with 413 BatchTooLarge before any storage work,
  • deduplicate requests carrying the same Idempotency-Key within red.batch.idempotency_window_secs, returning the previously cached response body verbatim,
  • report row-level validation failures by index so the caller can pinpoint the row that broke the all-or-nothing commit.

The actual commit + AnalyticsSchemaRegistry validation runs in the handler (server::handlers_entity::handle_batch_insert). This module only owns the pieces that need to be unit-testable without booting a full HTTP server.

Structs§

BatchInsertCache
In-memory (collection, idempotency-key) → CachedResponse map. Pruned lazily on every lookup/store rather than via a background task — the working set is bounded by the idempotency_window and a Mutex is cheap enough at the batch-insert call rate.
BatchInsertConfig
Knobs for the batch endpoint. Read from RED_BATCH_* env vars at process start; the red.batch.* config keys in the brief route to the same defaults until the broader config-overlay binding lands.
CachedResponse
A response previously served for a given (collection, idempotency-key). Stored verbatim so a replay returns byte-for-byte what the caller would have seen the first time, even if the underlying state has since drifted.

Enums§

BatchInsertError

Functions§

global_cache
Process-wide cache. The brief states the dedup window is “in-memory per primary” — one process, one cache.