Module batch_processor

Module batch_processor 

Source
Expand description

High-throughput batch processing pipeline

This module provides an optimized batch processing system that combines:

  • SIMD-accelerated JSON parsing for fast deserialization
  • Lock-free sharded queues for parallel ingestion
  • Memory pooling for zero-allocation hot paths

§Performance Target

  • 1M+ events/sec sustained throughput
  • Sub-millisecond p99 latency
  • Linear scalability with CPU cores

Structs§

ArenaBatchBuffer
Arena-backed batch buffer for zero-allocation processing
BatchProcessor
High-throughput batch event processor
BatchProcessorConfig
Configuration for batch processor
BatchProcessorStats
Statistics for batch processing
BatchResult
Result of batch processing operation
RawEventData
Raw event data for batch parsing