Struct RequestQueue 

pub struct RequestQueue { /* private fields */ }

Request queue for continuous batching.

Accumulates incoming requests and drains them in configurable batch sizes for efficient processing.

Implementations

impl RequestQueue

pub fn new(max_size: usize) -> Self

Create a new request queue with the given maximum capacity.

pub fn push(&mut self, request: BatchRequest) -> Result<(), RuntimeError>

Push a new request onto the queue.

Returns an error if the queue is full.

pub fn drain_batch(&mut self, batch_size: usize) -> Vec<BatchRequest>

Drain up to batch_size requests from the front of the queue.

Returns the drained requests in FIFO order.

pub fn len(&self) -> usize

Number of pending requests.

pub fn is_empty(&self) -> bool

Whether the queue is empty.

pub fn is_full(&self) -> bool

Whether the queue is at capacity.

pub fn capacity(&self) -> usize

Maximum queue capacity.