Skip to main content

REMEMBER_MAX_CONTROLLED_BATCH_CHUNKS

Constant REMEMBER_MAX_CONTROLLED_BATCH_CHUNKS 

Source
pub const REMEMBER_MAX_CONTROLLED_BATCH_CHUNKS: usize = 4;
Expand description

Ceiling on chunks per controlled micro-batch in remember.

The fastembed runtime uses BatchLongest padding, so oversized batches amplify the cost of the longest chunk. This ceiling keeps batches small even when chunks are short.