Skip to main content

EMBED_TIMEOUT_PER_EXTRA_BATCH_ITEM_SECS

Constant EMBED_TIMEOUT_PER_EXTRA_BATCH_ITEM_SECS 

Source
pub const EMBED_TIMEOUT_PER_EXTRA_BATCH_ITEM_SECS: u64 = 15;
Expand description

Extra timeout, in seconds, granted per item beyond the first in a batched embedding call (GAP-4).

The base budget is the already-configurable embedding.timeout_secs; this is only how that budget scales with batch width, so it takes no key of its own — a second knob governing the same deadline would be two ways to say one thing.