Skip to main content

DEFAULT_EMBED_TIMEOUT

Constant DEFAULT_EMBED_TIMEOUT 

Source
pub const DEFAULT_EMBED_TIMEOUT: Duration;
Expand description

How long one embeddings request may take before it is abandoned.

Ten seconds: long enough for a local server to load a model it had unloaded (seconds, once) and for a remote provider to answer a large batch, short enough that a provider which accepted the connection and then stopped talking does not hold a caller for minutes.