Enum InferenceError 

pub enum InferenceError {
    ModelNotFound(String),
    ModelLoadError(String),
    TokenizationError(String),
    InferenceError(String),
    InvalidInput(String),
    IoError(Error),
    OrtError(String),
    HubError(String),
    ExtractionFailed(String),
    Overloaded {
        active: usize,
        max: usize,
    },
    CandleError(String),
    GgufError(String),
    BinaryQuantError(String),
}

Errors that can occur during inference operations.

Variants

ModelNotFound(String)

Model not found or failed to download

ModelLoadError(String)

Failed to load model weights

TokenizationError(String)

Tokenization error

InferenceError(String)

Inference/forward pass error

InvalidInput(String)

Invalid input

IoError(Error)

IO error

OrtError(String)

ONNX Runtime error

HubError(String)

HuggingFace Hub error

ExtractionFailed(String)

External extraction provider error (EXT-1)

Overloaded

Cross-encoder reranker is at capacity — caller should fall back to unranked results.

Root cause of the LoCoMo SIGTERM (DAK-5893): 8 concurrent bench recall requests saturated the 2-session ONNX pool; the 7th/8th request waited >120s for a mutex slot, triggering client-side timeout + 8 retries = ~19-minute stall per question. Fix: return immediately so the API falls back to unranked results rather than queuing.

Fields

active: usize

max: usize

CandleError(String)

Candle tensor/model error (requires candle feature).

GgufError(String)

GGUF model loading or quantization error (requires candle feature).

BinaryQuantError(String)

Binary quantization error — dimension mismatch or empty input.

Trait Implementations

impl Debug for InferenceError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for InferenceError

fn fmt(&self, __formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Error for InferenceError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0:

use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0:

replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.

impl From<Box<dyn Error + Sync + Send>> for InferenceError

fn from(err: Error) -> Self

Converts to this type from the input type.

impl From<Error> for InferenceError

fn from(source: Error) -> Self

Converts to this type from the input type.

impl From<Error> for InferenceError

fn from(err: Error) -> Self

Converts to this type from the input type.