Enum RuntimeError 

pub enum RuntimeError {
    ModelNotLoaded,
    TokenizerNotAvailable,
    TokenizerError {
        message: String,
    },
    SamplingError {
        message: String,
    },
    KvCacheFull {
        max_ctx: usize,
    },
    ModelLoadError {
        message: String,
    },
    Cancelled,
    Arch(ArchError),
    Gguf(GgufError),
    Quant(QuantError),
    Io(Error),
    Grammar(GrammarError),
    AttentionError {
        message: String,
    },
    SnapshotIncompatible {
        detail: String,
    },
    ModelFingerprintMismatch {
        expected: String,
        found: String,
        detail: String,
    },
    OffloadEof {
        offset: u64,
        needed: usize,
        available: usize,
    },
    TensorNotFound(String),
    LockPoisoned,
    SpecSnapshotIncompatible(String),
    EmptySequence,
}

Errors that can occur during inference.

Variants

ModelNotLoaded

No model has been loaded yet.

TokenizerNotAvailable

Tokenizer is not available because neither tokenizer-wasm nor tokenizer-onig feature is enabled.

Enable the tokenizer-wasm feature (default, pure Rust) to use the HuggingFace tokenizers library.

TokenizerError

Tokenizer initialization or encoding/decoding failed.

Fields

message: String

Description of the tokenizer error.

SamplingError

Sampling operation failed.

Fields

message: String

Description of the sampling error.

KvCacheFull

KV cache has reached its maximum capacity.

Fields

max_ctx: usize

Maximum context length supported.

ModelLoadError

Model file could not be loaded.

Fields

message: String

Description of the loading error.

Cancelled

Generation was interrupted or cancelled.

Arch(ArchError)

Error propagated from architecture layer.

Gguf(GgufError)

Error propagated from GGUF parser.

Quant(QuantError)

Error propagated from quantization kernel.

Io(Error)

I/O error during model loading.

Grammar(GrammarError)

Grammar error from GBNF constrained sampling.

AttentionError

Attention computation error.

Fields

message: String

Description of the attention error.

SnapshotIncompatible

Snapshot format version is incompatible with this runtime.

Fields

detail: String

Details about the incompatibility.

ModelFingerprintMismatch

Model fingerprint in snapshot does not match the file on disk.

Fields

expected: String

The fingerprint expected (from snapshot).

found: String

The fingerprint found (computed from disk).

detail: String

Additional detail about the mismatch.

OffloadEof

Offload pager read past end of backing store.

Fields

offset: u64

Byte offset at which the read was attempted.

needed: usize

Number of bytes requested.

available: usize

Number of bytes available from offset to end.

TensorNotFound(String)

A tensor name was not found in the weight offset map.

LockPoisoned

An internal RwLock or Mutex was poisoned.

SpecSnapshotIncompatible(String)

Speculative-engine snapshot format is incompatible with this runtime.

EmptySequence

Cannot pool an empty sequence (seq_len = 0).

Trait Implementations

impl Debug for RuntimeError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for RuntimeError

fn fmt(&self, __formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Error for RuntimeError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0:

use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0:

replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.

impl From<ArchError> for RuntimeError

fn from(source: ArchError) -> Self

Converts to this type from the input type.

impl From<Error> for RuntimeError

fn from(source: Error) -> Self

Converts to this type from the input type.

impl From<GgufError> for RuntimeError

fn from(source: GgufError) -> Self

Converts to this type from the input type.

impl From<GrammarError> for RuntimeError

fn from(source: GrammarError) -> Self

Converts to this type from the input type.

impl From<QuantError> for RuntimeError

fn from(source: QuantError) -> Self

Converts to this type from the input type.

impl From<RuntimeError> for RewindError

fn from(source: RuntimeError) -> Self

Converts to this type from the input type.