Enum SpeculativeDecodingError 

pub enum SpeculativeDecodingError {
    VocabMismatch {
        draft: usize,
        target: usize,
    },
    DistributionWidthMismatch {
        expected: usize,
        got: usize,
    },
    DraftShapeMismatch {
        tokens: usize,
        logprobs: usize,
        distributions: usize,
    },
    TargetShapeMismatch {
        expected: usize,
        got: usize,
    },
    InvalidConfig(String),
    EmptyPrefix,
    TokenOutOfRange {
        token: usize,
        vocab_size: usize,
    },
    DegenerateDistribution,
    ModelError(String),
}

Errors that can be raised during speculative decoding.

Variants

VocabMismatch

A draft and target model disagreed on vocab_size.

Fields

draft: usize

target: usize

DistributionWidthMismatch

A distribution row had the wrong width.

Fields

expected: usize

got: usize

DraftShapeMismatch

The draft model returned a proposal whose tokens / token_logprobs / distributions vectors disagreed in length.

Fields

tokens: usize

logprobs: usize

distributions: usize

TargetShapeMismatch

The target model returned the wrong number of distribution rows.

Fields

expected: usize

got: usize

InvalidConfig(String)

The caller asked for k == 0 draft tokens (or similar degenerate configuration).

EmptyPrefix

The caller supplied an empty prefix to generate even though the configured models require at least one bos/sos token.

TokenOutOfRange

A token id produced by a model was outside the configured vocabulary.

Fields

token: usize

vocab_size: usize

DegenerateDistribution

A probability row collapsed to zero mass — typically because max(0, p_target - p_draft) was identically zero on every index, which happens iff p_target == p_draft everywhere. We fall back to the raw target distribution in that case; this variant is reserved for cases where even that is degenerate.

ModelError(String)

A model implementation returned a descriptive error.

Trait Implementations

impl Clone for SpeculativeDecodingError

fn clone(&self) -> SpeculativeDecodingError

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SpeculativeDecodingError

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for SpeculativeDecodingError

fn fmt(&self, __formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Error for SpeculativeDecodingError

fn source(&self) -> Option<&(dyn Error + 'static)>

Returns the lower-level source of this error, if any.

fn description(&self) -> &str

👎Deprecated since 1.42.0:

use the Display impl or to_string()

fn cause(&self) -> Option<&dyn Error>

👎Deprecated since 1.33.0:

replaced by Error::source, which can support downcasting

fn provide<'a>(&'a self, request: &mut Request<'a>)

🔬This is a nightly-only experimental API. (error_generic_member_access)

Provides type-based access to context intended for error reports.

impl From<SpeculativeDecodingError> for TrustformerError

fn from(err: SpeculativeDecodingError) -> Self

Converts to this type from the input type.

impl PartialEq for SpeculativeDecodingError

fn eq(&self, other: &SpeculativeDecodingError) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for SpeculativeDecodingError