Struct SpeculativeDecoderConfig 

pub struct SpeculativeDecoderConfig {
    pub k: usize,
    pub cost_ratio: f32,
    pub stop_on_eos: bool,
    pub eos_token: Option<TokenId>,
}

Configuration for the speculative decoder.

Fields

k: usize

Number of draft tokens to propose per round (default 4).

cost_ratio: f32

Cost ratio c_draft / c_target for speedup modeling (default 0.125).

stop_on_eos: bool

If true, the engine halts the generation loop on the first eos_token it emits.

eos_token: Option<TokenId>

Optional end-of-sequence token id (ignored unless stop_on_eos).

Implementations

impl SpeculativeDecoderConfig

pub fn with_k(self, k: usize) -> Self

Convenience builder: set draft depth.

pub fn with_cost_ratio(self, r: f32) -> Self

Convenience builder: set cost ratio for the speedup estimate.

pub fn with_eos(self, eos: TokenId) -> Self

Convenience builder: attach an EOS token and enable early stopping.

pub fn validate(&self) -> SpeculativeDecodingResult<()>

Validate the configuration, returning an SpeculativeDecodingError if any invariant is violated.

Trait Implementations

impl Clone for SpeculativeDecoderConfig

fn clone(&self) -> SpeculativeDecoderConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SpeculativeDecoderConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SpeculativeDecoderConfig

fn default() -> Self

Returns the “default value” for a type.

impl PartialEq for SpeculativeDecoderConfig

fn eq(&self, other: &SpeculativeDecoderConfig) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for SpeculativeDecoderConfig