Struct AsyncSpecConfig 

pub struct AsyncSpecConfig {
    pub spec_k: usize,
    pub draft_sampler: SamplerConfig,
    pub target_sampler: SamplerConfig,
    pub force_n1: bool,
    pub max_tokens: usize,
}

Configuration for the async speculative decoder.

Fields

spec_k: usize

Number of draft tokens to generate per speculation step (K).

Higher values increase potential throughput but also increase the cost of verification and rollback on divergence. A value of 4–8 is typical.

draft_sampler: SamplerConfig

Sampler configuration applied by the draft engine.

target_sampler: SamplerConfig

Sampler configuration applied by the target engine for verification and residual sampling.

force_n1: bool

Force N=1 verification mode regardless of target model type.

Set this to true when the target model is SSM-based (cannot rewind).

max_tokens: usize

Maximum number of tokens to generate (prompt + output combined).

Trait Implementations

impl Clone for AsyncSpecConfig

fn clone(&self) -> AsyncSpecConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for AsyncSpecConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for AsyncSpecConfig

fn default() -> Self

Returns the “default value” for a type.