Struct SpeculativeRunner 

pub struct SpeculativeRunner<'a> {
    pub draft: &'a dyn ModelExecutor,
    pub target: &'a dyn ModelExecutor,
    pub tensor_factory: Arc<dyn TensorFactory>,
    pub cfg: SpeculativeDecodingConfig,
}

Drives one round of speculative decoding against a (draft, target) pair of ModelExecutors. Owns neither executor — the engine keeps them and hands references per call.

Usage per decode iteration:

• Caller hands the runner the last sampled token for this request, plus the draft & target KV cache handles.
• step() runs N draft decodes, N+1 target decodes (sequentially for now — performance gain lands once the executors grow multi-position decode support), then applies verify_speculation.
• Returns the list of newly accepted tokens plus the updated KV handles for draft and target (caller installs them on the sequence state for the next iteration).

Fields

draft: &'a dyn ModelExecutor

target: &'a dyn ModelExecutor

tensor_factory: Arc<dyn TensorFactory>

cfg: SpeculativeDecodingConfig

Implementations

impl<'a> SpeculativeRunner<'a>

pub async fn step( &self, last_token: TokenId, draft_kv: Arc<dyn KvCacheHandle>, target_kv: Arc<dyn KvCacheHandle>, rng: &mut (dyn RngCore + Send), ) -> Result<SpeculativeStepOutcome>

Run one draft+verify cycle. last_token is the token that was sampled from the previous iteration (or the end of prefill) — it’s the starting point both executors advance from.