Skip to main content

verify_speculation

Function verify_speculation 

Source
pub fn verify_speculation(
    spec: Speculation<'_>,
    rng: &mut dyn RngCore,
) -> Result<SpeculationOutcome>
Expand description

Execute the DeepMind speculative-sampling accept/reject loop.

Per-draft decision (draft i, i = 0..N):

  • Let p_T = target prob of draft[i] at position i.
  • Let p_D = draft prob of draft[i] at position i.
  • Accept with probability min(1, p_T / p_D).
  • If rejected, sample a replacement from the residual (p_T - p_D)+.

If every draft is accepted, sample a bonus token from target_logits[N].