pub fn accept_or_resample(
target: &[f32],
draft: &DraftDist,
token: usize,
rng: &mut Sampler,
) -> Option<usize>Expand description
The speculative-sampling accept/reject decision for one drafted position.
target is the target model’s final sampling distribution for
this position – what sampling_distribution returns, i.e. after
penalties, temperature, top-k and top-p, because that is the
distribution the non-speculative path would have drawn from.
draft is the distribution token was drawn from.
Returns None when the draft token is accepted, and
Some(replacement) when it is rejected – the replacement is drawn
from the normalised residual max(0, target - draft), which is what
makes the combined procedure’s output distribution equal to
target exactly rather than approximately.
A token with draft(token) == 0.0 violates the DraftDist
contract (it could not have been sampled from draft); it is
accepted, matching the p/q -> infinity limit, rather than
silently biasing the result.