Struct EngineBeamAdapter 

pub struct EngineBeamAdapter<'a> { /* private fields */ }

Adapter that wraps crate::engine::InferenceEngine to implement BeamForwardPass.

Each call to forward_tokens resets the KV cache, prefills the prompt tokens, and returns the logits for the last token.

Implementations

impl<'a> EngineBeamAdapter<'a>

pub fn new(engine: &'a mut InferenceEngine) -> Self

Create an adapter over a loaded engine.

Trait Implementations

impl BeamForwardPass for EngineBeamAdapter<'_>

fn forward_tokens(&mut self, tokens: &[u32]) -> RuntimeResult<Vec<f32>>

Run the forward pass on tokens and return raw logits.

fn reset(&mut self)

Reset the internal state (e.g. clear the KV cache) so a fresh forward pass can be run for a different beam.