Skip to main content

forward_with_kv_context_persistent

fuel_tensor::persistent_decode

Function forward_with_kv_context_persistent 

Source 
pub fn forward_with_kv_context_persistent<M: DecodeBackbone + ?Sized>(
    model: &M,
    tokens: &[u32],
    cache: &mut KvCache,
    ctx: &mut InferenceContext,
    session: &mut Option<DecodeSession>,
    rope_inv_freq: Option<&[f64]>,
) -> Result<Vec<f32>>
Expand description

The persistent decode entry point, shared by every family on this path.

Three arms, exactly as the pre-GAP-029 LlamaModel sibling had them: seq != 1 falls back to D1 (prefill / spec-decode verification), a missing or stale session takes the D2 build, and a live session takes the rebind driver — with a TopologyChanged dropping the session and serving this one token through D1.