Struct InferenceSession 

pub struct InferenceSession<E: InferenceEngine> { /* private fields */ }

One live generation. Constructing it requires a LoadPermit, so a model that has not passed the provenance gate (ADR-006) cannot reach the runtime — the Conformist relationship is enforced in the type system.

Implementations

impl<E: InferenceEngine> InferenceSession<E>

pub fn new( id: SessionId, config: SessionConfig, engine: E, permit: LoadPermit, ) -> Self

pub fn phase(&self) -> Phase

pub fn output(&self) -> &[Token]

pub fn kv_len(&self) -> u32

pub fn config(&self) -> &SessionConfig

pub fn permit(&self) -> LoadPermit

The load permit this session was constructed with — evidence the model passed the provenance gate (ADR-006).

pub fn drain_events(&mut self) -> Vec<EventEnvelope>

Take the buffered domain events (a real build would stream these to the Telemetry subscriber).

pub fn load_prompt(&mut self, ports: &Ports, prompt: &[Token]) -> Result<()>

Compress (optional) → prefill → build KV. Valid only from Initialized.

pub fn generate(&mut self, ports: &Ports, max_tokens: u32) -> Result<StopReason>

Run the decode loop until EOS or max_tokens, deriving the rollback policy from the session’s device tier and safety mode (ADR-005/ADR-012).

pub fn generate_with_policy( &mut self, ports: &Ports, max_tokens: u32, policy: RollbackPolicy, ) -> Result<StopReason>

The checkpointed-rollback safety control loop (ADR-012).

Every step preserves the invariant order grammar mask → safety adjust → sample → commit (ADR-005). When a ChunkGuard is wired and the policy enables guarding, the loop additionally captures a checkpoint at each guard-verified-safe boundary, scores recent output every guard_every tokens, and on a hard-threshold breach rolls the KV cache and the committed output back to the last safe checkpoint — banning the offending token through the grammar mask so the resumed decode necessarily diverges. Rollbacks are bounded by max_rollbacks; on exhaustion — or with no checkpoint (e.g. under memory pressure) — the loop fails closed with a deterministic refusal (StopReason::Stopped).

Termination (EOS or max_tokens) is gated behind a mandatory final guard check: the loop scores the trailing chunk before honouring either stop condition, so no completion is ever returned unscored — including one shorter than guard_every or whose unsafe tail ends in EOS. A final check coincident with a cadence boundary is idempotent (re-scoring identical output yields the same verdict).

pub fn reset(&mut self) -> Result<()>

Reset for a fresh conversation on the same resident weights — the seam that turns a provider from “rebuild the engine every turn” into “load once, reuse” (ADR-018). Clears the session’s KV descriptors / output / prompt and resets the engine’s logical cache; distinct from a safety rollback (which rewinds within a generation).

Returns the engine’s reset_cache error rather than swallowing it: an engine that fails to discard its cache must not be reported as a clean fresh session. On error the session state is left untouched (so an empty session can never desync from a stale engine cache) and the caller is notified — it should drop/rebuild rather than reuse.

Buffered events are preserved (generic semantics): a telemetry consumer may still drain_events after a reset. Turn-level event isolation is the caller’s concern — a provider that reuses one session across turns should drain between turns (see the adapter providers).

reset_cache releases the previous conversation’s KV while keeping the resident weights loaded — that separation of conversation lifecycle from model lifecycle is the point of ADR-018.

pub fn close(&mut self) -> Result<()>

End the current conversation: release its volatile memory — engine KV (via reset_cache), KV descriptors, committed output, retained prompt, and buffered events — while keeping the resident model loaded so the session can serve a new conversation without reloading weights (ADR-018; PRD line 131 “KV caches … cleared on session end”; the AC-4 explicit release).

Takes &mut self, not self: dropping the session would also drop the (expensive) weights, which is the opposite of “load once, reuse.” Instead the engine releases the conversation’s KV in place — for candle’s quantized_qwen2, a position-0 forward overwrites and frees the prior K/V tensors (see its reset_cache). Unlike reset, close also frees the buffers’ capacity and discards buffered events, minimizing the idle footprint. Propagates a reset_cache failure (state untouched on error). To free the weights too, drop the session/provider (ownership).

pub fn consult_relay( &mut self, ports: &Ports, query: &[Token], ) -> Result<Vec<Token>>

Consult the opt-in LAN relay. Hard-fails with EdgeError::AirGapViolation unless hybrid_mode is enabled AND a relay is wired (ADR-004).