Struct Session 

pub struct Session { /* private fields */ }

Represents an active inference session with its own KV cache state.

Sessions hold runtime state (KV cache, token history, etc.) that persists across prefill and decode phases. Multiple sessions can exist simultaneously, each with its own independent state.

Sessions are intentionally not Clone — cloning would imply duplicating KV cache state, which is not a cheap or well-defined operation.

Implementations

impl Session

pub fn id(&self) -> Uuid

Return the unique session ID for tracking and logging.

pub fn new() -> Self

Create a new inference session with a random UUID.

pub fn with_id(id: Uuid) -> Self

Create a session with an explicit ID (useful for testing/replay).

Trait Implementations

impl Debug for Session

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Session

fn default() -> Self

Returns the “default value” for a type.