Struct ResidentRuntime 

pub struct ResidentRuntime<A, B, S>
where
    B: NeuralBackend,
    S: RuntimeState<B>,
    A: LayeredArchitecture<B, S>,
{ /* private fields */ }

Fully resident runtime using the same lifecycle as bounded execution.

Implementations

impl<A, B, S> ResidentRuntime<A, B, S>

where B: NeuralBackend, S: RuntimeState<B>, A: LayeredArchitecture<B, S>,

pub fn new( architecture: A, context: &<B::Tensor as Tensor>::Context, ) -> Result<Self, A::Error>

Builds every execution unit once and keeps it resident.

pub fn forward<'a>( &mut self, input: A::Input<'a>, state: &mut S, context: &<B::Tensor as Tensor>::Context, ) -> Result<B::Tensor, A::Error>

Runs one complete prefill or decode pass without dynamic dispatch.

pub fn forward_with_context<'a>( &mut self, input: A::Input<'a>, state: &mut S, context: &<B::Tensor as Tensor>::Context, ) -> Result<(B::Tensor, A::ForwardContext), A::Error>

Runs one complete pass and returns its architecture-owned context.

This is the resident counterpart of the bounded runtime’s context result and lets callers retain target captures without storing request-local tensors on the model object.

pub fn forward_with_traversal_hook<'a, H>( &mut self, input: A::Input<'a>, state: &mut S, context: &<B::Tensor as Tensor>::Context, hook: &mut H, ) -> Result<(B::Tensor, A::ForwardContext), A::Error>

where H: LayeredTraversalHook<B, A::ForwardContext, A::Error> + ?Sized,

Runs one resident pass through a statically dispatched traversal hook.

pub const fn architecture(&self) -> &A

Borrows the architecture and its pinned parameter topology.

pub fn architecture_mut(&mut self) -> &mut A

Mutably borrows the architecture.

pub fn units(&self) -> &[Vec<A::Unit>]

Borrows resident execution units for loading or inspection.

pub fn units_mut(&mut self) -> &mut [Vec<A::Unit>]

Mutably borrows resident execution units for parameter binding.

pub fn into_parts(self) -> (A, Vec<A::Unit>)

Decomposes the runtime without cloning backend-native values.