Struct NativeSession 

pub struct NativeSession<T> { /* private fields */ }

Owned stream-bound model state scheduled alongside prepared GPU graphs.

Stages execute synchronously on the calling thread and may borrow inputs and progress callbacks. They reserve the compute lane, without blocking unrelated upload/download/NPU lanes. Earlier submissions drain before admission; later compute cannot overtake a waiting stage. Admission uses the runtime’s bounded submission capacity. This is a stage boundary, not asynchronous inference or separate transfer scheduling within a stage.

Implementations

impl<T> NativeSession<T>

pub unsafe fn new( runtime: &Runtime, state: T, fence: fn(&mut T) -> Result<()>, ) -> Self

Own a model’s private native state and its completion fence.

Safety

State must own every private stream, graph and allocation used by its stages, on this runtime’s selected device. fence must drain all those streams before returning success, including after a failed stage. No concurrent external access to this state is allowed. Construction must leave no pending work; subsequent work must go through Self::run.

pub fn state(&self) -> Result<&T>

Inspect idle model metadata. Returns DeviceLost after quarantine.

pub unsafe fn run<R>(&mut self, stage: impl FnOnce(&mut T) -> R) -> Result<R>

Reserve compute, run a borrowed stage, then fence on every exit path. Client errors may be returned as R = Result<_, ClientError> and do not poison a successfully drained session. A panic is resumed after fencing. A failed or panicking fence quarantines the entire owner, including its allocation charges, and rejects all subsequent stages.

Safety

The stage must retain all resources until the fence, submit only on the owned streams, and not expose pending raw uses outside this session. It must not transfer private native owners out of the state, including through its return value; quarantine must retain the complete owner. Stages may access only private native storage, not coordinated buffers; use Graph::gpu_scoped for operations needing tracked memory hazards. Do not wait for coordinated compute or another native session in this runtime from the callback: this stage already holds its compute lane.