Struct InferenceModel 

pub struct InferenceModel<B, InP, OutP, const MAX_BATCH: usize>
where
    B: ComputeBackend<InP, OutP>,
    InP: Payload,
    OutP: Payload + Default + Copy,
{ /* private fields */ }

Generic 1×1 inference node for any backend (dyn-free).

• MAX_BATCH is a compile-time cap used for the no-alloc path.
• When alloc is enabled, the batched path uses Vec (still no unsafe).

Implementations

impl<B, InP, OutP, const MAX_BATCH: usize> InferenceModel<B, InP, OutP, MAX_BATCH>

where B: ComputeBackend<InP, OutP>, InP: Payload, OutP: Payload + Default + Copy,

pub fn new<'desc>( backend: B, desc: B::ModelDescriptor<'desc>, node_policy: NodePolicy, node_caps: NodeCapabilities, input_acceptance: [PlacementAcceptance; 1], output_acceptance: [PlacementAcceptance; 1], ) -> Result<Self, B::Error>

Construct a new InferenceModel node.

• backend: concrete compute backend (e.g., Tract, TFLM adapter).
• desc: backend-specific, borrowed model descriptor (e.g., bytes, artifact).
• node_policy: batching/budget/deadline policies for the node.
• node_caps: advertised capabilities (e.g., device streams).
• input_acceptance / output_acceptance: zero-copy placement preferences.

pub fn backend_capabilities(&self) -> BackendCapabilities

Return cached backend capabilities for this node.

pub fn model_metadata(&self) -> ModelMetadata

Return cached model metadata for this node.

impl<const MAX_BATCH: usize> InferenceModel<TestTensorBackend, Tensor<u32, TEST_TENSOR_ELEMENT_COUNT, 2>, Tensor<u32, TEST_TENSOR_ELEMENT_COUNT, 2>, MAX_BATCH>

pub fn new_identity( node_capabilities: NodeCapabilities, node_policy: NodePolicy, input_placement_acceptance: [PlacementAcceptance; 1], output_placement_acceptance: [PlacementAcceptance; 1], ) -> Result<Self, InferenceError>

Construct the identity test node with your policy/capability params.

pub fn kind() -> NodeKind

Handy constant for tests.

Trait Implementations

impl<B, InP, OutP, const MAX_BATCH: usize> Node<1, 1, InP, OutP> for InferenceModel<B, InP, OutP, MAX_BATCH>

where B: ComputeBackend<InP, OutP>, InP: Payload + Default + Copy, OutP: Payload + Default + Copy,

fn set_policy(&mut self, policy: NodePolicy)

TEST ONLY method used to override batching policis for node contract tests.

fn describe_capabilities(&self) -> NodeCapabilities

Return the node’s capability descriptor.

fn input_acceptance(&self) -> [PlacementAcceptance; 1]

Return the node’s port placement acceptances (zero-copy compatibility).

fn output_acceptance(&self) -> [PlacementAcceptance; 1]

Return the node’s output placement preferences (zero-copy compatibility).

fn policy(&self) -> NodePolicy

Return the node’s policy bundle.

fn node_kind(&self) -> NodeKind

Return the type of node (Model, processing, source, sink).

fn initialize<C, T>( &mut self, _clock: &C, _telemetry: &mut T, ) -> Result<(), NodeError>

where T: Telemetry,

Prepare internal state, acquire buffers, and register telemetry series.

fn start<C, T>( &mut self, _clock: &C, _telemetry: &mut T, ) -> Result<(), NodeError>

where T: Telemetry,

Optional warm-up (e.g., compile kernels, prime pools). Default: no-op.

fn process_message<C>( &mut self, msg: &Message<InP>, _sys_clock: &C, ) -> Result<ProcessResult<OutP>, NodeError>

where C: PlatformClock + Sized,

Per-message processing hook.

fn step<'g, 't, 'c, InQ, OutQ, InM, OutM, C, Tel>( &mut self, ctx: &mut StepContext<'g, 't, 'c, 1, 1, InP, OutP, InQ, OutQ, InM, OutM, C, Tel>, ) -> Result<StepResult, NodeError>

where InQ: Edge, OutQ: Edge, InM: MemoryManager<InP>, OutM: MemoryManager<OutP>, C: PlatformClock + Sized, Tel: Telemetry + Sized,

Execute one cooperative step using the provided context.

fn step_batch<'g, 't, 'c, InQ, OutQ, InM, OutM, C, Tel>( &mut self, ctx: &mut StepContext<'g, 't, 'c, 1, 1, InP, OutP, InQ, OutQ, InM, OutM, C, Tel>, ) -> Result<StepResult, NodeError>

where InQ: Edge, OutQ: Edge, InM: MemoryManager<InP>, OutM: MemoryManager<OutP>, C: PlatformClock + Sized, Tel: Telemetry + Sized,

Default batched-step implementation that honors all NodePolicy batching variants while delegating actual consumption to the implementor’s single-message process_message() method.

fn on_watchdog_timeout<C, Tel>( &mut self, clock: &C, _telemetry: &mut Tel, ) -> Result<StepResult, NodeError>

where C: PlatformClock + Sized, Tel: Telemetry,

Handle watchdog timeouts by applying over-budget policy (degrade/default/skip).

fn stop<C, Tel>( &mut self, _clock: &C, _telemetry: &mut Tel, ) -> Result<(), NodeError>

where Tel: Telemetry,

Flush and release resources, if any. Default: no-op.