Struct Runtime

pub struct Runtime { /* private fields */ }

The runtime wraps a multitude of inference models with batching support, and support for time-limited execution.

Implementations

impl Runtime

pub fn new() -> Self

Create a new empty runtime.

pub fn add_inferer(&mut self, inferer: impl Inferer + 'static + Send) -> BrainId

Add a new inferer to this runtime. The new infererer will be at the end of the inference queue when using timed inference.

pub fn push( &mut self, brain: BrainId, agent: AgentId, state: State<'_>, ) -> Result<(), CervoError>

Queue the state to brain for agent, to be included in the next inference batch.

pub fn infer_single( &mut self, brain_id: BrainId, state: State<'_>, ) -> Result<Response<'_>, CervoError>

Run a single item through the specific brain. If there’s pending data for the brain, this’ll have some extra overhead for new allocations.

pub fn run( &mut self, ) -> Result<HashMap<BrainId, HashMap<AgentId, Response<'_>>>, CervoError>

Executes all models with queued data.

pub fn run_for( &mut self, duration: Duration, ) -> Result<HashMap<BrainId, HashMap<AgentId, Response<'_>>>, CervoError>

Executes all models with queued data. Will attempt to keep total time below the provided duration, but due to noise or lack of samples might miss the deadline. See the note in the root.

pub fn output_shapes( &self, brain: BrainId, ) -> Result<&[(String, Vec<usize>)], CervoError>

Retrieve the output shapes for the provided brain.

pub fn input_shapes( &self, brain: BrainId, ) -> Result<&[(String, Vec<usize>)], CervoError>

Retrieve the input shapes for the provided brain.

pub fn clear(&mut self) -> Result<(), CervoError>

Clear all models and all related data. Will error (after clearing all data) if there was queued items that are now orphaned.

pub fn remove_inferer(&mut self, brain: BrainId) -> Result<(), CervoError>

Clear a model and related data. Will error (after clearing all data) if there was queued items that are now orphaned.

Trait Implementations

impl Default for Runtime

fn default() -> Self

Returns the “default value” for a type.