Struct StreamingModelRunner 

pub struct StreamingModelRunner { /* private fields */ }

Runs ML inference on stream events with automatic batching.

Events are collected until either batch_size events accumulate or max_latency_ms elapses, then inference is run on the batch.

Implementations

impl StreamingModelRunner

pub fn new(config: ModelConfig) -> Self

Creates a new streaming model runner.

pub fn enqueue( &self, features: Array1<f64>, event_id: String, ) -> Option<Vec<Prediction>>

Enqueues an event for prediction.

Returns predictions if a batch was triggered.

pub fn flush_if_due(&self) -> Option<Vec<Prediction>>

Flushes any pending events if the latency threshold has been exceeded.

Returns predictions if flush was needed.

pub fn flush(&self) -> Vec<Prediction>

Forces inference on all pending events regardless of thresholds.

pub fn predict(&self, events: &[(Array1<f64>, String)]) -> Vec<Prediction>

Runs batched inference directly on a slice of stream events.

pub fn update_weights(&self, weights: Array1<f64>, bias: f64)

Updates the model weights.

pub fn stats(&self) -> ModelRunnerStats

Returns runner statistics.

pub fn pending_count(&self) -> usize

Returns the number of pending events.