Struct StreamCache 

pub struct StreamCache {
    pub hits: u64,
    pub misses: u64,
    /* private fields */
}

Stream-aware cache for inference pipelines.

Key insight: during inference, the filter STATE is fixed (from training). Only the DATA changes. So we cache:

1. Filter states (from training) — keyed by config_hash + training_data_hash
2. Chunk results — keyed by config_hash + state_hash + chunk_hash

This means: if the same chunk passes through the same filter with the same trained state, the result is returned from cache instantly.

Fields

hits: u64

Stats

misses: u64

Implementations

impl StreamCache

pub fn new(max_chunks: usize) -> Self

pub fn load_state(&mut self, filter_id: &str, state_key: CacheKey, state: Value)

Load a filter’s trained state into the stream cache.

pub fn get_state(&self, filter_id: &str) -> Option<&Value>

Get a filter’s cached state (for forward() during inference).

pub fn get_chunk(&mut self, chunk_key: &CacheKey) -> Option<&Value>

Try to get a cached chunk result. chunk_key = hash(config_hash + state_hash + chunk_data_hash)

pub fn put_chunk(&mut self, chunk_key: CacheKey, value: Value)

Cache a chunk result.

pub fn hit_rate(&self) -> f64

Cache hit rate.