Skip to main content

StreamingPipeline

threecrate_algorithms::streaming

Trait StreamingPipeline 

Source 
pub trait StreamingPipeline<T> {
    type Output;

    // Required methods
    fn process_chunk(&mut self, chunk: &[T]) -> Result<()>;
    fn finalize(self) -> Result<Self::Output>;

    // Provided method
    fn memory_bytes(&self) -> usize { ... }
}
Expand description

Trait for chunk-based streaming processors.

Implementations accumulate state across calls to process_chunk and produce a final result via finalize. The chunk size controls peak RAM usage: smaller chunks use less memory at the cost of more function-call overhead.

Required Associated Types§

Source

type Output

The type produced after all chunks have been processed.

Required Methods§

Source

fn process_chunk(&mut self, chunk: &[T]) -> Result<()>

Ingest one chunk of items. Called repeatedly until the source is exhausted. chunk will never be empty.

Source

fn finalize(self) -> Result<Self::Output>

Consume the pipeline and return the accumulated output.

Provided Methods§

Source

fn memory_bytes(&self) -> usize

Estimated number of bytes currently held by this pipeline stage. Default returns 0; override to expose real memory usage.

Implementors§