Module streaming 

Real-time Streaming Data Ingestion

Provides async stream processing with windowed analysis, real-time pattern detection, backpressure handling, and comprehensive metrics collection.

Features

• Async stream processing for continuous data ingestion
• Tumbling and sliding window analysis
• Real-time pattern detection with callbacks
• Automatic backpressure handling
• Throughput and latency metrics

Example

use futures::stream;
use std::time::Duration;

let config = StreamingConfig {
    window_size: Duration::from_secs(60),
    slide_interval: Duration::from_secs(30),
    max_buffer_size: 10000,
    ..Default::default()
};

let mut engine = StreamingEngine::new(config);

// Set pattern callback
engine.set_pattern_callback(|pattern| {
    println!("Pattern detected: {:?}", pattern);
});

// Ingest stream
let stream = stream::iter(vectors);
engine.ingest_stream(stream).await?;

// Get metrics
let metrics = engine.metrics();
println!("Processed: {} vectors, {} patterns",
         metrics.vectors_processed, metrics.patterns_detected);

Structs

StreamingConfig

Configuration for the streaming engine

StreamingEngine

Streaming engine for real-time data ingestion and pattern detection

StreamingEngineBuilder

Builder for StreamingEngine with fluent API

StreamingMetrics

Streaming metrics for monitoring performance