Struct RateLimitedPipeline 

pub struct RateLimitedPipeline { /* private fields */ }

A rate-limited wrapper around any PhaseSink.

Uses a token bucket [RateLimiter] to control emission rate. Each call to emit() acquires a token before forwarding to the inner sink, blocking if the rate is exceeded.

Also adds a monotonic sequence number and optional progress events to the output stream.

Thread safety note: The rate limiter may sleep while holding an internal mutex. This is correct for single-threaded generation (the current model) but would serialize all callers if emit() is called from multiple threads concurrently. If multi-threaded emission is needed in the future, the lock-then-sleep pattern must be restructured.

Implementations

impl RateLimitedPipeline

pub fn new( inner: Box<dyn PhaseSink>, events_per_second: f64, burst_size: u32, progress_interval: u64, ) -> Self

Wrap a PhaseSink with rate limiting.

• events_per_second: target rate (0 = unlimited)
• burst_size: token bucket burst capacity
• progress_interval: emit a _progress event every N items (0 = disabled)

pub fn set_rate(&self, events_per_second: f64)

Update the rate limit dynamically (e.g., from a REST endpoint).

Trait Implementations

impl PhaseSink for RateLimitedPipeline

fn emit( &self, phase: &str, item_type: &str, item: &Value, ) -> Result<(), StreamError>

Emit a single generated item.

fn phase_complete(&self, phase: &str) -> Result<(), StreamError>

Signal that a generation phase has completed.

fn flush(&self) -> Result<(), StreamError>

Flush any buffered data to the underlying sink.

fn stats(&self) -> StreamStats

Return current streaming statistics.