Struct QueryResultIterator 

pub struct QueryResultIterator {
    pub metadata: QueryMetadata,
    pub total_rows_hint: Option<u64>,
    /* private fields */
}

Streaming query result iterator for memory-efficient processing

Instead of materializing all rows into a Vec, this iterator yields rows lazily via a channel, allowing processing of arbitrarily large result sets within the 128MB memory budget.

Memory Budget

To stay within the 128MB target, callers MUST create a bounded channel with capacity from StreamingConfig::buffer_size. Assuming average row size of 1KB:

• buffer_size: 1024 = ~1MB in flight
• chunk_size: 10_000 = ~10MB per chunk
• Total peak usage: ~11MB (well within 128MB budget)

For rows with large blobs/text, reduce buffer sizes proportionally.

Contract

1. The caller MUST create a bounded channel with mpsc::channel(config.buffer_size)
2. The iterator does NOT own the sender; the caller must spawn a task to send rows
3. The iterator is consumed once; create a new one for subsequent queries

Example

let config = StreamingConfig::default();
let (tx, rx) = tokio::sync::mpsc::channel(config.buffer_size);

// Spawn producer
tokio::spawn(async move {
    for row in rows {
        if tx.send(Ok(row)).await.is_err() {
            break; // Consumer dropped
        }
    }
});

// Create iterator from receiver
let mut iterator = QueryResultIterator::new(rx, metadata);

while let Some(row_result) = iterator.next_async().await {
    let row = row_result?;
    writer.write_row(&row)?;
}

Fields

metadata: QueryMetadata

Query metadata (columns, etc.)

total_rows_hint: Option<u64>

Total rows hint (if known from query planning)

Implementations

impl QueryResultIterator

pub fn new( receiver: Receiver<Result<QueryRow, Error>>, metadata: QueryMetadata, ) -> Self

Create a new streaming result iterator

pub fn with_total_hint(self, total: u64) -> Self

Create with a known total row count hint

pub async fn next_async(&mut self) -> Option<Result<QueryRow, Error>>

Receive next row (async)

Returns None when all rows have been received.

pub async fn collect_chunk( &mut self, size: usize, ) -> Result<Vec<QueryRow>, Error>

Collect into chunks of specified size

Returns a chunk of rows up to size. May return fewer rows if the stream ends or an error occurs.

Arguments

• size - Maximum number of rows to collect. Limited to MAX_CHUNK_SIZE (100,000) to prevent unbounded memory allocation.

Returns

A vector of rows, which may be smaller than size if the stream ends or an error occurs.

pub fn rows_received(&self) -> u64

Get count of rows received so far

pub fn progress_percent(&self) -> Option<f64>

Get progress as a percentage (if total is known)