Struct BlockStreamIterator 

pub struct BlockStreamIterator {
    pub all_blocks: Vec<(String, Vec<u8>)>,
    pub position: usize,
    pub config: StreamConfig,
    pub buffer: VecDeque<BlockChunk>,
    pub state: BlockStreamState,
    pub stats: StreamStats,
}

Synchronous, single-threaded streaming iterator over a block collection.

Backpressure

If the internal buffer reaches StreamConfig::max_buffer_chunks chunks, next_chunk returns None and increments StreamStats::backpressure_events. The caller must call drain_one before production can resume.

Example

use ipfrs_storage::block_stream::{BlockStreamIterator, StreamConfig};

let blocks: Vec<(String, Vec<u8>)> = (0u8..10)
    .map(|i| (format!("cid-{i}"), vec![i; 32]))
    .collect();
let mut iter = BlockStreamIterator::new(blocks, StreamConfig::default());

while !iter.is_exhausted() {
    if let Some(chunk) = iter.next_chunk() {
        println!("chunk {}: {} blocks", chunk.chunk_index, chunk.len());
    } else {
        // backpressure — drain before continuing
        iter.drain_one();
    }
}
// drain any remaining buffered chunks
while let Some(_chunk) = iter.drain_one() {}

Fields

all_blocks: Vec<(String, Vec<u8>)>

Source (CID, data) pairs.

position: usize

Current read position in all_blocks.

config: StreamConfig

Stream configuration.

buffer: VecDeque<BlockChunk>

Buffered but not yet consumed chunks.

state: BlockStreamState

Current state of the iterator.

stats: StreamStats

Accumulated statistics.

Implementations

impl BlockStreamIterator

pub fn new(blocks: Vec<(String, Vec<u8>)>, config: StreamConfig) -> Self

Construct a new iterator over blocks with the given config.

pub fn next_chunk(&mut self) -> Option<BlockChunk>

Attempt to produce the next chunk.

Returns None when:

• The buffer is already at capacity (backpressure — call drain_one).
• The iterator is exhausted.

On success, advances position, pushes a BlockChunk onto the buffer, and returns a clone of that chunk.

pub fn drain_one(&mut self) -> Option<BlockChunk>

Remove and return the oldest buffered chunk.

Updates StreamStats::chunks_drained and StreamStats::bytes_streamed. If the buffer becomes empty and all source blocks have been consumed, transitions to BlockStreamState::Exhausted.

pub fn fill_buffer(&mut self)

Fill the buffer from source blocks up to max_buffer_chunks.

Calls next_chunk repeatedly until the buffer is full, backpressure fires, or the source is exhausted.

pub fn is_exhausted(&self) -> bool

Returns true when all blocks have been produced and the buffer has been fully drained.

pub fn remaining_blocks(&self) -> usize

Number of source blocks not yet chunked.

pub fn buffered_chunks(&self) -> usize

Number of chunks currently sitting in the buffer.

Trait Implementations

impl Debug for BlockStreamIterator

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.