Struct StreamingDecoder 

pub struct StreamingDecoder<R> { /* private fields */ }

Asynchronous streaming decoder — yields blocks one at a time without buffering the entire payload.

This is the primary API for large payloads or network streams. The decoder reads the header first, then yields blocks as they are fully received. Backpressure is handled naturally: the stream only reads the next block when the caller awaits the next item.

Unlike the synchronous BcpDecoder which requires the entire payload in memory, StreamingDecoder reads incrementally from any AsyncRead source (files, TCP sockets, HTTP response bodies, etc.).

Important: whole-payload compression disables streaming

When the header’s COMPRESSED flag is set (whole-payload compression), the streaming decoder falls back to buffering the entire payload before yielding any blocks. This is unavoidable: zstd requires the full compressed input to decompress. The API surface remains the same (you still call next() in a loop), but the memory and latency characteristics become identical to BcpDecoder::decode.

True incremental streaming is only achieved with uncompressed or per-block compressed payloads. If streaming is critical for your use case, prefer per-block compression ([BcpEncoder::compress_blocks]) over whole-payload compression ([BcpEncoder::compress_payload]).

Content store

To decode payloads with IS_REFERENCE blocks, provide a content store via with_content_store.

Example

use bcp_decoder::StreamingDecoder;
use tokio::io::AsyncRead;

async fn decode_from_reader(reader: impl AsyncRead + Unpin) {
    let mut stream = StreamingDecoder::new(reader);
    while let Some(event) = stream.next().await.transpose().unwrap() {
        // Process each DecoderEvent...
    }
}

Implementations

impl<R: AsyncRead + Unpin> StreamingDecoder<R>

pub fn new(reader: R) -> Self

Create a new streaming decoder over the given async reader.

The decoder starts in ReadHeader state and will read the 8-byte file header on the first call to next.

pub fn with_content_store(self, store: Arc<dyn ContentStore>) -> Self

Attach a content store for resolving IS_REFERENCE blocks.

When a block has the IS_REFERENCE flag set, its 32-byte body is looked up in this store to retrieve the original content.

pub async fn next(&mut self) -> Option<Result<DecoderEvent, DecodeError>>

Read the next event from the stream.

Returns Ok(Some(event)) for each decoded event, Ok(None) when the stream is exhausted (END sentinel reached), or Err on any decode error.

The first call always yields DecoderEvent::Header. Subsequent calls yield DecoderEvent::Block until the END sentinel.