Skip to main content

OriginByteStream

Type Alias OriginByteStream 

Source
pub type OriginByteStream = Pin<Box<dyn Stream<Item = Result<Bytes>> + Send + Sync + 'static>>;
Expand description

Boxed byte stream returned by Origin::fetch. Each Bytes item is one chunk of the blob’s payload; the stream completes when the origin signals end-of-data.

The bound matches iroh_blobs::api::blobs::Blobs::add_stream: Send + Sync + 'static so the engine can hand the stream straight through to the store layer without an intermediate buffer. 'static means impls move whatever per-fetch state they hold (file handle, HTTP response, S3 ByteStream) into the stream by move-into-async-block; nothing borrows from &self on the origin.

Errors flow as io::Error. They are routed through retry classification by one of two paths (#519):

  • Buffer-then-commit (small blobs): when the adapter’s size_hint is at or below cache.origin_retry.buffered_max_bytes (default 4 MiB), the engine drains the stream into a bounded BytesMut before handing to iroh-blobs. Drain errors are classified via crate::retry::classify_io_error and re-feed the retry loop. No on-disk amplification — the partial bytes never reach the store.
  • Abort + restart (large blobs / unknown size_hint): the engine streams directly into iroh_blobs::Blobs::add_stream and captures any mid-stream io::Error via the side channel. After temp_tag().await completes, the captured error is classified; on Transient the partial TempTag is dropped (iroh-blobs GC reclaims the bytes at cache.gc_interval_sec cadence, #518) and the retry loop restarts from the headers phase. Worst-case orphaned bytes per fetch are (1 + max_retries) * max_blob_bytes until GC.

Typed Permanent markers short-circuit retry regardless of which path applies: the internal BlobTooLargeMarker surfaces as crate::CacheError::BlobTooLarge (size-cap breach is deterministic — retry won’t fit a bigger blob into a smaller cap), and crate::OriginError::DecompressionFailed surfaces as Permanent (corrupt body — retry sees the same bytes again).

Aliased Type§

#[repr(transparent)]
pub struct OriginByteStream { /* private fields */ }