Trait Origin 

pub trait Origin:
    Debug
    + Send
    + Sync
    + 'static {
    // Required methods
    fn fetch(
        &self,
        hash: Hash,
        max_bytes: u64,
    ) -> Pin<Box<dyn Future<Output = Result<OriginFetch, OriginPullError>> + Send + '_>>;
    fn kind(&self) -> OriginKind;

    // Provided methods
    fn fetch_range_data(
        &self,
        _hash: Hash,
        _req: OriginRangeRequest,
    ) -> Pin<Box<dyn Future<Output = Result<OriginRangeFetch, OriginPullError>> + Send + '_>> { ... }
    fn fetch_outboard(
        &self,
        _hash: Hash,
        _outboard_max_bytes: u64,
    ) -> Pin<Box<dyn Future<Output = Result<OutboardFetch, OriginPullError>> + Send + '_>> { ... }
    fn size(
        &self,
        _hash: Hash,
    ) -> Pin<Box<dyn Future<Output = Result<Option<u64>, OriginPullError>> + Send + '_>> { ... }
    fn enumerate(
        &self,
    ) -> Pin<Box<dyn Future<Output = Result<Vec<Hash>, OriginPullError>> + Send + '_>> { ... }
}

An origin backend. Implementors fetch a blob identified by its BLAKE3 hash.

The origin is not responsible for verifying the hash — the cache engine does that after the bytes come back. Returning the wrong bytes is a detectable protocol violation, not a security breach: no data is trusted until it matches the content address.

Error classification

Adapters return OriginPullError::Transient for failures that can be retried (HTTP 5xx/408/429, transient I/O), and OriginPullError::Permanent for failures that won’t be cured by retrying (HTTP 4xx other than 404, encoding/cap breaches, permission denied). The cache engine’s retry loop keys off this distinction. NotFound is not an error — adapters return OriginFetch::NotFound.

Required Methods

fn fetch( &self, hash: Hash, max_bytes: u64, ) -> Pin<Box<dyn Future<Output = Result<OriginFetch, OriginPullError>> + Send + '_>>

Fetch the blob with the given hash. max_bytes is an advisory cap — implementations should short-circuit if they can cheaply detect the payload would exceed it (e.g. from a Content-Length header), and the cache engine enforces the cap on the returned bytes regardless.

fn kind(&self) -> OriginKind

Tag identifying the backend type. Surfaced through crate::EvictionPreview::origin_kinds so admin dry-run callers can estimate origin egress cost (#439) — Filesystem is a local read; Http may bill metered bandwidth.

Provided Methods

fn fetch_range_data( &self, _hash: Hash, _req: OriginRangeRequest, ) -> Pin<Box<dyn Future<Output = Result<OriginRangeFetch, OriginPullError>> + Send + '_>>

Fetch the chunk-group-aligned byte span req of the blob hash, for a range-scoped pull (ADR 037 §Origin-tier pull-through, #823). Each call covers at most crate::RANGE_PULL_WINDOW_BYTES; a longer span takes several calls, in order. The engine fetches the sibling {H}.obao4 outboard separately through Self::fetch_outboard.

The default implementation returns OriginRangeFetch::Unsupported, so the engine degrades to a whole-blob Self::fetch pull. A range pull needs BOTH this method and Self::fetch_outboard: an origin that overrides only one never range-pulls. The three shipped adapters override both.

data must be exactly req.len() bytes. A wrong-length window fails verification: the engine degrades the range pull, and the own-origin serve-miss path reports crate::CacheError::VerifyFailed.

Like Self::fetch, the origin is a dumb byte store: the returned data is untrusted and verified against the root H by the engine: one bao_tree::io::fsm::encode_ranges_validated pass on the crate::CacheEngine::origin_range_wire path — before any byte is imported or served.

Returning OriginRangeFetch::Unsupported is the correct, expected answer whenever the optimization can’t apply (no Range/206, a short object) — it is not an error. Only genuine transport / permission failures surface as OriginPullError.

fn fetch_outboard( &self, _hash: Hash, _outboard_max_bytes: u64, ) -> Pin<Box<dyn Future<Output = Result<OutboardFetch, OriginPullError>> + Send + '_>>

Fetch just the sibling {H}.obao4 outboard for the blob hash — no accompanying blob data (#1130, feeds a later “stream-while-store” serve path: a node that holds the outboard can start verifying against H before the full blob has finished pulling through). outboard_max_bytes caps the read — the engine derives it from the blob size (an outboard is O(blob/256)), and an oversize outboard is malformed/foreign. The engine caches a non-empty outboard of at most 256 MiB (a blob of up to 64 GiB) per hash across range pulls, so it reads such an outboard once while the entry stays cached. It reads an empty outboard, or a larger one, again on each draw. Each pull fetches its span window by window through Self::fetch_range_data.

The default implementation returns OutboardFetch::Unsupported, which also disables range pulls (see Self::fetch_range_data). The three shipped adapters override it.

Like Self::fetch_range_data, returning OutboardFetch::Unsupported or OutboardFetch::NotFound is never an error — only a genuine transport / permission failure surfaces as OriginPullError.

fn size( &self, _hash: Hash, ) -> Pin<Box<dyn Future<Output = Result<Option<u64>, OriginPullError>> + Send + '_>>

Best-effort total byte size of the blob hash, used to scope a range-pull (Self::fetch_range_data, #823): the bao tree needs the blob’s exact total length to verify a sub-range against the root H, and the {H}.obao4 outboard alone only pins the size to within one chunk group (IROH_BLOCK_SIZE, 16 KiB — the final group’s true length isn’t in the tree). The cdn/client/v1 serving handler calls this on a cold cache miss — via crate::CacheEngine::origin_size — before it signs a response and runs the two-leg serve-miss spine over crate::CacheEngine::origin_range_wire.

The probe is cheap (HTTP HEAD / S3 HeadObject / fs metadata), and Ok(None) is an authoritative “this origin does not hold the object — or holds it in a shape the caller cannot range against”: a genuine 404 / not-found, or a backend reporting a Content-Encoding whose advertised length is the encoded size (not the canonical blob length, same trap as Self::fetch’s size_hint). Anything that answers neither way — a transport failure, a 5xx/408/429 outage, a permission decline, a disabled redirect — surfaces as OriginPullError (Transient for the retriable statuses and transport faults, Permanent for the rest) and must NOT degrade to None inside the adapter. Fault-aware callers depend on the split: the origin-only serve gate and the DHT announce set read Ok(None) as an absence they may memoise, sign as NotFound to a paying client, and unschedule from republish — an outage folded into None does all three. CacheEngine::origin_size advances the origin chain past a per-origin fault, but a probe that ends on faults surfaces the last one (#1129): the serve path must report a degraded node, not an empty one. The default returns Ok(None), so a custom Origin needs no change and simply never range-pulls.

fn enumerate( &self, ) -> Pin<Box<dyn Future<Output = Result<Vec<Hash>, OriginPullError>> + Send + '_>>

Enumerate the hashes this origin currently holds, so a node can advertise them (probe has_blob / DHT announce) before any pull-through — closing the “cold origin blob is undiscoverable until first pulled” gap (#1130).

The default returns an empty list. Only an enumerable backend overrides it: the local filesystem, whose sharded directory listing is the hash set. HTTP has no listing endpoint, and S3’s ListObjectsV2 is deliberately not walked (egress + unbounded bucket), so remote origins are advertised via the operator’s pinned_hashes instead of this method.

Enumeration is presence-only (trust model): it does not read or hash file contents. Callers verify served bytes against the content address at serve time, and a wrong/corrupt file is the publisher’s error (it only hurts the publisher — an announced-but-unservable hash costs bandwidth + local reputation, never a bond slash, provided the serve path never signs an ok:false refusal).

Dyn Compatibility

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors

impl Origin for FilesystemOrigin

impl Origin for HttpOrigin

impl Origin for S3Origin