Skip to main content

Streamer

Struct Streamer 

Source
pub struct Streamer<'a, P, F> { /* private fields */ }
Expand description

Stream a single blob’s verified front to a consumer, paced by consumption.

Holds the injected pull machinery — the blob’s holders, the [SourceProvider] that builds their lanes, the command-wide [PeerHealth], and one funder — plus a scratch directory for the fill store. The front is fetched through [crate::acquire] across a small, bounded set of lanes; a lane that faults cools and returns. Streamer::open consumes it to start one stream.

Read inside StreamDrive::alongside, so the drive keeps paying and draining open legs while the reader waits on its consumer. See the stream example for the whole sequence.

use std::path::Path;
use std::sync::Arc;

use decdn_client::source::BlobSource;
use decdn_client::{
    Funder, NoCache, ProgressClock, PullConfig, StaticSources, StopPolicy, StreamCandidate,
    Streamer,
};

async fn stream<S: BlobSource, F: Funder>(
    candidates: Vec<StreamCandidate<S>>,
    funder: F,
    hash: [u8; 32],
    total_bytes: u64,
    scratch: &Path,
) -> anyhow::Result<()> {
    let streamer = Streamer::new(StaticSources::new(candidates)?, funder, scratch);
    let stop = StopPolicy::new(true, None, Arc::new(ProgressClock::new()));
    let (mut reader, mut drive) = streamer
        .open(hash, total_bytes, &PullConfig::new(), Arc::new(NoCache), stop)
        .await?;
    let mut out = tokio::io::stdout();
    drive.alongside(tokio::io::copy(&mut reader, &mut out)).await?;
    Ok(())
}

Implementations§

Source§

impl<'a, P, F> Streamer<'a, P, F>

Source

pub fn new(provider: P, funder: F, scratch: &'a Path) -> Self

Build a streamer whose holders and lanes provider finds and builds, funded through funder, filling into scratch.

The stream starts by discovering its holders, under a fresh RecoveryGate. The builder methods change both.

Source

pub fn holders(self, holders: Vec<Holder>) -> Self

Start the stream from holders, in preference order, instead of discovering them. A probe’s measured holders go here.

Source

pub fn recovery(self, recovery: Arc<RecoveryGate>) -> Self

Run the funding recovery step under recovery, the fetch’s gate. A caller that has already run part of the fetch under a gate (a first open, an earlier pass) passes that gate, so the progress rule spans the whole fetch.

Source

pub fn credentials(self, slot: CredentialSlot) -> Self

Pay as a delegate under slot’s credential, as crate::Downloader::credentials does.

Source

pub const fn max_blob_bytes(self, max_blob_bytes: u64) -> Self

Cap the stream at max_blob_bytes (0 for no cap). A size claim above the cap is clamped to it before the fill store is sized, and a blob that holds bytes past the cap ends the stream with crate::BlobTooLarge ([crate::AcquireEnv::max_blob_bytes]).

Source§

impl<P, F> Streamer<'_, P, F>
where P: SourceProvider, F: Funder,

Source

pub async fn open<'a>( self, hash: [u8; 32], total_bytes: u64, config: &PullConfig, cache: Arc<dyn BlobCache>, stop: StopPolicy, ) -> Result<(VerifiedReader, StreamDrive<'a>)>
where P: 'a, F: 'a,

Open a consumption-paced stream over hash (whose content length is total_bytes), returning the VerifiedReader the caller drains and the StreamDrive that fetches into it.

The caller MUST poll the drive for as long as it reads — for example with StreamDrive::alongside around its read loop. The drive runs apart from the reads so a paid leg keeps paying while the consumer is slow or paused; the read-ahead window, not the reads, bounds it. Dropping the drive stops the fetch, and the reader then ends with an error after the verified bytes it already holds.

The drive borrows 'a from its sources — a real PeerSource over a borrowed Endpoint is not 'static — so the caller keeps the endpoint (and provider) alive for as long as it polls the drive. The reader owns its state and borrows nothing.

stop decides when a stream that makes no verified progress gives up; the reader then ends with crate::GaveUp’s message.

On a REVISIT — cache already holds the whole blob — the reader serves straight from the cache, the drive is already complete, and the network is never touched. Otherwise the fetch never runs more than config.read_ahead_bytes (raised to at least [PULL_WINDOW_FLOOR]) ahead of the consumer’s cursor, and the verified blob is teed to cache for the next revisit.

total_bytes is the first size claim, a hint. A resumed store’s bound wins, and a leg that verifies the final chunk proves the size. hash is the bao root every byte is verified against; a wrong hash surfaces as a read error, never as silent corruption.

§Errors

A cache read fault at open, or a fill-store open/create I/O error. Faults DURING the fetch (a refused or stalled pull, a verification failure) surface later, from the reader.

Trait Implementations§

Source§

impl<P, F> Debug for Streamer<'_, P, F>

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

§

impl<'a, P, F> !RefUnwindSafe for Streamer<'a, P, F>

§

impl<'a, P, F> !UnwindSafe for Streamer<'a, P, F>

§

impl<'a, P, F> Freeze for Streamer<'a, P, F>
where P: Freeze, F: Freeze,

§

impl<'a, P, F> Send for Streamer<'a, P, F>
where P: Send, F: Send,

§

impl<'a, P, F> Sync for Streamer<'a, P, F>
where P: Sync, F: Sync,

§

impl<'a, P, F> Unpin for Streamer<'a, P, F>
where P: Unpin, F: Unpin,

§

impl<'a, P, F> UnsafeUnpin for Streamer<'a, P, F>
where P: UnsafeUnpin, F: UnsafeUnpin,

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> MaybeSend for T
where T: Send,

Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more