Skip to main content

S3Origin

Struct S3Origin 

Source
pub struct S3Origin { /* private fields */ }
Expand description

Origin backed by an S3-compatible object store.

Clone is cheap: the inner Client is Clone and uses an Arc internally; the bucket and prefix fields use Arc<str> so adding labels to diagnostic logs doesn’t allocate.

Implementations§

Source§

impl S3Origin

Source

pub async fn new(cfg: &S3OriginConfig) -> Result<Self>

Build an S3Origin from validated config. Returns an error if a Static credential variant carries empty bytes (defense in depth against a config-resolver gap or a hand-built S3OriginConfig).

I/O posture: construction itself is non-blocking — aws_config builds lazy provider chains and resolves credentials on the first signed request, not at load() time. The one operator-visible caveat is the AWS SSO provider (enabled via the sso feature on aws-config): if the ~/.aws/config profile in use is configured for SSO and the cached SSO token has expired, the first fetch call may fire an HTTP request to the SSO endpoint to refresh the token. That’s a property of the AWS credential chain, not this constructor; the construction itself stays I/O-free.

BehaviorVersion::latest() is set explicitly even though aws-config’s behavior-version-latest feature would imply the same. Belt-and-braces against a future feature-flag edit dropping the implicit path — older SDK versions panicked when no BehaviorVersion was set; current versions surface a runtime error. Either way, the explicit call site cannot regress.

Runtime: the origin’s pooled connections live on the tokio runtime this future runs on, and every later request send runs there (see origin::io_runtime). Build it on a runtime that outlives every fetch — the node’s main runtime — never on a short-lived one such as a pull leg’s.

§Errors

Fails when a Static credential carries an empty key or secret, or when this future does not run inside a tokio runtime.

Source

pub const fn with_decompress_mode(self, mode: DecompressMode) -> Self

Set the DecompressMode. See that type for the semantics of Auto vs Strict. Defaults to Auto. Mirrors super::HttpOrigin::with_decompress_mode so the wiring layer can honour an operator-configured decompress knob uniformly across backends.

Trait Implementations§

Source§

impl Clone for S3Origin

Source§

fn clone(&self) -> Self

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for S3Origin

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Origin for S3Origin

Source§

fn kind(&self) -> OriginKind

Tag identifying the backend type. Surfaced through crate::EvictionPreview::origin_kinds so admin dry-run callers can estimate origin egress cost (#439) — Filesystem is a local read; Http may bill metered bandwidth.
Source§

fn fetch( &self, hash: Hash, max_bytes: u64, ) -> Pin<Box<dyn Future<Output = Result<OriginFetch, OriginPullError>> + Send + '_>>

Fetch the blob with the given hash. max_bytes is an advisory cap — implementations should short-circuit if they can cheaply detect the payload would exceed it (e.g. from a Content-Length header), and the cache engine enforces the cap on the returned bytes regardless.
Source§

fn fetch_range_data( &self, hash: Hash, req: OriginRangeRequest, ) -> Pin<Box<dyn Future<Output = Result<OriginRangeFetch, OriginPullError>> + Send + '_>>

Fetch the chunk-group-aligned byte span req of the blob hash, for a range-scoped pull (ADR 037 §Origin-tier pull-through, #823). Each call covers at most crate::RANGE_PULL_WINDOW_BYTES; a longer span takes several calls, in order. The engine fetches the sibling {H}.obao4 outboard separately through Self::fetch_outboard. Read more
Source§

fn fetch_outboard( &self, hash: Hash, outboard_max_bytes: u64, ) -> Pin<Box<dyn Future<Output = Result<OutboardFetch, OriginPullError>> + Send + '_>>

Fetch just the sibling {H}.obao4 outboard for the blob hash — no accompanying blob data (#1130, feeds a later “stream-while-store” serve path: a node that holds the outboard can start verifying against H before the full blob has finished pulling through). outboard_max_bytes caps the read — the engine derives it from the blob size (an outboard is O(blob/256)), and an oversize outboard is malformed/foreign. The engine caches a non-empty outboard of at most 256 MiB (a blob of up to 64 GiB) per hash across range pulls, so it reads such an outboard once while the entry stays cached. It reads an empty outboard, or a larger one, again on each draw. Each pull fetches its span window by window through Self::fetch_range_data. Read more
Source§

fn size( &self, hash: Hash, ) -> Pin<Box<dyn Future<Output = Result<Option<u64>, OriginPullError>> + Send + '_>>

Best-effort total byte size of the blob hash, used to scope a range-pull (Self::fetch_range_data, #823): the bao tree needs the blob’s exact total length to verify a sub-range against the root H, and the {H}.obao4 outboard alone only pins the size to within one chunk group (IROH_BLOCK_SIZE, 16 KiB — the final group’s true length isn’t in the tree). The cdn/client/v1 serving handler calls this on a cold cache miss — via crate::CacheEngine::origin_size — before it signs a response and runs the two-leg serve-miss spine over crate::CacheEngine::origin_range_wire. Read more
Source§

fn enumerate( &self, ) -> Pin<Box<dyn Future<Output = Result<Vec<Hash>, OriginPullError>> + Send + '_>>

Enumerate the hashes this origin currently holds, so a node can advertise them (probe has_blob / DHT announce) before any pull-through — closing the “cold origin blob is undiscoverable until first pulled” gap (#1130). Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

Source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more