Skip to main content

S3Store

Struct S3Store 

Source
pub struct S3Store { /* private fields */ }
Expand description

A content-addressable store backed by an S3 (or S3-compatible) bucket.

Construct one with S3Store::connect (resolves the standard AWS credential chain) or S3Store::from_client (an already-built SDK client, e.g. for tests against an emulator).

Implementations§

Source§

impl S3Store

Source

pub fn connect( store_url: &str, endpoint_url: Option<&str>, ) -> Result<Self, StoreError>

Connects to the s3://bucket/prefix store, resolving credentials and region via the standard AWS chain (aws_config::load_defaults).

The HTTP client is pinned to the ring rustls provider (see the module docs). An optional endpoint_url selects an S3-compatible backend (path-style addressing is enabled when an endpoint is given, as emulators rarely support virtual-host addressing).

§Errors

StoreError::Backend if the tokio runtime cannot be created or the AWS configuration cannot be loaded.

Source

pub fn connect_with( store_url: &str, endpoint_url: Option<&str>, config: TransferConfig, ) -> Result<Self, StoreError>

Like connect, but carries a TransferConfig for concurrency / bandwidth control. The existing connect delegates here with TransferConfig::default.

§Errors

StoreError::Backend if the tokio runtime cannot be created or the AWS configuration cannot be loaded.

Source

pub fn from_client( client: Client, location: S3Location, ) -> Result<Self, StoreError>

Builds a store from an already-configured SDK Client and a parsed location, owning a fresh tokio runtime for the sync bridge. Intended for tests (e.g. wiring a client at an emulator endpoint).

§Errors

StoreError::Backend if the tokio runtime cannot be created.

Source

pub fn with_meter(self, meter: Option<Arc<Meter>>) -> Self

Attaches (or clears) an optional progress Meter, rides alongside config. The transfer paths record bytes-in / bytes-out + per-object progress into it; None (the constructor default) means zero recording and byte-identical behavior. The CLI sets this after construction.

Source

pub fn location(&self) -> &S3Location

The parsed bucket/prefix this store targets.

Source

pub fn transfer_config(&self) -> &TransferConfig

The TransferConfig (concurrency / bandwidth) this store was built with. Consumed by the transfer loops in later gates.

Trait Implementations§

Source§

impl Store for S3Store

Source§

fn get_manifest(&self, id: &str) -> Result<Manifest, StoreError>

Reads and parses the manifest stored under id’s sharded path, verifying that its bytes hash back to id before returning it. Read more
Source§

fn fetch_files( &self, manifest: &Manifest, dest: &Path, ) -> Result<(), StoreError>

Materializes every entry of manifest under dest, pulling each referenced object from the store and reconstructing the directory tree (files, directories, permissions) rooted at dest. Read more
Source§

fn push(&self, manifest: &Manifest, source: &Path) -> Result<(), StoreError>

Uploads the objects referenced by manifest (reading their bytes from the tree rooted at source) and then the manifest itself, filing each under its sharded address. Read more
Source§

impl StreamStore for S3Store

Source§

fn has_object(&self, checksum: &str) -> Result<bool, StoreError>

Returns true if an object with this content-address already exists in the store. Read more
Source§

fn get_object(&self, checksum: &str) -> Result<Vec<u8>, StoreError>

Reads the raw object blob filed under checksum, verifying its bytes hash (BLAKE3) back to checksum before returning them. Read more
Source§

fn put_object(&self, checksum: &str, bytes: Vec<u8>) -> Result<(), StoreError>

Writes a raw object blob at its content-address, verifying bytes hash (BLAKE3) to checksum before storing anything. Read more
Source§

fn put_manifest(&self, id: &str, manifest: &Manifest) -> Result<(), StoreError>

Writes the manifest object for id, verifying the manifest’s bytes hash back to id before storing it. Read more
Source§

fn objects_needed( &self, checksums: &[String], ) -> Result<Vec<String>, StoreError>

Returns the subset of checksums NOT present in the store, PRESERVING INPUT ORDER. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> FutureExt for T

Source§

fn with_context(self, otel_cx: Context) -> WithContext<Self>

Attaches the provided Context to this type, returning a WithContext wrapper. Read more
Source§

fn with_current_context(self) -> WithContext<Self>

Attaches the current Context to this type, returning a WithContext wrapper. Read more
Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> IntoRequest<T> for T

Source§

fn into_request(self) -> Request<T>

Wrap the input message T in a tonic::Request
Source§

impl<Unshared, Shared> IntoShared<Shared> for Unshared
where Shared: FromUnshared<Unshared>,

Source§

fn into_shared(self) -> Shared

Creates a shared type from an unshared type.
Source§

impl<L> LayerExt<L> for L

Source§

fn named_layer<S>(&self, service: S) -> Layered<<L as Layer<S>>::Service, S>
where L: Layer<S>,

Applies the layer to a service and wraps it in Layered.
Source§

impl<T> Pointable for T

Source§

const ALIGN: usize

The alignment of pointer.
Source§

type Init = T

The type for initializers.
Source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
Source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
Source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
Source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more