Struct StageUploader 

pub struct StageUploader { /* private fields */ }

Uploads staged objects to an ObjectStore and cleans them up. The store is injected so tests drive it against object_store::memory::InMemory.

Implementations

impl StageUploader

pub fn new( store: Arc<dyn ObjectStore>, location: StagingLocation, ) -> StageUploader

Build an uploader over an explicit store (used in tests) + location.

pub fn from_location( location: StagingLocation, ) -> Result<StageUploader, FaucetError>

Build an uploader whose store is constructed from the location’s scheme + ambient credentials.

pub fn location(&self) -> &StagingLocation

The location this uploader stages into.

pub async fn stage_page( &self, spec: &StagingSpec, scope: &str, run_id: &str, seq: usize, records: &[Value], pre_encoded: Option<Vec<u8>>, ) -> Result<StagedFile, FaucetError>

Serialize + (optionally) compress + upload one page as a single object, returning its StagedFile. pre_encoded lets a sink pass already-serialized bytes (e.g. Parquet from Arrow); otherwise the records are serialized per spec.format.

pub async fn cleanup( &self, files: &[StagedFile], cleanup: StagingCleanup, succeeded: bool, )

Delete staged objects per the cleanup policy + whether the load succeeded. Best-effort: a delete error is logged, not propagated, so it never masks the load result.