Struct ObjectStoreBackend 

pub struct ObjectStoreBackend { /* private fields */ }

Object storage backend with optional local caching.

Delegates storage operations to a cloud object store (S3, GCS, Azure) via the object_store crate. When a DiskCache is configured, reads are served from the local cache when available.

Segment Path Mapping

SegmentPath { namespace: "default", shard_id: 3, name: "seg_001.csx" } maps to the object key ns_default/shard_3/seg_001.csx relative to the configured prefix.

Implementations

impl ObjectStoreBackend

pub async fn new(config: ObjectStoreConfig) -> Result<Self>

Create a new object store backend from configuration.

Errors

Returns InvalidConfig if the URL scheme is not supported.

pub fn from_store(store: Arc<dyn ObjectStore>, cache: Option<DiskCache>) -> Self

Create a backend from an existing ObjectStore instance.

Useful for testing with in-memory stores. Create a backend from an existing ObjectStore instance.

Uses the default multipart threshold (8 MiB).

pub fn from_store_with_threshold( store: Arc<dyn ObjectStore>, cache: Option<DiskCache>, multipart_threshold_bytes: usize, ) -> Self

Create a backend from an existing ObjectStore with a custom multipart threshold.

pub async fn segment_size(&self, path: &SegmentPath) -> Result<Option<usize>>

Return the size of a remote segment via a HEAD request.

This avoids re-downloading the entire segment just to verify its size after upload. Returns None if the object does not exist.

Errors

Returns an error if the HEAD request fails (other than NotFound).

pub async fn object_size(&self, key: &str) -> Result<Option<usize>>

Return the size of an arbitrary object via a HEAD request.

None means the object is not there. Used to verify an upload landed intact before the caller deletes the source.

Errors

Returns an error if the HEAD request fails for any reason other than NotFound.

pub async fn put(&self, key: &str, data: Bytes) -> Result<()>

Put an object into the store.

Errors

Returns ObjStoreError::Store on object store failures.

pub async fn get(&self, key: &str) -> Result<Bytes>

Get an object from the store.

Errors

Returns ObjStoreError::NotFound if the object does not exist, or ObjStoreError::Store on other failures.

pub async fn delete(&self, key: &str) -> Result<()>

Delete an object from the store.

Errors

Returns ObjStoreError::Store on object store failures.

pub async fn list(&self, prefix: &str) -> Result<Vec<String>>

List objects under a prefix.

Errors

Returns ObjStoreError::Store on object store failures.

Trait Implementations

impl Debug for ObjectStoreBackend

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl StorageBackend for ObjectStoreBackend

async fn put_segment(&self, path: &SegmentPath, data: &[u8]) -> Result<()>

Write a complete segment file to storage.

async fn get_segment(&self, path: &SegmentPath) -> Result<Vec<u8>>

Read a complete segment file from storage.

async fn get_range( &self, path: &SegmentPath, offset: u64, length: usize, ) -> Result<Vec<u8>>

Read a byte range from a segment file.

async fn delete_segment(&self, path: &SegmentPath) -> Result<()>

Delete a segment file from storage.

async fn list_segments( &self, namespace: &NamespaceId, shard_id: &ShardId, ) -> Result<Vec<SegmentPath>>

List all segment files in a namespace/shard.

async fn exists(&self, path: &SegmentPath) -> Result<bool>

Check whether a segment file exists.