Crate objstore_r 

objstore-r

Object stores with generation preconditions, for data that is published as immutable snapshots plus append-only segments (graph-r, link-r).

Every object carries a generation (a u64, never zero) that changes on each write. Writes take a Cond:

• Cond::CreateOnly: succeed only if the key does not exist. Immutable objects (snapshots, log segments) are written this way, so a retry after a lost response is harmless and a second writer can never clobber the first.
• Cond::IfGeneration: succeed only if the object is still at that generation. A small HEAD object updated this way gives single-writer compare-and-swap across machines, with no lock service.

Backends

Backend	Feature	Use
LocalDir	always	a directory; atomic rename, fsync, File::lock for conditions
Mem	always	tests; deterministic, with scripted fault injection
Gcs	gcs	Google Cloud Storage (JSON API), ADC tokens, retries, ranged reads

Design tenets

• Named futures, no boxing. ObjectStore methods return associated future types (GATs), so callers compose them without Box<dyn Future> and without Send/'static bounds imposed by this crate. Futures are Send whenever the backend is, so multi-threaded runtimes work too.
• Enums over dyn. Choose a backend at compile time, or wrap several in your own enum; nothing here needs a trait object.
• Zero-copy bodies. Objects are bytes::Bytes; a body is read or written once and shared by reference count from there on.
• No unsafe code.

Content encoding

Opt in per object with ObjectStore::put_encoded (feature gzip): the body is stored compressed and get decodes it once into the returned buffer, bounded against decompression bombs; a ranged read of an encoded object is refused (Error::EncodedRange). ObjectStore::stat and ObjectStore::list_stat report the encoding. See encoding.

Example

use objstore_r::{Cond, Mem, ObjectStore};
let store = Mem::default();
let g1 = store.put("snap/0001", "v1".into(), Cond::CreateOnly).await?;
assert!(store.put("snap/0001", "again".into(), Cond::CreateOnly).await.is_err());
let head = store.put("HEAD", "0001".into(), Cond::None).await?;
store.put("HEAD", "0002".into(), Cond::IfGeneration(head)).await?;       // CAS
assert_eq!(store.get("snap/0001").await?.unwrap().bytes, "v1");

Re-exports

pub use encoding::ContentEncoding;

pub use encoding::MAX_INFLATED_BYTES;

pub use error::Error;

pub use error::PolicyError;

pub use error::Result;

pub use key::Key;

pub use local::LocalDir;

pub use mem::Fault;

pub use mem::Mem;

pub use gcs::Gcs;

pub use gcs::GcsConfig;

Modules

encoding

Per-object content encoding (Content-Encoding), opt in per write.

error

Errors.

fenced

Fenced single-writer publishing, the protocol every snapshot publisher (crate::mirror for graph-r stores, link-r’s remote for indexes) shares:

gcs

Google Cloud Storage over the JSON API (feature gcs).

http

Client: the reqwest-backed Transport (feature http).

join

Executor-free concurrency: join_all drives many futures on the caller’s task, and spin_on runs a future that never waits on real I/O (tests over in-memory stores and scripted transports). Runtime-driven I/O from a synchronous main is http::block_on (feature blocking).

key

Object keys.

local

A directory as an object store.

mem

An in-memory store with scripted fault injection, for tests.

mirror

graph-r-mirror

net

Network policy shared by every HTTP client of the stack (the GCS store, the authorised transport, link-r’s crawler): which statuses are transient, backoff timing, Retry-After, endpoint rules and error mapping. One implementation, so all clients behave identically.

policy

Request policies for crate::http::Client (feature http).

sse

Incremental server-sent-events decoder, for streaming responses read through crate::transport::BodyStream.

testing

A conformance suite for ObjectStore implementations, and the harness every live test of the stack shares (feature testing).

token

OAuth access tokens for Google APIs (feature gcs).

transport

A minimal authorised-HTTP seam for Google REST APIs beyond Cloud Storage (Vertex AI, BigQuery), so dependants reuse this crate’s TLS stack, token providers, retry policy and size caps instead of carrying their own HTTP client.

Structs

Bytes

A cheaply cloneable and sliceable chunk of contiguous memory.

Meta

A listed object.

Object

An object’s content and generation.

Stat

Meta and the object’s content encoding, from ObjectStore::stat and ObjectStore::list_stat.

Enums

Cond

A write precondition.

Traits

ObjectStore

An object store with generation preconditions.