Trait ObjectStore

Source

pub trait ObjectStore: Send + Sync {
Show 32 methods    // Required methods
    fn get_blob(&self, hash: &ContentHash) -> Result<Option<Blob>>;
    fn put_blob(&self, blob: &Blob) -> Result<ContentHash>;
    fn has_blob(&self, hash: &ContentHash) -> Result<bool>;
    fn get_tree(&self, hash: &ContentHash) -> Result<Option<Tree>>;
    fn put_tree(&self, tree: &Tree) -> Result<ContentHash>;
    fn has_tree(&self, hash: &ContentHash) -> Result<bool>;
    fn get_state(&self, id: &ChangeId) -> Result<Option<State>>;
    fn put_state(&self, state: &State) -> Result<()>;
    fn has_state(&self, id: &ChangeId) -> Result<bool>;
    fn list_states(&self) -> Result<Vec<ChangeId>>;
    fn get_action(&self, id: &ActionId) -> Result<Option<Action>>;
    fn put_action(&self, action: &mut Action) -> Result<ActionId>;
    fn list_actions(&self) -> Result<Vec<ActionId>>;
    fn list_blobs(&self) -> Result<Vec<ContentHash>>;
    fn list_trees(&self) -> Result<Vec<ContentHash>>;

    // Provided methods
    fn blob_size(&self, hash: &ContentHash) -> Result<Option<u64>> { ... }
    fn loose_blob_path(&self, _hash: &ContentHash) -> Option<PathBuf> { ... }
    fn promote_to_loose_uncompressed(&self, _hash: &ContentHash) -> Result<bool> { ... }
    fn put_blob_with_hash(
        &self,
        blob: &Blob,
        hash: ContentHash,
    ) -> Result<ContentHash> { ... }
    fn put_blob_bytes_with_hash(
        &self,
        data: &[u8],
        hash: ContentHash,
    ) -> Result<ContentHash> { ... }
    fn put_tree_serialized(
        &self,
        data: &[u8],
        hash: ContentHash,
    ) -> Result<ContentHash> { ... }
    fn put_state_serialized(&self, data: &[u8], id: ChangeId) -> Result<()> { ... }
    fn put_action_serialized(&self, data: &[u8], id: ActionId) -> Result<()> { ... }
    fn get_pack_object(
        &self,
        id: &PackObjectId,
    ) -> Result<Option<(ObjectType, Vec<u8>)>> { ... }
    fn put_blobs_packed(&self, blobs: Vec<(ContentHash, Vec<u8>)>) -> Result<()> { ... }
    fn install_pack(
        &self,
        pack_data: &[u8],
        index_data: &[u8],
    ) -> Result<Vec<PackObjectId>> { ... }
    fn install_pack_streaming(
        &self,
        pack_path: &Path,
        index_path: &Path,
    ) -> Result<()> { ... }
    fn pack_objects(&self, aggressive: bool) -> Result<(u64, u64)> { ... }
    fn prune_loose_objects(&self) -> Result<(u64, u64)> { ... }
    fn begin_snapshot_write_batch(&self) -> Result<()> { ... }
    fn flush_snapshot_write_batch(&self) -> Result<()> { ... }
    fn abort_snapshot_write_batch(&self) { ... }
}

Expand description

Trait for object storage backends.

Required Methods§

Source

fn list_trees(&self) -> Result<Vec<ContentHash>>

Provided Methods§

Source

fn blob_size(&self, hash: &ContentHash) -> Result<Option<u64>>

Return the uncompressed byte length of the blob identified by hash, or Ok(None) when the blob is not in the store.

The contract is “size without paying for content”: backends are expected to honour this with a header read or index lookup rather than a full decompression. This is the hot path for directory listings (ls -l over a thread mount) where loading every blob just to learn its size would dominate.

The default implementation falls back to get_blob so backends without a cheap size accessor still satisfy the contract; native stores (FsStore, InMemoryStore) override this with a header- or hashmap-only path.

Source

fn loose_blob_path(&self, _hash: &ContentHash) -> Option<PathBuf>

Filesystem path of the loose blob whose on-disk bytes are byte-identical to the blob’s uncompressed content, suitable for hard_link/clonefile materialization without going through get_blob.

Returns None when the blob is missing, is only available via a packfile, is stored compressed (the on-disk bytes wouldn’t match what a worktree consumer needs to read), or the backend doesn’t expose stable filesystem paths (e.g. InMemoryStore, S3Store). The default impl returns None so non-FsStore backends silently fall through to the bytes path.

Source

fn promote_to_loose_uncompressed(&self, _hash: &ContentHash) -> Result<bool>

Ensure the blob identified by hash is materialized as an uncompressed loose file at the canonical loose path so that loose_blob_path returns Some(path) on a subsequent call.

This is the “warm canonical store” path that lets the hardlink-first materializer keep its 5–10× wall-clock and storage-allocation wins after pack_objects + prune_loose_objects has moved everything into a packfile. Without this, the lazy hardlink path silently degrades to fs::write(decompressed) on every materialize, because loose_blob_path returns None for pack-only and compressed-loose blobs.

Cost-amortization: the first promotion of a blob pays decompress + atomic write. Every subsequent materialize of the same blob — into the same worktree on goto, or into a sibling worktree on delegate — is a single link(2). Net win for any N > 1 materializations; break-even at N == 1.

Pack invariants are preserved: this method does not remove the pack-resident copy. The blob lives in both pack and loose- uncompressed until the next prune_loose_objects cycle, at which point the loose mirror is discarded and a future materialize re-promotes on demand.

Idempotent: a blob that’s already loose-and-uncompressed is a no-op fast path. A blob that’s loose-but-compressed is rewritten in place (atomically) with the uncompressed bytes. A blob that’s pack-resident is decompressed out of the pack and written loose without touching the pack.

Returns Ok(true) when the call did real work (a write happened), Ok(false) when it was a no-op (blob was already loose+uncompressed), and Err when the blob isn’t in the store at all. The default impl returns Ok(false) for backends that don’t expose loose paths (InMemoryStore, S3Store), since the hardlink path is fundamentally inapplicable there.

Source

fn put_blob_with_hash( &self, blob: &Blob, hash: ContentHash, ) -> Result<ContentHash>

Source

fn put_blob_bytes_with_hash( &self, data: &[u8], hash: ContentHash, ) -> Result<ContentHash>

Source

fn put_tree_serialized( &self, data: &[u8], hash: ContentHash, ) -> Result<ContentHash>

Source

fn put_state_serialized(&self, data: &[u8], id: ChangeId) -> Result<()>

Source

fn put_action_serialized(&self, data: &[u8], id: ActionId) -> Result<()>

Source

fn get_pack_object( &self, id: &PackObjectId, ) -> Result<Option<(ObjectType, Vec<u8>)>>

Source

fn put_blobs_packed(&self, blobs: Vec<(ContentHash, Vec<u8>)>) -> Result<()>

Bulk-write a batch of blobs as a single durable unit. The default implementation falls back to per-blob writes; backends that support packfiles (i.e. FsStore) override this to install one packfile + index — two fsyncs total instead of N. Used by the snapshot hot path so writing 1000 small files takes ~one fsync, not 1000.

Blobs already present in the store are skipped on the way in (the caller would otherwise duplicate them in the pack).

Source

fn install_pack( &self, pack_data: &[u8], index_data: &[u8], ) -> Result<Vec<PackObjectId>>

Source

fn install_pack_streaming( &self, pack_path: &Path, index_path: &Path, ) -> Result<()>

Install a pack and its index from on-disk files (typically produced by StreamingPackBuilder). The default impl reads both files fully and delegates to install_pack, so any backend that doesn’t override this still works (at the cost of giving back the bounded-memory promise). Real fs- backed stores override this to rename(2) both files into the pack directory without ever loading them.

On success, the source files at pack_path/index_path may have been moved or removed depending on the backend; callers shouldn’t continue to rely on them.

Returns nothing — callers that need the list of installed ids can read the freshly-installed pack via the store. Most callers (including Importer) already track inserted ids out-of-band via the sha map and don’t need a return value.

Source