alkblobs 0.2.0

Content-addressed blob storage: multi-hash put/get/verify store with pluggable backends, transport- and protocol-agnostic
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129

//! `alkblobs` — content-addressed blob storage in the alk* family: a
//! put/get/verify store keyed under the git-family oid derivations
//! (each address formed under a caller-named algorithm — canonical
//! `"blob <len>\0" + content` under SHA-256, legacy SHA-1 tolerated,
//! the git-object family's preimage-identity members — ADR-002, ADR-018,
//! ADR-020), with tier engines underneath and no wire/protocol layer
//! welded on.
//!
//! ## Layer map
//!
//! - **[`store::Store`]** — the facade consumers program against:
//!   `put_as`/`get`/`stat`/`read_range`/`has`/`list`, the [`Batch`]
//!   write seam, and the GC protocol (`sweep`, direct `delete`,
//!   liveness registration). Engine-blind: consumers name engines only
//!   through named constructors. The size-threshold routing between a
//!   store's kv tier and its large tier is the pure function in
//!   [`dispatch`], threshold [`DEFAULT_DISPATCH_THRESHOLD`].
//! - **Engines** — one module per deployment shape: [`local`] (the
//!   filesystem large tier, always present) and the feature-gated
//!   [`mem`] (feature `mem`, default) and [`sqlite`] (feature
//!   `sqlite`, default); the postgres shapes (`postgres`, fleet CAS;
//!   `pg-lo`, large-object tier) are gated but not hyper-linked here —
//!   they resolve only under their feature's doc run. Engines implement the crate-internal
//!   [`backend`] trait; they are never named in the public surface.
//! - **`ops`** (feature `ops`) — the network ops surface (verified
//!   fetch, put, JSON control ops) riding the alkcall substrate;
//!   authorization rides alkcall's `AccessControl` seam, not an
//!   in-band scheme.
//! - **Types** — [`Key`] and [`Digest`]/[`Algorithm`] (the addressing
//!   vocabulary), [`EntryMeta`] and the facade carriers in [`facade`],
//!   [`Error`] (the error enum), the GC-side carriers in [`gc`].
//!
//! ## Substrate-agnostic posture
//!
//! The store never knows where bytes came from. `Store::put_as` names
//! the derivation and takes a stream and the optional length; the ops
//! layer, a local writer, and a network offer are all just callers.
//! Transport, framing, and serialization live above this crate (in
//! `ops`, gated off by default) or nowhere.
//!
//! ## Construction
//!
//! No `new`: the `Store` is built through named constructors, each
//! declaring its tier shape (see [`StoreMode`], three ADR-pinned
//! variants). The mem shapes — [`Store::mem_only`],
//! [`Store::kv_only_mem`], [`Store::dual_tier_mem_local`] — are the
//! feature-`mem` baseline; the sqlite pair ([`Store::kv_only_sqlite`],
//! [`Store::dual_tier_sqlite_local`]) is the default build's durable
//! shape; the postgres constructors (feature `postgres`) add the
//! fleet-shared variants (`*_fleet`, over a `FleetSharedMedia::asserted`
//! declaration); the pg-lo pairings
//! (feature `pg-lo`) pair a kv tier with the large-object tier.
//! [`Store::mode`] reports what was built.
//!
//! ## Spec and ADR record
//!
//! This crate's contract lives in `docs/architecture/` within the
//! repository: `store-api.md` is the facade contract this rustdoc
//! paraphrases, `overview.md` the system orientation, and
//! `decisions/` the ADRs the prose cites by number. Where this doc
//! and the specs disagree, the specs win.
//!
//! The features above compose: `postgres` and `pg-lo` are independent
//! (both pull the tokio-postgres pool), `ops` composes with anything.

pub mod backend;
pub mod dispatch;
pub mod error;
pub mod facade;
pub mod gc;
pub mod hashing;
pub mod keys;
pub mod local;
pub mod store;

pub use dispatch::DEFAULT_DISPATCH_THRESHOLD;
pub use error::Error;
pub use facade::{
    Batch, EntryMeta, GcAbortCause, LivenessRegistration, LivenessSource, Pin, PinState,
    ProtectCallback, Registration, SweepReport,
};
pub use hashing::{Algorithm, Digest};
pub use keys::{Key, KeyBytes};
pub use store::{Store, StoreMode, DEFAULT_DELETE_WINDOW_DELAY};

#[cfg(feature = "postgres")]
pub use store::{FleetSharedMedia, DEFAULT_PIN_TTL};

#[cfg(feature = "mem")]
pub mod mem;

#[cfg(feature = "pg-lo")]
pub mod pg_lo;

#[cfg(feature = "ops")]
pub mod ops;

#[cfg(feature = "ops")]
pub use ops::{
    delete_pair, fetch_openables, have_pair, put_openables, stat_pair, FanoutChunk,
    FanoutPublisher, FanoutReceiver, FanoutRegistry, OpsConfig, OpsContext, PinConverter, PinToken,
    RenewalOutcome, SlowSubscriberPolicy, TokenRegistry, BLOBS_CHANNEL_ALPN, BLOB_MISSING,
    DIGEST_MISMATCH, FETCH_OP_ID, GC_REFUSED, INVALID_DIGEST, INVALID_RENEWAL_FORM, PUT_OP_ID,
    PUT_REFUSED, VERIFICATION_REFUSED,
};

/// Fuzz/test-only surface: the ranged-fetch bounds ingress
/// (`fetch_ranges_ingress`) and its data carriers (`FetchRange`,
/// `RangedBounds`), exposed for the ops-ingress fuzz target's drive of
/// the exact production function — not contract surface, not consumer
/// API, no stability guarantee.
#[cfg(feature = "ops")]
#[doc(hidden)]
pub use ops::{fetch_ranges_ingress, FetchRange, RangedBounds};

#[cfg(all(test, feature = "ops"))]
mod fuzz_surface_tests;

#[cfg(any(feature = "postgres", feature = "pg-lo"))]
mod pg_common;

#[cfg(feature = "postgres")]
pub mod postgres;

#[cfg(any(feature = "postgres", feature = "pg-lo"))]
pub use pg_common::PgOpts;

#[cfg(feature = "sqlite")]
pub mod sqlite;