---
status: reviewed
last_updated: 2026-10-20
---
# Substrate composition: the shipped backend stack
## What this is
The architecture of the batteries-included backend — the
`alkgit-substrate` crate's implementation of the five backend traits
over alkstore + alkblobs (ADR-019, housed by ADR-025's two-crate
workspace; the traits it implements are specified in
[backend.md](backend.md)). Everything here is the *composition*: the
two substrate crates' surfaces woven into object storage, ref rows, the
reactive registry, and GC. The embedder's alternative — implementing
the traits over foreign storage — consumes only the protocol core
(`alkgit`, which carries no substrate at any feature level), not this
doc.
Evidence base: POC-O (objects/refs/pipelines,
`poco-findings.md`), POC-R (registry/reactivity, `pocr-findings.md`),
POC-X (replication, `pocx-findings.md`), POC-L (LFS, `pocl-findings.md`
+ [lfs.md](lfs.md)). The substrate crates in their current published
shapes (`alkblobs 0.2`, the alkstore family 0.1) are the reference.
Feature pin (a wave-6 wiring note, 2026-10-22): alkblobs' alkcall
surface rides its optional `ops` feature (`default = [mem, sqlite]` —
no alkcall by default; alkblobs 0.2.2's `ops` requires
`alkcall = "0.10.1"`, the review-013-fixed release). The
composition's reactive halves and the ops wiring
(§registry adapters, ADR-019 §3–5) consume that surface, so the
substrate manifest enables it when those land; the object-storage
core alone (ADR-018's derivation-family puts, §object storage below)
stays alkcall-free.
## Object storage: alkblobs as the git object store
- **Loose objects under the git-object derivation family**
(alkblobs ADR-018: registry bytes `0x03`/`0x04`; the stored preimage
is git's loose-object form — `<type> <len>\0` + content, header
*inside* the value; key = tag + git oid). Verification-on-get
re-derives under git's own oid function; dedup-by-construction
within the family; external-oid addressing is map-free
(`Key::from_bytes` over `[tag] + oid bytes` — family-explicit,
rejects-never-guesses). The odb layer owns the header it writes
(parses it back out on read; `stat` returns the *preimage* length,
the family semantic — alkblobs ADR-018 §3).
- **Object graph access via a custom gix `Find` fronted by the store**
(the POC-O `SubstrateOdb` pattern): implements
`gix_object::Find` + `Exists` and `gix_pack::Find`; the
location/entry/pack-offset legs return nothing (no packed objects) —
the pipeline's `FindExt` blanket impls pick up the rest. No disk odb
exists anywhere on the serving path.
- **Packs**: generation streams through the pack pipeline (ADR-004,
Find-generic throughout — the same composition POC-2 proved against
the disk odb). Two serving shapes, both validated:
1. *direct* — the generated pack streams through sideband, no
storage round-trip;
2. *through-store* — the pack is put once (the store's own tier
dispatch routes big packs to the large tier), then served through
`read_range` windows (per-window slice digests verified
mid-flight); a request-digest pack cache lets repeat fetches ride
the stored pack; releases/evictions are pins the next sweep
reclaims. *Large tier required for through-store serving* — a
kv-only shape rejects over-threshold packs.
- **Ingestion** per ADR-020 §2: index into a budgeted host working
area (thin bases through the substrate Find), unpack-to-loose into
the store (verification-on-put = oid re-derivation — corruption
cannot enter), **pin bridge** holding every object until the ref CAS
lands, full-graph connectivity walk over the substrate Find
(parents included; `gix-fsck`'s `Connectivity::check_commit` does
not walk parents — the owning implementation walks), working area
discarded. Pack mirroring / delta synthesis: designed-not-built
(optimization bucket).
- **The pin bridge's mechanism and failure semantics** (what POC-O
built; pinned so the composition cannot improvise): the bridge is the
**set of put-returned pin guards** — one `Pin` per object put, held
by the ingest flow across the ingest → CAS span. Releases are
explicit and ordered: guards commit (release) only after the ref-CAS
transaction commits; abort drops the guards. Crash/failure is safe
by construction: neither the ref CAS nor the registry row has landed
before the guards release, so a dropped-bridge crash leaves only
sweep-eligible orphans (and CAS-shared entries that other refs
already cover) — a failed push is a failed push, not partial repo
state. The put side uses the store's **batch form** (`put_in_batch`
— pin-before-publish ordering inherited from the store's own
contract) so the working set lands with one commit discipline. The
pin's durability is the process's: alkblobs pins are in-memory
custody; a crashed ingest's entries are reclaimed by the *next*
sweep after process restart — recovery is GC-decreed, not
session-restored.
- **Pin custody across the frozen trait seam**: the pinned trait
signatures carry no guards (`prepare` returns `PreparedPush`; the
wire layer then calls `apply_updates`), so the composition owns the
custody internally — the substrate impls of `GitPackIngest` and
`GitRefs` are constructed together and share a composition-internal
**pending-pin registry**: `prepare` retains the just-put objects'
guards there, keyed by `repo_id` and correlated by content with the
prepared `RefUpdate` list it produced (content-addressed
correlation: each apply drains the pending entry whose update list
matches — identical update lists guard identical CAS requirements, so
which twin drains is immaterial); `apply_updates` drains the matching
entry and commits its guards only after the CAS transaction commits.
The normal failure path (prepare returned, apply never arrives —
client disconnect, malformed push between the calls) leaves a stale
pending entry: it is pruned on the repo's next `apply_updates`
(stale-entry sweep alongside the drain), and its guards otherwise
recover the same way the crash case does — process-lifetime custody,
reclaimed by the next sweep after restart. The cost of a leaked
guard is bounded (sweep-eligible objects held slightly longer), never
partial repo state.
- **Object ids are sha1** (OQ-05); the store's `GitObjectSha256`
family byte is registered — the sha256 policy change stays an
ecosystem-wait.
## Refs: the co-tenant row table
- Table shape: `(repo_id, name) → oid, symref_target`; HEAD is a
symref row (`RefLine`'s `oid: None` unborn shape). Name validation
per git ref rules + the reserved-namespace deny-list is impl-side
(ADR-013 §7's `gix_validate::reference` usage moves from gix-ref
files to the table layer).
- **HEAD seeding at record creation** (composition-internal, pinned so
clones are never checkout-less by construction): the substrate's
`put_repo` seeds the repo's `HEAD → refs/heads/main` symref row in
the same row-write transaction as the registry record (git parity —
`git init --bare` creates the (dangling) HEAD at init, not at first
push; git's push grammar carries no HEAD, and the op layer is
core-side, so the record-write is the one moment the composition owns
both surfaces). V1 pins the target `refs/heads/main`; a constructor
knob is additive later (two-way — the row is CAS-rewriteable). The
dangling symref is exactly the unborn-HEAD shape `ls-refs=unborn`
serves (OQ-19's rider (a) fixture falls out of this rule).
- `apply_updates` = all CAS checks + applies inside **one
transaction** (`BEGIN IMMEDIATE`-class on sqlite): stale/atomic
rollback/delete/create/force verified by POC-O against real `git
push` behaviors. This is what makes `atomic` correct (ADR-013 §7's
one-transaction home, now row-implemented).
- No file locks, no gix-ref file storage, no HEAD symlink mechanics —
the last disk dependency of the serving path is gone (ADR-019 §2).
## The registry: co-tenant tables behind the family adapter pattern
The alkstore-shipped non-goal is consumer row storage (alkstore's
`TxHandle` carries no consumer SQL), so alkgit owns its tables and
connections, co-tenant on the same engine instance (alkblobs ADR-013
§3's composition posture) with the **family adapter pattern** (the
alkcall ownership template — POC-R's validated shape):
- **Read side (sync)**: `resolve` — the advertisement path's hot read —
rides the **cached provider** below; a miss on the full-table
snapshot is a *definitive* unresolved id (the snapshot is the whole
table, so there is no fall-through I/O and no partially-populated
cache); the unknown-repo ≡ unauthorized rule (ADR-008) is evaluated
at resolve's caller, exactly as before.
- **Write side (async, `#[async_trait]`)**: `put`/`update`/`remove` —
each bumps the monotone `registry_seq` counter inside the row-write
transaction (transactionally exact; the watermark's anchor), then
announces post-commit (the reactive halves).
- **In-memory default adapter**: the batteries-included degenerate
case — always-populated, always-correct, single-process; the
completeness shape in the crate's base (no engine dependency — no
`mem` feature exists; ADR-025's placement rule).
- **Engine twins**: one trait pair, two implements — the sqlite
adapter (`sqlite` feature, `alkstore-sqlite` + the co-tenant
rusqlite connection) and the postgres twin (`postgres` feature).
The reactive composition code is engine-generic over the alkstore
`Store` trait and ran unchanged over all three engines (POC-R).
- **Schema migrations (ADR-024)**: both engine adapters migrate their
co-tenant tables at construction — ordered monomorphic steps per
twin, one transaction per step, concurrent-open-safe (the alkstore
race-verification pattern), loud failure. One alkgit-owned stamp
table versions the co-tenant schema as a unit; additive-first is the
post-publish discipline (an unknown-forward stamp is safe to open
against; a non-additive step is major-version + its own ADR). The
in-memory adapter builds fresh every construction — stamp-free.
### The reactive halves (wired by default — ADR-019 §5)
Per record write, after the durable commit, independently and
fail-soft:
1. **Publish** → the durable change feed (alkstore stream, keyed by
repo id) — per-consumer durable cursors, monotone saves, replay
from head; *per-seq idempotency is the consumer's rule* (keyed
publishes are metadata, not partitioning/dedup — the alkstore
contract).
2. **Notify** → the wake hint (alkstore `notify`/`listen`) — the
invalidation trigger for every cached provider.
3. **Enqueue** → the maintenance queue — housekeeping jobs
(sweep/pack work) riding the scheduler chain.
**The reconcile sweep** closes any lost post-commit effect:
`rows_since(applied_watermark)` → snapshot install → watermark
advance — idempotent by construction, driven by the scheduler tick
(and callable on demand). The crash-window shape (row committed,
effects suppressed) closed in one pass when demonstrated (POC-R).
**The per-push watermark leg**: pushes do not modify `RepoRecord`, so
the receive flow anchors the event/watermark legs with a
composition-internal per-repo mutation-marker write (the "registry
put" of POC-R's receive-flow recipe; ADR-019 §7) — this is what
drives diff-driven replication's prompt passes without record-change
noise.
**The sqlite/postgres asymmetry is designed-in** and carried by both
the composition and the docs:
- **sqlite**: the co-tenant file is itself the invalidation bus —
alkstore's watcher `data_version` poll fans a wake to listeners on
*any* commit to the file, including alkgit's own registry writes on
their own connection, including cross-process (~3.2 ms measured for
an unannounced co-tenant write). A lost hint cannot exist while any
write lands; the reconcile sweep is a correctness backstop, not a
latency bound.
- **postgres**: no file-level bump exists; LISTEN/NOTIFY needs the
explicit post-commit notify, and a lost/failed hint leaves caches
stale until the reconcile tick — **the tick's cadence is the real
staleness bound on postgres**.
### The coherence contract (the cached read provider; ADR-019 §4)
- **Cached set**: the whole record table as one immutable snapshot
(an `Arc` map swapped under a short lock; whole-table by contract —
ADR-019 §4).
- **Hot path**: one lock read + `Arc` clone + lookup; zero I/O on the
steady state.
- **Invalidation**: alkstore wakes (the postgres notify hint or the
sqlite file-wide bump), the provider's own writes (write-through),
the reconcile tick.
- **Staleness bound**: mutation visibility ≤ one wake delivery + one
refresh; on sqlite milliseconds (measured); on postgres the
reconcile cadence after a lost hint.
- **Failure posture**: refresh failures leave the previous snapshot
serving; the next wake, write, or tick retries.
## GC: the wired maintenance chain
The substrate composition wires GC as a chain of alkstore mechanisms
feeding alkblobs (POC-R's validated cross-process run):
```
schedule(@every) → alkstore leader-elected scheduler
→ firing lands in the maintenance queue
→ worker claim (visibility-timeout discipline)
→ named-TTL leader lock ("sweeper" — cross-process exclusion;
a locked-out sweep is a skip + short retry, a value not an error)
→ alkblobs Store::sweep() -> Result<SweepReport, Error>
(the report is the GC outcome carrier — stages, deletes,
aborts; `Err` is substrate-level failure, not a GC verdict)
```
- **Liveness sources (alkgit-owned, in `alkgit-substrate`'s base)**:
1. the *ref-table closure* — every ref's full reachable graph walk
over the substrate Find (this is what makes orphaned and
delete-uncovered objects reclaimable: liveness, not timestamps,
decides);
2. the *LFS media rows* under the `lfs` feature (ADR-022 §4 —
cross-reference sharing requires re-protection while any row
references a key; the refcount column ships in the initial
schema — ADR-024 §5).
- Sweep-window semantics are alkblobs' (stage → delete across the
sweep window); pins from an in-flight push hold objects across the
window (the pin bridge, ADR-020 §2).
- Repo deletion (the op or replication-side) removes registry + ref
rows; the store's sweep reclaims the uncovered objects — no bulk
delete exists.
## The receive-pack flow, composed
The full push path, as the pieces above compose (each durable surface
in its own transaction; no cross-store machinery):
```
pack stream → budgeted index (thin bases via substrate Find)
→ unpack-to-loose into alkblobs (pinned; ADR-020 §2)
→ connectivity walk (full closure)
→ ref-table CAS (one transaction; ADR-019 §2)
→ per-repo mutation-marker write (the composition-internal row that
bumps the seq/watermark on every push — pushes do not touch
`RepoRecord`; ADR-019 §7)
→ post-commit announce: publish + notify + enqueue (fail-soft each)
→ band-1 status report
```
- Crash windows: pinned objects survive sweeps until the CAS lands
and pins release (the object window; POC-O's verified closure); the
reconcile sweep closes the event window (POC-R's recipe).
- The bounded pack-header peek (packless deletes — ADR-020 §1) sits at
the wire layer, *before* this flow; packless pushes enter at the
ref-CAS step.
- Client-side (the consumer half's push contract): a packless push
keeps its write side open until the report — half-closing turns the
server's bounded peek into an EOF error instead of the packless
timeout (POC-X's measured fact; ADR-020 §3). `GitSession`'s push
must hold the stream open across the report wait.
## Assembly shapes (the constructor-level matrix)
Engine choice is one assembly-level statement (alkstore ADR-016's
deployment honesty) — three shapes, zero assembly code:
| Single-host (the minimal real setup) | `alkstore-sqlite` (`open(path, SqliteOpts)`) | sqlite shapes (`kv_only_sqlite`; `dual_tier_sqlite_local` when LFS or through-store packs) | sqlite twin |
| Fleet / durable-shared | `alkstore-postgres` (`PgStore`/`PgOpts`, schema namespaced) | postgres shapes (kv-only / +dual-tier local / `dual_tier_postgres_pglo`) | postgres twin |
| Ephemeral / tests / completeness | `alkstore-mem` | mem shapes (`mem_only`, `kv_only_mem`, `dual_tier_mem_*`) | in-memory default adapter |
- **The sqlite+pg-lo combination is unconstructible**, not merely
discouraged (their constructors have no such shape; a sqlite fleet
is a recorded alkblobs non-option).
- **`:memory:` sqlite confers no co-tenancy** (per-connection
databases; alkblobs' sqlite rejects memory mode — WAL cannot
engage): when the whole assembly is ephemeral, compose the mem
engines, never sqlite-in-memory (DQ-7's ruling).
- **Connection budget** (the worst-case assembly datum): alkstore
postgres pool + its dedicated LISTEN connection; alkblobs kv pool
(+ a second pool in pg-lo dual shapes); alkgit's registry
connection — four pools per process, documented here so an assembly
sizes `max_size` with it in view.
- **Tuning datums for sqlite assemblies** (POC-O/POC-R measurements;
assembly-level knobs, not structure): the 1 ms watcher poll cadence
wakes on any commit (file-wide fanout; self-traffic machinery ~
5 refreshes/s at defaults — raise the cadence on busy assemblies);
single-writer serialization across all co-tenant connections —
receive-pack latency measured indistinguishable with the full
machinery co-tenant (5 ms medians); `SqliteOpts::{max_readers,
poll_interval}` are the knobs; `busy_timeout` is pinned (5000 ms —
contracted at both substrate crates since the wave-14 ask landed).
## Public API surface (composition-shaped types)
Crate-root re-exports of the `alkgit-substrate` crate (backend.md
§public API carries the per-crate partition) — this list is the
freeze inventory ADR-025 names:
- the registry adapter trait pair + engine constructors + the in-memory
default adapter;
- the cached read provider; the reactor (announce/wake-runner/stream-
consumer/reconcile) — construction wired by a single composition
entry that takes the substrate engine handles (the assembly-level
one-statement engine choice);
- **the five backend-trait implementations** — the registry pair's
concrete adapters plus the object-side trio (`GitRefs`/`GitPackGen`/
`GitPackIngest` impls): public by necessity, the assembler constructs
the backend from them (public API = authorized-surface rule: the op
and session layers reach them only through the core's traits);
including the substrate `Find`/odb front (`SubstrateOdb`-class —
constructed once, per-session views; the handle-sharing shape);
- the receive-pack composition (unpack-to-loose + pin bridge + the
pending-pin registry) — **internal** to the trait impls, not a
public member;
- the GC liveness sources; the LFS liveness/pointer-lifecycle surface
under `lfs`.
The exact Rust names are the implementation task's to pin under OQ-03's
inventory (this doc fixes the *members*, not the identifiers).
## Design Decisions
| [019](decisions/019-substrate-baseline.md) | Substrate re-baseline | objects/refs/registry on the substrate; reactive-by-default; coherence contract; GC wiring |
| [020](decisions/020-receive-pack-duplex-amendments.md) | receive-pack duplex | packless deletes (bounded peek); unpack-to-loose + pin bridge |
| [021](decisions/021-feature-graph-and-dependency-posture.md) | Feature graph | **superseded by ADR-025** (graph re-partitioned) |
| [022](decisions/022-lfs.md) | LFS | identity-family media keying; alkhttp door; wired liveness rows |
| [025](decisions/025-two-crate-workspace.md) | Two-crate workspace | this composition is `alkgit-substrate`; the placement rule |
| [023](decisions/023-replication-v1.md) | Replication v1 | protocol-native via `GitSession`; this composition's stream/watermark recipes carry it |
| [004](decisions/004-pack-pipeline.md) | Pack pipeline | gen/ingest shapes (Find-generic, unchanged in pipeline) |
| [013](decisions/013-receive-pack-state-machine.md) | receive-pack | the state machine (§3/§6 amended: ADR-020) |
| [018](decisions/018-backend-trait-signatures-and-storage-error-model.md) | Trait signatures | the seam this composition implements (zero-drift record) |
## Open Questions
- **OQ-19**: implementation-phase wire/odb validations (unborn-HEAD
rider; `common_haves` over the substrate odb — POC-X's NAK-only
caveat).
- OQ-05 (sha256) rides alkblobs' registered family byte; unchanged.
- OQ-17 resolved: schema migrations are construction-run,
version-stamped, additive-first (ADR-024 — the registry section
above carries the composition text).
## References
- [backend.md](backend.md) (the trait seam + schemas this implements),
[lfs.md](lfs.md), [transport.md](transport.md),
[doors.md](doors.md) (assembly section)
- `docs/research/poco-findings.md`, `pocr-findings.md`,
`pocx-findings.md`, `pocl-findings.md` — the four evidence records
- `docs/research/substrate-recomposition.md` (the re-baseline doc; the
DQ ledger)
- ADR-019..023 (the re-baseline ADR package — all rationale)
- alkstore ADR-016 (deployment honesty — the assembly-level engine
statement); alkblobs ADR-013 (§3 co-tenancy composition contract,
§4 feature-graph retirement), ADR-017 (CAS global-pool caveat),
ADR-018 (the git-object derivation family), ADR-019/020 (mint
admission cap; pre-keyed wire put) — sibling-crate decisions, always
cited with their crate prefix in this doc set