Expand description
The usage ledger (2026-10-06): every namespace’s stock meters, re-measured
only where something changed — what UsageBatch.ReadMany answers from.
The usage ledger: every namespace’s stock meters, kept current by
re-measuring only what changed (2026-10-06).
§Why
monetize’s poller asked Usage.Read once per tenant every 300 s, and every
one of those reads listed the WHOLE catalog (RepoCatalog::list is
O(repositories)) and walked the WHOLE repository root for tombstones
(ServerStores::tombstones). One poll was therefore O(tenants²) disk and
catalog work: MEASURED on the UpCloud mockup’s load test at ~25–30 s of 1.4
cores on main per period at 1,000 tenants, and on oden’s debug bench
(tests/usage_poll_scale.rs) 8.5 ms a call at 1,000 tenants — at ~10,000
tenants one poll took the whole period.
§What it keeps
Per namespace: its repositories (from the catalog, re-indexed only when the
catalog document’s VERSION changes — one pointer read per settle), each
repository’s last measured RepoBytes, the namespace’s tombstoned bytes,
the indexed objects its OPEN stores hold, and the sequence number of the last
time any of that — or its flow — changed. A read of a namespace nothing has
touched is a copy out of memory.
§What makes it re-measure (the marks)
A repository’s bytes can only change while its store is OPEN, or by a store-level operation on its directory (soft delete, purge, reclaim, a growth’s remount). So:
- opening a store marks the repository (
ServerStores::repo_store’s one open site), and every store that is open, or was open at the last look, is re-measured at the next settle (the open set is bounded by the handle cache, 256) — a push, a fetch that grows the exploded cache, a background index drain, a gc: all happen on an open store; - a transfer, a refusal or an auth failure marks the namespace through
the flow ledger’s observer (the
*_todaymeters moved; LFS bytes may have); - the catalog’s version changing re-indexes: a namespace or repository created or deleted is a change;
- tombstone operations mark their namespace; a growth’s resume marks everything;
- and the reconcile pass (
UsageLedger::reconcile_step, on the flow flush’s one-minute tick) re-measures a slice of namespaces every tick, so anything that moved bytes without a mark is corrected within one cycle and COUNTED (UsageLedger::drift_corrections) rather than silently absorbed.
§The cursor
<epoch>.<utc day>.<seq>. epoch is this ledger’s (a restart is a new
ledger and every old cursor reads as “everything”); the day is in it because
the *_today meters reset at midnight UTC without any event; seq is the
ledger’s change counter, taken BEFORE the read settles so a change that lands
during the read is answered again next time rather than lost.
Structs§
- Held
- One namespace’s stock meters as the ledger holds them.
- Usage
Ledger - See the module doc.
Constants§
- RECONCILE_
CYCLE - The reconcile pass’s cycle: every namespace re-measured at least this often.
Functions§
- tombstoned_
bytes - Bytes still held by this namespace’s soft-deleted repositories — the
tombstones directly in its own directory (namespaces nest, and a nested
namespace’s tombstones are its own). The walk
ServerStores::tombstonesmakes over the whole root, narrowed to one directory.
Type Aliases§
- Exact
UsageLedger::exact’s answer: the namespace’s total and its repositories’ bytes (name, bytes, store directory), sorted by name.- Page
UsageLedger::page’s answer: the rows (namespace, what it holds), whether there are more, and whether achanged_sincewas honoured.