Skip to main content

Module usage_ledger

Module usage_ledger 

Source
Expand description

The usage ledger (2026-10-06): every namespace’s stock meters, re-measured only where something changed — what UsageBatch.ReadMany answers from. The usage ledger: every namespace’s stock meters, kept current by re-measuring only what changed (2026-10-06).

§Why

monetize’s poller asked Usage.Read once per tenant every 300 s, and every one of those reads listed the WHOLE catalog (RepoCatalog::list is O(repositories)) and walked the WHOLE repository root for tombstones (ServerStores::tombstones). One poll was therefore O(tenants²) disk and catalog work: MEASURED on the UpCloud mockup’s load test at ~25–30 s of 1.4 cores on main per period at 1,000 tenants, and on oden’s debug bench (tests/usage_poll_scale.rs) 8.5 ms a call at 1,000 tenants — at ~10,000 tenants one poll took the whole period.

§What it keeps

Per namespace: its repositories (from the catalog, re-indexed only when the catalog document’s VERSION changes — one pointer read per settle), each repository’s last measured RepoBytes, the namespace’s tombstoned bytes, the indexed objects its OPEN stores hold, and the sequence number of the last time any of that — or its flow — changed. A read of a namespace nothing has touched is a copy out of memory.

§What makes it re-measure (the marks)

A repository’s bytes can only change while its store is OPEN, or by a store-level operation on its directory (soft delete, purge, reclaim, a growth’s remount). So:

  • opening a store marks the repository (ServerStores::repo_store’s one open site), and every store that is open, or was open at the last look, is re-measured at the next settle (the open set is bounded by the handle cache, 256) — a push, a fetch that grows the exploded cache, a background index drain, a gc: all happen on an open store;
  • a transfer, a refusal or an auth failure marks the namespace through the flow ledger’s observer (the *_today meters moved; LFS bytes may have);
  • the catalog’s version changing re-indexes: a namespace or repository created or deleted is a change;
  • tombstone operations mark their namespace; a growth’s resume marks everything;
  • and the reconcile pass (UsageLedger::reconcile_step, on the flow flush’s one-minute tick) re-measures a slice of namespaces every tick, so anything that moved bytes without a mark is corrected within one cycle and COUNTED (UsageLedger::drift_corrections) rather than silently absorbed.

§The cursor

<epoch>.<utc day>.<seq>. epoch is this ledger’s (a restart is a new ledger and every old cursor reads as “everything”); the day is in it because the *_today meters reset at midnight UTC without any event; seq is the ledger’s change counter, taken BEFORE the read settles so a change that lands during the read is answered again next time rather than lost.

Structs§

Held
One namespace’s stock meters as the ledger holds them.
UsageLedger
See the module doc.

Constants§

RECONCILE_CYCLE
The reconcile pass’s cycle: every namespace re-measured at least this often.

Functions§

tombstoned_bytes
Bytes still held by this namespace’s soft-deleted repositories — the tombstones directly in its own directory (namespaces nest, and a nested namespace’s tombstones are its own). The walk ServerStores::tombstones makes over the whole root, narrowed to one directory.

Type Aliases§

Exact
UsageLedger::exact’s answer: the namespace’s total and its repositories’ bytes (name, bytes, store directory), sorted by name.
Page
UsageLedger::page’s answer: the rows (namespace, what it holds), whether there are more, and whether a changed_since was honoured.