# 0.12 Explicit Memory Runtime Redesign
## Status
Implemented and validated on 2026-07-30. This document is the landing design and
validation record for the pre-1.0 hard cut from split global/TLS runtime state
to one explicit owner per backing memory instance.
No compatibility layer is planned. The current allocation-ledger,
payload-envelope, protected commit-slot, and stable-cell encodings remain
unchanged.
## 1. Reconstructed 0.11.1 Ownership Model
The 0.11.1 default path splits one logical runtime across two scopes:
```text
process
├── STATIC_MEMORY_DECLARATIONS: Mutex<registry>
├── EAGER_INIT_HOOKS: Mutex<Vec<fn()>>
├── BOOTSTRAPPED: AtomicBool
└── COMMITTED_ALLOCATIONS: Mutex<Option<CommittedAllocations>>
thread
├── DEFAULT_MEMORY_MANAGER: MemoryManager<DefaultMemoryImpl>
└── DEFAULT_LEDGER_CELL: RefCell<Option<Cell<...>>>
```
Bootstrap first consults the process-global committed capability. When present,
it returns without opening or recovering the calling thread's ledger memory.
Opening and diagnostics then combine that global capability/lifecycle claim
with the calling thread's `MemoryManager` and ledger cell. The scopes do not
describe one owner and can disagree.
The registry also seals separately from eager-hook execution. Concurrent
snapshot attempts can drain the eager queue independently of snapshot
construction, and declaration order is preserved rather than canonicalized.
## 2. Confirmed Defect and Reproduction
A temporary downstream crate was built against the repository's 0.11.1 source
with two libtest tests containing the reported range and declaration. Running:
```text
cargo test -- --test-threads=1
```
produced:
```text
first_libtest_runtime ... ok
second_libtest_runtime ... FAILED
LedgerCommit(Recovery(NoValidGeneration))
```
Libtest still creates a fresh native thread for the second test. That thread
receives a fresh `DefaultMemoryImpl`, `MemoryManager`, and ledger cell, but the
process-global bootstrap flag and committed capability still describe the first
thread's memory.
## 3. Final Authority Model
```text
linked program
└── declaration registry
├── constructor registration hooks
├── eager declaration hooks
└── atomic deterministic seal
└── immutable SealedDeclarationSnapshot
├── canonical declarations
├── canonical ranges
└── declaration authority metadata
│ supplied by reference
▼
backing memory M ──> MemoryRuntime<M>
├── MemoryManager<M>
├── allocation-ledger Cell<..., VirtualMemory<M>>
└── RuntimeLifecycle
├── Unbootstrapped
└── Bootstrapped {
committed_allocations
}
```
Only linked-program declaration authority is process-global. Recovery,
persistence, lifecycle, allocation-open authority, diagnostics, and live memory
sizes are runtime-local.
## 4. `MemoryRuntime<M>` API and Lifecycle
`MemoryRuntime<M>` is the canonical runtime:
```rust,ignore
pub struct MemoryRuntime<M: Memory> {
memory_manager: MemoryManager<M>,
ledger_cell: Option<LedgerCell<M>>,
lifecycle: RuntimeLifecycle,
}
```
It requires only `M: ic_stable_structures::Memory`. It does not require
`Send`, `Sync`, `Clone`, or `'static`.
The intended public operations are:
```rust,ignore
MemoryRuntime::new(memory)?
runtime.bootstrap(&sealed_declarations, &policy)
runtime.committed_allocations()
runtime.open_memory(stable_key, expected_id)
runtime.diagnostic_export()
runtime.commit_recovery_diagnostic()
runtime.doctor_report(&sealed_declarations, &policy)
runtime.is_bootstrapped()
```
Construction preflights raw backing memory before
`ic_stable_structures::MemoryManager::init()`. Empty memory may be initialized;
nonempty memory must contain the current `MGR` magic and layout version.
Foreign or unsupported memory returns `RuntimeConstructionError` without
modifying the backing bytes.
Successful bootstrap moves the lifecycle directly from `Unbootstrapped` to
`Bootstrapped { committed_allocations }` only after the mutated stable-cell
record is written. Failure leaves it unbootstrapped. Repeated bootstrap on the
same object returns its existing capability without recovering, evaluating
policy, persisting, or advancing a generation.
## 5. Static Declaration Sealing
The process registry has one state machine:
```text
Open
-> Sealing { owner thread }
-> Sealed(Arc<SealedDeclarationSnapshot>)
-> Failed(seal error)
```
A dedicated seal lock serializes snapshot construction. The sealing thread runs
generated registration hooks first and user eager hooks second. Registrations
made by those hooks are accepted; concurrent registrations from other threads
fail as late registrations. Concurrent snapshot callers wait for the same seal
operation and receive clones of the same immutable `Arc`.
Before validation, declarations are sorted by stable key and slot and ranges by
range/authority metadata. Duplicate checking therefore reports a stable first
conflict and serialized declaration bytes do not depend on constructor order.
The effective internal governance declaration, governance range, and authority
metadata are built once into the sealed snapshot.
Sealing is not runtime bootstrap. Policy evaluation, recovery, staging, and
persistence still run independently for every `MemoryRuntime<M>`.
## 6. Default TLS Runtime
The retained convenience layer owns exactly one value:
```rust,ignore
thread_local! {
static DEFAULT_RUNTIME:
RefCell<MemoryRuntime<DefaultMemoryImpl>> = ...;
}
```
All default free functions and `ic_memory_key!` opens enter this object. They do
not own parallel lifecycle or capability state. Native threads therefore get
independent runtimes over independent `DefaultMemoryImpl` values. On IC Wasm,
execution is single-threaded, so the TLS object naturally has canister-instance
lifetime.
There is no public or runtime reset operation. Unit-only registry reset support
may isolate tests of the process-global declaration state, but it is not a
runtime recovery mechanism.
## 7. Concurrency and Reentrancy
The sealed snapshot is immutable and `Arc`-backed, so native threads can share
the same declaration meaning. Runtime objects are not made thread-safe
artificially; callers place each runtime where its backing `Memory` belongs.
Independent threads construct and bootstrap independent runtime objects.
Default TLS entry uses `try_borrow`/`try_borrow_mut`. Re-entry returns a typed
runtime-state error. It is not reported as mutex poisoning, and it does not
fall back to another runtime's capability.
The registry continues to classify actual mutex poisoning separately. Eager
hook execution occurs outside the registry mutex so a hook can register
declarations without recursive locking.
## 8. Diagnostics and Error Ownership
`MemoryRuntime<M>` reads the ledger virtual memory and application virtual
memories from its own `MemoryManager<M>`. Its diagnostic export, commit-recovery
diagnostic, doctor report, and live size inspection therefore cannot cross
runtime boundaries.
Errors remain layered:
- stable-cell corruption is a stable-cell error;
- protected recovery corruption/ambiguity is a ledger commit error;
- declaration and range failures come from snapshot sealing or validation;
- policy rejection remains parameterized by the caller's policy error;
- use before bootstrap is `RuntimeOpenError::NotBootstrapped`;
- default TLS re-entry/unavailability is a runtime-state error.
Doctor output reports the lifecycle of the runtime being inspected. Preflight
uses the supplied immutable declaration snapshot, not a process-global
bootstrap result.
## 9. Persisted-Format Assessment
No durable format change is required. The redesign reuses:
```text
MemoryManager ID 0
-> ic-stable-structures Cell envelope
-> StableCellLedgerRecord
-> LedgerCommitStore
-> current LedgerPayloadEnvelope
-> current AllocationLedger
```
The same preflight decoder, staging logic, commit-slot checksums, CBOR
structures, field names, format marker, and format version remain in place.
Only the in-memory owner that holds the `MemoryManager`, cell, and committed
capability changes.
## 10. Public API Hard Cuts
The public canonical addition is `MemoryRuntime<M>` plus
`SealedDeclarationSnapshot` and its one sealing entry point.
Superseded registry collection/snapshot functions that can expose unsealed or
separately assembled views are removed. The default free functions remain
because the TLS default runtime is an intentional convenience API, not because
they forward to a legacy implementation.
The doctor report is generalized from a default-manager-named type to
`MemoryRuntimeDoctorReport`. Default TLS borrow conflicts become typed errors.
Process-global bootstrap flags, committed capabilities, default manager/cell
globals, and runtime reset support are deleted.
## 11. Required Tests
The landing tests cover:
1. two downstream-style libtest tests under `--test-threads=1`;
2. runtime A not bootstrapping runtime B;
3. separate `VectorMemory` runtimes persisting independent ledgers;
4. data isolation between separate backing memories;
5. runtime-local ledger diagnostics and live sizes;
6. no stale default capability on a second libtest thread;
7. concurrent callers receiving one completely sealed snapshot;
8. concurrent independent runtime bootstraps;
9. typed open-before-bootstrap failure;
10. wrong stable key and expected ID failures;
11. identity-bound idempotent same-runtime bootstrap with no generation
advance;
12. recovery through a fresh runtime over populated backing memory;
13. failed bootstrap leaving no capability;
14. lifecycle agreement in doctor and diagnostic APIs; and
15. Wasm target compilation of the default runtime.
Registry tests additionally cover deterministic ordering, duplicate rejection,
late registration, eager-hook ordering, and reentrant/concurrent sealing.
## 12. Ordered Implementation Slices
1. Record the defect, authority model, format decision, API, and validation
baseline in this document.
2. Replace the registry's independent reads/seal flag and runtime-owned eager
queue with atomic canonical `SealedDeclarationSnapshot` construction.
3. Introduce `MemoryRuntime<M>` and move bootstrap, ledger cell persistence,
capability publication, opens, diagnostics, and doctor logic into it.
4. Replace the four split default-runtime globals with one fallibly borrowed TLS
runtime; delete runtime reset state and stale singleton tests.
5. Add generic, concurrency, recovery, negative-space, and exact libtest
regressions.
6. Update root exports, macros, maintained examples, README, advanced/safety
guidance, whitepaper operational guidance, and changelog.
7. Run formatting, checks, tests, feature/MSRV/Wasm/docs/Clippy/package
validation, stale-symbol searches, and focused size/performance comparisons;
record exact evidence below.
Each slice preserves the current durable format and is intended to land
together because the pre-1.0 policy forbids retaining the superseded runtime in
parallel.
## 13. Validation Evidence
Baseline before implementation:
- `cargo test -- --test-threads=1`: passed all repository tests (186 unit, 1
compile-fail harness, 1 runtime macro integration test, and doctests).
- Exact temporary downstream reproduction: failed on its second libtest test
with `LedgerCommit(Recovery(NoValidGeneration))`, confirming the defect.
- Worktree was clean before edits.
Post-implementation:
- `cargo fmt --all --check`: passed.
- `cargo check --all-features --all-targets`: passed.
- `cargo check --no-default-features --all-targets`: passed (the package
currently declares no optional features, so this also confirms the minimal
feature surface).
- `cargo +1.85.0 check --all-targets`: passed at the declared MSRV.
- `cargo clippy --all-targets -- -D warnings`: passed.
- `cargo doc --no-deps` and `cargo test --doc`: passed. There is one exercised
doctest and five intentionally ignored integration sketches.
- `cargo test -- --test-threads=1`: passed 182 unit tests, the compile-fail
harness and all five UI cases, the public explicit-runtime integration test,
both default-runtime libtest regressions, and doctests.
- `cargo test --test runtime_macros -- --test-threads=1`: passed both explicit
first/second libtest-thread regressions.
- `cargo test concurrent_ -- --test-threads=1`: passed concurrent snapshot and
independent-runtime bootstrap tests; the full suite also passed the
concurrent late-registration test.
- The exact temporary downstream reproduction that failed against 0.11.1
passed both tests against 0.12.0.
- `cargo check --target wasm32-unknown-unknown --tests`: passed.
- `cargo build --release`: passed.
- `cargo package --allow-dirty`: packaged and verified 85 files, 669.3 KiB
uncompressed and 220.4 KiB compressed.
- `cargo tree -e normal,features` showed no new dependency; the runtime redesign
uses the existing standard library and dependency graph.
- Fixture and persisted-format files have no diff. Stale runtime-global and
superseded public registry symbols are absent. The surviving process globals
are the declaration registry, its seal serialization lock, and a test-only
registry lock.
Raw non-gzipped Wasm comparison used identical two-export `cdylib` probes
against committed 0.11.1 and this tree. Both were built with Rust 1.97.1,
`wasm32-unknown-unknown`, `opt-level = "z"`, fat LTO, one codegen unit,
`panic = "abort"`, and stripped symbols:
| 0.11.1 default bootstrap + open | 224,415 | baseline |
| 0.12.0 default bootstrap + open | 232,480 | +8,065 (+3.59%) |
Focused native release probes used the same one-declaration default-runtime
program. Bootstrap timing was measured inside 51 separate processes so the
0.11.1 process-global bootstrap state could not invalidate later samples. Open
timing used 21 samples of 100,000 opens after one bootstrap:
| Fresh bootstrap | 5,166,844 ns | 5,221,306 ns | +1.05% |
| Committed open | 68 ns/op | 67 ns/op | -1.47% |
These are focused local microbenchmarks rather than whole-canister performance
claims. They show no material hot-path regression; the Wasm ownership and
fallible-sealing machinery adds 8,065 raw bytes in this representative probe.
## 14. Remaining Risks
- `ic-stable-structures::MemoryManager` and `VirtualMemory` internally use
`Rc<RefCell<_>>`; the runtime can make its own TLS entry fallible but cannot
change reentrancy behavior inside upstream memory handles.
- Static constructors cannot return errors to the operating system. Generated
registration work is therefore deferred into the fallible seal operation so
declaration failures surface at snapshot/bootstrap time.
- Default-runtime policy is evaluated once per runtime object. Repeated
bootstrap returns the established capability only when the sealed snapshot
and explicit `RuntimeBootstrapPolicy` identity match the successful
bootstrap; mismatches return a typed error without touching the ledger.
- Whole-canister size and latency depend on downstream reachability. Repository
probes can compare this crate's representative paths but cannot substitute
for an integration canister's release budget.
- The representative Wasm probe grew by 8,065 raw bytes (+3.59%). Downstream
canisters should measure reachability and optimization in their own release
artifacts.