Crate ic_memory

Expand description

§ic-memory

Animated warning banner

EARLY INFRASTRUCTURE: validate before opening stable memory.

ic-memory helps Internet Computer canisters avoid opening the wrong stable memory after an upgrade.

It remembers this mapping forever:

logical store -> physical stable-memory slot

If a future version tries to move that store to a different slot, or reuse that slot for a different store, ic-memory rejects the layout before stable-memory handles are opened.

§Why Use It?

Use ic-memory when a canister has more than one stable store and the layout can change over time.

It is most useful for frameworks, generated canisters, multi-store apps, plugin systems, and canister families that evolve across releases.

You probably do not need it for a tiny canister with one hand-written stable structure and a fixed layout.

Meme showing ic-memory keeping ic-stable-structures stable memory allocations from drifting

§The Bug

Version 1 ships with:

app.users.v1  -> MemoryManager ID 100
app.orders.v1 -> MemoryManager ID 101

A later upgrade accidentally ships with:

app.users.v1  -> MemoryManager ID 101
app.orders.v1 -> MemoryManager ID 100

That can still compile. It can even install.

But now the canister may open orders data as users data, and users data as orders data. ic-memory catches that mismatch first.

§Quick Start

Declare the MemoryManager IDs your crate owns:

ic_memory::ic_memory_range!(start = 120, end = 129);

Open stable structures through ic_memory_key!:

use std::cell::RefCell;

thread_local! {
    pub static USERS: RefCell<UsersStore> = RefCell::new(UsersStore::init(
        ic_memory::ic_memory_key!(
            "icydb.test_db.users.data.v1",
            UsersStore,
            120,
        )
    ));
}

Bootstrap once before touching stable data:

#[ic_cdk::init]
fn init() {
    ic_memory::bootstrap_default_memory_manager().expect("valid stable-memory layout");
}

#[ic_cdk::post_upgrade]
fn post_upgrade() {
    ic_memory::bootstrap_default_memory_manager().expect("valid stable-memory layout");
}

That is the normal path.

The default runtime API is exported from the crate root. Use helpers such as ic_memory::bootstrap_default_memory_manager(), ic_memory::bootstrap_default_memory_manager_with_policy(...), ic_memory::open_default_memory_manager_memory(...), and the macros shown above; implementation modules are private.

§Multi-Crate Composition

Every crate registers into the same linked ic-memory runtime. Crates do not need to import or name each other:

mod package_a {
    ic_memory::ic_memory_range!(start = 100, end = 109);

    thread_local! {
        pub static USERS: RefCell<UsersStore> = RefCell::new(UsersStore::init(
            ic_memory::ic_memory_key!("package_a.users.v1", UsersStore, 100)
        ));
    }
}

mod package_b {
    ic_memory::ic_memory_range!(start = 110, end = 119);

    thread_local! {
        pub static ORDERS: RefCell<OrdersStore> = RefCell::new(OrdersStore::init(
            ic_memory::ic_memory_key!("package_b.orders.v1", OrdersStore, 110)
        ));
    }
}

Bootstrap validates the complete layout from every linked crate, commits the allocation ledger, and publishes validated allocations. TLS-backed stores open when your code first touches the thread_local!.

Duplicate stable keys, duplicate MemoryManager IDs, overlapping ranges, and out-of-range declarations fail before stable structures open.

ic-memory follows the ic-stable-structures::MemoryManager ID domain exactly: IDs 0..=254 are usable, and ID 255 is always the unallocated sentinel. It is not an application slot and cannot be declared or reserved.

The default runtime reserves MemoryManager IDs 0..=9 and stable keys under ic_memory.* for allocation-governance records. The ledger itself lives at ID 0; it remains in the durable ledger for recovery, but public runtime helpers do not publish or open that internal allocation as application memory.

Range claims are authoritative in the default runtime. If a crate registers ic_memory_range!, its declared memories must stay inside that range. Framework adapters that want their own range policy, such as Canic, should register only the ranges they want ic-memory to enforce and put the rest in their policy adapter.

The validated allocation state is an in-memory capability produced by bootstrap; it is not a serde payload and should not be treated as configuration.

§Diagnostics

Use default_memory_manager_doctor_report() for operator-facing preflight and runtime diagnostics. It can be called before or after bootstrap and reports the stable-cell status, protected commit recovery state, recovered ledger export, registered declarations, range authority, validation preflight, and live MemoryManager slot sizes when they can be recovered.

Use default_memory_manager_commit_recovery_diagnostic() when you only need the dual-slot protected commit status, including empty, corrupt, ambiguous, or recoverable physical ledger state.

§Stable Keys

Stable keys are permanent logical store names. They should describe ownership and purpose, not the current memory ID.

namespace.component.store_or_role.vN

Examples:

use ic_memory::StableKey;

StableKey::parse("app.orders.v1").expect("app key");
StableKey::parse("myapp.audit_log.v1").expect("app key");
StableKey::parse("icydb.test_db.users.data.v1").expect("database key");

Changing a key creates a new logical allocation identity. If the durable store is the same, keep the stable key and update schema metadata instead.

Schema metadata is optional diagnostic metadata for the in-place store schema. Construct it with SchemaMetadata::new(Some(version)); version 0 is reserved for absence and is rejected.

§More Detail

The short version:

declare ranges
register stable stores
bootstrap once
only then open stable memory

Framework authors and policy adapters should read ADVANCED.md. The non-negotiable invariants are recorded in SAFETY.md. The protocol whitepaper lives in whitepaper/src/SUMMARY.md and builds as an mdBook with make maintainer-build.

ic-memory is early infrastructure extracted from Canic. It owns allocation governance, not schema migration, endpoint routing, authorization, or data semantics.

Retro computer warning: don't overwrite your memory

Stable-memory allocation-governance primitives for Internet Computer canister upgrades.

ic-memory prevents stable-memory slot drift.

Once a stable key is committed to a physical allocation slot, future binaries must either reopen that same stable key on that same slot or declare a new stable key.

The crate records and validates durable ownership in both directions: an active stable key cannot move to a different physical slot, and an active physical slot cannot be reused by a different stable key.

The intended integration flow is:

Recover the persisted allocation ledger.
Declare the stable stores expected by the current binary.
Validate those declarations against ledger history and any framework policy.
Commit the next generation.
Only then open stable-memory handles through a validated allocation session.

This crate owns allocation invariants, not framework policy. Namespace rules, controller authorization, endpoint lifecycle, schema migrations, and application validation belong to the framework or application.

For the default MemoryManager runtime, registered ic-memory range claims are generic allocation policy and are enforced before caller-supplied policy. A framework such as Canic that wants higher-level range semantics should adapt to this contract deliberately: either register the ranges it wants ic-memory to enforce, or omit user ranges and enforce application space through its own AllocationPolicy.

Use these primitives before opening stable-memory handles. Integrations should recover the historical ledger, declare the stores expected by the current binary, validate declarations against history and policy, commit a new generation, and only then publish a validated allocation session that can open slots through a storage substrate.

AllocationBootstrap is the golden path for whichever layer owns a given ledger store. Canic may own bootstrap for a framework canister and compose IcyDB/application declarations through its registry; IcyDB may own bootstrap directly for generated database stores; or a standalone application canister may own bootstrap itself. Exactly one owner should bootstrap one ledger store. Multiple layers in the same canister must either compose declarations into that owner or use distinct ledger stores and allocation domains.

ic-stable-structures MemoryManager IDs are the first-class supported physical slot substrate. That ID domain is u8: IDs 0..=254 are usable, and ID 255 is always the ic-stable-structures unallocated sentinel. The crate still keeps narrow internal abstractions for storage adapters and diagnostics, but the native IC path is MemoryManager ID 0 -> ic-stable-structures::Cell<StableCellLedgerRecord, _> -> LedgerCommitStore -> CommittedGenerationBytes -> LedgerPayloadEnvelope -> RecoveredLedger -> ValidatedAllocations.

ic-memory is not a replacement for ic-stable-structures collections and does not wrap typed stores such as StableBTreeMap.

Re-exports§

pub use ic_stable_structures as stable_structures;

Macros§

eager_init: Register one pre-bootstrap hook.
ic_memory_declaration: Register a MemoryManager allocation declaration during static initialization.
ic_memory_key: Declare and open a validated MemoryManager slot by stable key.
ic_memory_range: Declare a MemoryManager allocation range during static initialization.

Structs§

AllocationBootstrap: AllocationBootstrap
AllocationDeclaration: AllocationDeclaration
AllocationHistory: AllocationHistory
AllocationLedger: AllocationLedger
AllocationRecord: AllocationRecord
AllocationRetirement: AllocationRetirement
AllocationSession: AllocationSession
AllocationSlotDescriptor: AllocationSlotDescriptor
AuthoritativeSlot: AuthoritativeSlot
BootstrapCommit: BootstrapCommit
CommitSlotDiagnostic: CommitSlotDiagnostic
CommitStoreDiagnostic: CommitStoreDiagnostic
CommittedGenerationBytes: CommittedGenerationBytes
DeclarationCollector: DeclarationCollector
DeclarationSnapshot: DeclarationSnapshot
DefaultMemoryManagerDoctorReport: DefaultMemoryManagerDoctorReport
DiagnosticCheck: DiagnosticCheck
DiagnosticDeclaration: DiagnosticDeclaration
DiagnosticExport: DiagnosticExport
DiagnosticGeneration: DiagnosticGeneration
DiagnosticMemorySize: DiagnosticMemorySize
DiagnosticRangeAuthority: DiagnosticRangeAuthority
DiagnosticRecord: DiagnosticRecord
DiagnosticStableCell: DiagnosticStableCell
DualCommitStore: DualCommitStore
GenerationRecord: GenerationRecord
LedgerAnchor: LedgerAnchor
LedgerCommitStore: LedgerCommitStore
LedgerPayloadEnvelope: LedgerPayloadEnvelope
MemoryManagerAuthorityRecord: MemoryManagerAuthorityRecord
MemoryManagerIdRange: MemoryManagerIdRange
MemoryManagerRangeAuthority: MemoryManagerRangeAuthority
RecoveredLedger: RecoveredLedger
SchemaMetadata: SchemaMetadata
SchemaMetadataRecord: SchemaMetadataRecord
StableCellLedgerRecord: StableCellLedgerRecord
StableKey: StableKey
StableKeyError: StableKeyError
StaticMemoryDeclaration: StaticMemoryDeclaration
StaticMemoryRangeDeclaration: StaticMemoryRangeDeclaration
ValidatedAllocations: ValidatedAllocations

Enums§

AllocationReservationError: AllocationReservationError
AllocationRetirementError: AllocationRetirementError
AllocationSessionError: AllocationSessionError
AllocationSlot: AllocationSlot
AllocationStageError: AllocationStageError
AllocationState: AllocationState
AllocationValidationError: AllocationValidationError
BootstrapError: BootstrapError
BootstrapReservationError: BootstrapReservationError
BootstrapRetirementError: BootstrapRetirementError
CommitRecoveryError: CommitRecoveryError
CommitSlotIndex: CommitSlotIndex
DeclarationSnapshotError: DeclarationSnapshotError
DiagnosticCheckStatus: DiagnosticCheckStatus
DiagnosticStableCellStatus: DiagnosticStableCellStatus
LedgerCommitError: LedgerCommitError
LedgerIntegrityError: LedgerIntegrityError
LedgerPayloadEnvelopeError: LedgerPayloadEnvelopeError
MemoryManagerRangeAuthorityError: MemoryManagerRangeAuthorityError
MemoryManagerRangeError: MemoryManagerRangeError
MemoryManagerRangeMode: MemoryManagerRangeMode
MemoryManagerSlotError: MemoryManagerSlotError
RuntimeBootstrapError: RuntimeBootstrapError
RuntimeDiagnosticError: RuntimeDiagnosticError
RuntimeOpenError: RuntimeOpenError
RuntimePolicyError: RuntimePolicyError
SchemaMetadataError: SchemaMetadataError
StableCellLedgerError: StableCellLedgerError
StableCellPayloadError: StableCellPayloadError
StaticMemoryDeclarationError: StaticMemoryDeclarationError

Constants§

IC_MEMORY_AUTHORITY_OWNER: Diagnostic owner label for ic-memory allocation-governance infrastructure.
IC_MEMORY_AUTHORITY_PURPOSE: Diagnostic purpose for the ic-memory allocation-governance authority range.
IC_MEMORY_LEDGER_LABEL: Diagnostic label of the allocation ledger when backed by the current MemoryManager substrate.
IC_MEMORY_LEDGER_STABLE_KEY: Stable key of the allocation ledger when backed by the current MemoryManager substrate.
IC_MEMORY_STABLE_KEY_PREFIX: Stable-key namespace prefix reserved for ic-memory allocation-governance infrastructure.
MEMORY_MANAGER_GOVERNANCE_MAX_ID: Last MemoryManager ID reserved for ic-memory governance in the current substrate.
MEMORY_MANAGER_INVALID_ID: MemoryManager unallocated-bucket sentinel. This is not a usable slot.
MEMORY_MANAGER_LEDGER_ID: MemoryManager ID used by the allocation ledger in the current MemoryManager substrate.
MEMORY_MANAGER_MAX_ID: Last usable MemoryManager virtual memory ID.
MEMORY_MANAGER_MIN_ID: First usable MemoryManager virtual memory ID.
STABLE_CELL_HEADER_SIZE: Stable-cell header byte length.
STABLE_CELL_LAYOUT_VERSION: Stable-cell layout version supported by this adapter.
STABLE_CELL_MAGIC: Stable-cell magic prefix written by ic-stable-structures::Cell.
STABLE_CELL_VALUE_OFFSET: Byte offset where the stable-cell value payload starts.
WASM_PAGE_SIZE_BYTES: WebAssembly page size used by ic-stable-structures memory implementations.

Traits§

AllocationPolicy: AllocationPolicy
DualProtectedCommitStore: DualProtectedCommitStore
ProtectedGenerationSlot: ProtectedGenerationSlot
StorageSubstrate: StorageSubstrate
Validate: Validate

Functions§

bootstrap_default_memory_manager: Bootstrap the default MemoryManager<DefaultMemoryImpl> runtime using generic policy.
bootstrap_default_memory_manager_with_policy: Bootstrap the default runtime and layer caller-supplied policy over generic range checks.
collect_static_memory_declarations: Add currently registered static allocation declarations to a collector.
decode_stable_cell_ledger_record: Decode a StableCellLedgerRecord from stable-cell value bytes.
decode_stable_cell_payload: Decode the raw value payload from an ic-stable-structures::Cell memory.
default_memory_manager_commit_recovery_diagnostic: Build a protected commit recovery diagnostic for the default ledger store.
default_memory_manager_diagnostic_export: Build a diagnostic export for the default MemoryManager runtime.
default_memory_manager_doctor_report: Build a preflight and runtime diagnostic report for the default runtime.
is_default_memory_manager_bootstrapped: Return true once default runtime bootstrap has completed.
is_ic_memory_stable_key: Return true when stable_key belongs to the ic-memory namespace.
memory_manager_governance_range: MemoryManager range reserved for ic-memory governance in the current substrate.
open_default_memory_manager_memory: Open a validated MemoryManager memory by stable key and expected ID.
register_static_memory_declaration: Register one allocation declaration before bootstrap seals the snapshot.
register_static_memory_manager_declaration: Register one MemoryManager declaration before bootstrap seals the snapshot.
register_static_memory_manager_declaration_with_schema: Register one MemoryManager declaration with schema metadata.
register_static_memory_manager_range: Register one MemoryManager authority range before bootstrap seals the snapshot.
register_static_memory_range_declaration: Register one authority range declaration before bootstrap seals the snapshot.
select_authoritative_slot: Select the highest-generation valid physical slot.
static_memory_declaration_snapshot: Seal currently registered static allocation declarations into a snapshot.
static_memory_declarations: Return the currently registered static allocation declarations.
static_memory_range_authority: Return the currently registered static range declarations as an authority table.
static_memory_range_declarations: Return the currently registered static range declarations.
validate_allocations: Validate a committed ledger and current declarations before opening.
validate_memory_manager_id: Validate that a MemoryManager ID is usable as an allocation slot.
validate_stable_cell_ledger_memory: Validate an existing stable-cell ledger record before opening it with ic-stable-structures::Cell.
validated_allocations: Return the published validated allocations for the default runtime substrate.

Crate ic_memory

Crate ic_memory Copy item path

§ic-memory

§Why Use It?

§The Bug

§Quick Start

§Multi-Crate Composition

§Diagnostics

§Stable Keys

§More Detail

Re-exports§

Macros§

Structs§

Enums§

Constants§

Traits§

Functions§

Crate ic_memory