Crate ic_memory 

canic  ·  icydb  ·  ic-timers  ·  ic-memory  ·  ic-query  ·  ic-backup  ·  ic-blob-storage  ·  ic-testkit

Jump to: What it does · Why it matters · Is it useful? · How it works · Choose an integration style · Quick start · Troubleshooting · Advanced integration

What it does

ic-memory is a safety system for an Internet Computer application’s persistent data.

When an application is upgraded, its code changes but its stored data remains. Each database, log, queue, or settings store must reconnect to the same storage location it used before. If two stores are accidentally swapped, the new code can interpret one kind of data as another.

ic-memory remembers which storage location belongs to each named store. Before the application opens any of those stores, it checks the new layout against the saved ownership records. A conflicting upgrade stops with an error instead of opening the wrong data.

ic-memory protects the connection between a store and its storage location. It is not a backup system, a database schema migrator, or a data validator.

Why this matters

Imagine that version 1 of an application stores users and orders separately:

Users  -> storage location 100
Orders -> storage location 101

A later version accidentally reverses those locations:

Users  -> storage location 101
Orders -> storage location 100

The diagram below shows the same mistake and the point where ic-memory intervenes:

A changed store-to-location mapping is rejected before application data opens.

The program can still compile, and the upgrade can still install. Without an allocation check, it may then read order records as users and user records as orders. ic-memory detects the changed ownership before either location is opened.

This protection matters because a stable-memory location is a durable part of an application’s storage layout, even though it can look like an ordinary number in source code.

Is it useful for my application?

Use ic-memory for evolving multi-store layouts, not as a backup or migration system.

ic-memory is most useful for frameworks, generated canisters, multi-store applications, plugin systems, and canister families that evolve over time.

How it works

The lifecycle has three parts:

1. Name each store. The application gives every persistent store a permanent identity, such as app.users.v1.
2. Remember its location. On the first successful installation, ic-memory records which storage location belongs to each name.
3. Check before opening. On every installation or upgrade, the new application version declares the layout it expects. ic-memory compares that layout with the remembered one before any application store opens.

Name stores, retain their locations, and check the complete layout before opening data.

If the layouts agree, the application receives permission to open its stores. If a known name moved, a location changed owner, or another allocation rule is broken, the check returns an error and ic-memory grants no permission to open application stores.

This check-and-record operation is called bootstrap. Its important guarantee is validation before open: a diagnostic report or an uncommitted validation result cannot grant access to a store. The runtime publishes permission only after the allocation ledger has been recovered, checked, and durably updated.

The ledger stores one current record per allocated identity and the latest schema metadata. It keeps no per-upgrade or schema-change history. Omitted and retired identities retain their IDs to prevent accidental reuse. A commit counter and two protected commit slots support stale-proof checks and corruption detection.

What happens when a check fails?

Bootstrap returns an error before application stores are opened. ic-memory does not silently repair, move, discard, or reinterpret a conflicting allocation. A failed attempt publishes no permission to open stores.

Treat the error as an upgrade-safety signal: keep the existing stable memory, inspect the declared keys, IDs, ranges, policy, and diagnostics, then correct the new application version. Do not erase the allocation ledger or replace it with an empty one to make the error disappear. See the symptom-based troubleshooting guide for safe next steps.

What it protects

Allocation safety complements backups, schema migration, authorization, and disaster recovery.

Retiring a store does not make its old location reusable. That tombstone is intentional: reusing the location could make a rollback open unrelated data.

Terminology

• Stable memory is persistent storage that survives an Internet Computer canister upgrade.
• A stable key is a permanent, human-readable name for one store, such as app.orders.v1.
• A memory ID or slot is the numbered MemoryManager storage location behind that name.
• An authority is an ownership label. It prevents one component from claiming storage assigned to another component.
• A range grant is a group of memory IDs that an authority may use.
• Bootstrap is the check-and-commit step that must succeed before the application opens its stores.
• The allocation ledger is ic-memory’s durable record of which stable key owns which slot.

Choose an integration style

Most integrations use one of these paths:

Choose who assigns the memory ID without changing its durable ownership.

Fixed and automatic declarations may coexist in one host. The application that owns the concrete runtime still bootstraps the combined layout exactly once. Libraries adopting that runtime verify their own requirements and then open their committed keys; they do not bootstrap independently.

Quick start for developers

Add the crate:

[dependencies]
ic-memory = "0.27.3"

ic-memory re-exports its exact ic-stable-structures dependency through ic_memory::ic_stable_structures. Import collections, backing memories, and traits through that namespace so their types match the runtime:

use ic_memory::{
    RuntimeMemory,
    ic_stable_structures::{Cell, DefaultMemoryImpl},
};

type CounterStore = Cell<u64, RuntimeMemory<DefaultMemoryImpl>>;

A separate ic-stable-structures dependency is unnecessary for those imports.

Declare a fixed storage location

Give the component an authority, grant it a range, and declare the permanent key and ID of each store:

const MEMORY_AUTHORITY: &str = "example_app";

ic_memory::ic_memory_range!(
    authority = MEMORY_AUTHORITY,
    start = 120,
    end = 129,
);

ic_memory::ic_memory_declaration!(
    authority = MEMORY_AUTHORITY,
    key = "example_app.users.v1",
    label = "UsersStore",
    id = 120,
);

fn initialize_stable_storage() -> Result<(), Box<dyn std::error::Error>> {
    ic_memory::bootstrap_default_memory_manager()?;
    let users = ic_memory::open_default_memory_manager_memory(
        "example_app.users.v1",
        120,
    )?;
    drop(users);
    Ok(())
}

Call the bootstrap function from both the canister’s initialization and post-upgrade lifecycle before code touches any stable collection.

fn bootstrap_memory() {
    ic_memory::bootstrap_default_memory_manager()
        .expect("stable-memory allocation layout must be valid");
}

#[ic_cdk::init]
fn init() {
    bootstrap_memory();
}

#[ic_cdk::post_upgrade]
fn post_upgrade() {
    bootstrap_memory();
}

These lifecycle functions must run before any thread-local or deferred initialization opens a stable collection. Applications using a custom policy or bucket configuration call the corresponding bootstrap helper in the same locations.

The default range mode is Reserved: it permits declared fixed IDs but does not provide new automatic allocations.

Let the host assign a location

Libraries can request a durable key without choosing an ID. The application that owns the runtime grants an explicit Allowed pool:

ic_memory::ic_memory_range!(
    authority = "example_app",
    start = 10,
    end = 254,
    mode = Allowed,
);
ic_memory::ic_memory_declaration!(
    authority = "example_app",
    key = "example_app.users.v1",
);

fn initialize() -> Result<(), Box<dyn std::error::Error>> {
    ic_memory::bootstrap_default_memory_manager()?;
    let assigned_id =
        ic_memory::default_memory_manager_memory_id("example_app.users.v1")?;
    let users =
        ic_memory::open_default_memory_manager_memory_by_key("example_app.users.v1")?;
    drop((assigned_id, users));
    Ok(())
}

Known keys keep their committed IDs. New requests are sorted by stable key and receive the lowest unclaimed ID in their authority’s Allowed ranges. Fixed, reserved, omitted, and retired allocations remain unavailable. If no eligible ID remains, bootstrap returns MemoryResolutionError::Exhausted.

For examples of both allocation styles, see examples/key_only.rs and examples/composed_host.rs.

Applications composed from several libraries

Every linked crate contributes declarations to one immutable registry. The application grants each component only its intended range and bootstraps the combined layout once.

Libraries contribute requirements; the application owns one combined bootstrap.

Libraries adopting an already bootstrapped host can verify that all of their requirements were included without rerunning bootstrap or replacing the host’s policy:

fn adopt_host() -> Result<(), Box<dyn std::error::Error>> {
    let requirements = ic_memory::sealed_declaration_snapshot()?;
    ic_memory::verify_default_memory_manager_authority(
        &requirements,
        "library_name",
    )?;
    Ok(())
}

Duplicate stable keys, duplicate memory IDs, overlapping ranges, and out-of-range declarations fail before stable structures open. Hosts must also declare or reserve allocations used by raw MemoryManager clients; diagnostics cannot infer ownership from bytes alone.

The default runtime reserves IDs 0..=9 and keys under ic_memory.* for its own governance records. ID 0 contains the allocation ledger. Application code may use IDs through 254; ID 255 is the upstream unallocated sentinel and can never be declared.

Stable keys

A stable key names the durable identity of a store, not its current numeric location:

namespace.component.store_or_role.vN

use ic_memory::StableKey;

StableKey::parse("app.orders.v1").expect("app key");
StableKey::parse("myapp.audit_log.v1").expect("app key");
StableKey::parse("icydb.test_db.users.data.v1").expect("database key");

Changing a key creates a new allocation identity. If the durable store is still the same store, keep its key and change its optional diagnostic schema metadata instead.

Frequently asked questions

Does ic-memory move or migrate application data?

No. It validates allocation identity. Schema and data migrations remain the application’s responsibility.

Does it back up stable memory or recover corrupted application data?

No. Keep a separate backup and disaster-recovery plan. ic-memory fails closed when its allocation metadata cannot be recovered safely.

What happens when validation fails?

Bootstrap returns an error and publishes no open capability. Fix the proposed layout or policy; do not erase the ledger to bypass the conflict.

Can a retired memory location be reused?

No. Retirement is a permanent tombstone so a future version or rollback cannot mistake unrelated data for the retired store.

Does every library bootstrap separately?

No. One owner bootstraps each concrete runtime. Libraries verify their requirements against the host’s committed layout and open only their keys.

When should I use fixed versus automatic allocation?

Use fixed IDs when the application deliberately manages its layout. Use automatic allocation when a host should place reusable components within explicitly granted ranges. Both preserve the assigned ID after commitment.

Can an upgrade add a new store safely?

Yes, provided its key is new, its fixed ID or automatic range is eligible, and the complete layout passes current policy and historical validation.

Operations and advanced integration

• Operations and diagnostics explains memory attribution, bucket configuration, and runtime reports.
• Troubleshooting maps common symptoms to safe recovery steps.
• Advanced ic-memory covers explicit runtimes, custom policies, recovery, and manual bootstrap.
• Safety invariants records the guarantees future changes must preserve.
• Documentation index separates current guidance from historical engineering evidence.
• Release guide is for maintainers preparing and publishing a release.

ic-memory is pre-1.0 infrastructure extracted from Canic. Current releases use the current API and durable format only; earlier pre-1.0 wire formats are not a compatibility target. Stable-memory allocation-governance primitives for Internet Computer canister upgrades.

ic-memory prevents stable-memory slot drift.

Once a stable key is committed to a physical allocation slot, future binaries must either reopen that same stable key on that same slot or declare a new stable key.

The crate records and validates durable ownership in both directions: an active stable key cannot move to a different physical slot, and an active physical slot cannot be reused by a different stable key.

The intended runtime integration flow is:

1. Recover the persisted allocation ledger.
2. Admit consumer identity and authorized historical selections from bounded metadata under the host’s RuntimeBootstrapPolicy.
3. Resolve logical requests under explicit host grants, combine them with sealed fixed declarations, then validate against retained ownership and current policy.
4. Stage and durably persist the next generation.
5. Only then open stable-memory handles through committed allocation authority.

This crate owns allocation invariants, not framework policy. Namespace rules, controller authorization, endpoint lifecycle, schema migrations, and application validation belong to the framework or application.

For the default MemoryManager runtime, registered ic-memory range claims are generic allocation policy and are enforced before caller-supplied policy. A framework such as Canic that wants higher-level range semantics should adapt to this contract deliberately. Register explicit grants for logical placement and historical selection. When no user ranges are registered, the framework’s AllocationPolicy can enforce fixed application claims directly.

Use these primitives before opening stable-memory handles. Integrations should recover the historical ledger, declare the stores expected by the current binary, admit recovered identity, resolve requests, validate against history and policy, persist a new generation, and only then publish authority before opening slots through the storage owner.

Bounded physical attribution is available through MemoryRuntime::memory_allocations and default_memory_manager_memory_allocations. It reports actual persisted buckets and explicit residuals without decoding retained ownership. Virtual extent is not payload occupancy. Opens return RuntimeMemory; explicit MemoryManagerConfig selects fresh-state buckets or checks a persisted setting without migration. The default remains 128 pages.

MemoryRuntime is the canonical owner for one backing memory instance. It owns that memory’s manager, ledger persistence, bootstrap lifecycle, committed capability, opens, and diagnostics. Each bootstrap attempt fallibly decodes its ledger record and uses a temporary cell for capacity-checked writes. Linked code contributes declarations to one immutable SealedDeclarationSnapshot, which is supplied to each runtime independently.

AllocationBootstrap is the golden path for whichever layer owns a given ledger store. Canic may own bootstrap for a framework canister and compose IcyDB/application declarations through its registry; IcyDB may own bootstrap directly for generated database stores; or a standalone application canister may own bootstrap itself. Exactly one owner should bootstrap one ledger store. Multiple layers in the same canister must either compose declarations into that owner or use distinct ledger stores and allocation domains.

ic-stable-structures MemoryManager IDs are the first-class supported physical slot substrate. That ID domain is u8: IDs 0..=254 are usable, and ID 255 is always the ic-stable-structures unallocated sentinel. The crate still keeps narrow internal abstractions for storage adapters and diagnostics, but the native IC path is MemoryManager ID 0 -> ic-stable-structures::Cell<StableCellLedgerRecord, _> -> LedgerCommitStore -> CommittedGenerationBytes -> LedgerPayloadEnvelope -> RecoveredLedger -> ValidatedAllocations -> CommittedAllocations.

ic_stable_structures re-exports the exact substrate version used by this crate. Use its collections and traits with RuntimeMemory handles; ic-memory owns allocation governance without wrapping typed collections.

Re-exports

pub use ic_stable_structures;

Macros

eager_init

Register one pre-bootstrap hook.

ic_memory_declaration

Register a MemoryManager allocation declaration during static initialization.

ic_memory_key

Declare and open a committed MemoryManager slot by stable key.

ic_memory_range

Declare a MemoryManager allocation range during static initialization.

Structs

AllocationBootstrap

AllocationBootstrap

AllocationDeclaration

AllocationDeclaration

AllocationLedger

AllocationLedger

AllocationRangeClaim

AllocationRangeClaim

AllocationRecord

AllocationRecord

AllocationRetirement

AllocationRetirement

BootstrapAdmission

BootstrapAdmission

CommitStoreDiagnostic

CommitStoreDiagnostic

CommittedAllocations

CommittedAllocations

CommittedGenerationBytes

CommittedGenerationBytes

DeclarationSnapshot

DeclarationSnapshot

DiagnosticDeclaration

DiagnosticDeclaration

DiagnosticExport

DiagnosticExport

DiagnosticFailure

DiagnosticFailure

DiagnosticMemorySize

DiagnosticMemorySize

DiagnosticRangeAuthority

DiagnosticRangeAuthority

DiagnosticRecord

DiagnosticRecord

DiagnosticRuntimeBinding

DiagnosticRuntimeBinding

DiagnosticStableCell

DiagnosticStableCell

DualCommitStore

DualCommitStore

GenericRangePolicy

GenericRangePolicy

LedgerCommitStore

LedgerCommitStore

LedgerPayloadEnvelope

LedgerPayloadEnvelope

MemoryAllocation

MemoryAllocation

MemoryAllocationSummary

MemoryAllocationSummary

MemoryAllocations

MemoryAllocations

MemoryBindingSummary

MemoryBindingSummary

MemoryManagerAuthorityRecord

MemoryManagerAuthorityRecord

MemoryManagerConfig

MemoryManagerConfig

MemoryManagerIdRange

MemoryManagerIdRange

MemoryManagerRangeAuthority

MemoryManagerRangeAuthority

MemoryManagerSlot

MemoryManagerSlot

MemoryRequest

MemoryRequest

MemoryRuntime

MemoryRuntime

MemoryRuntimeDoctorReport

MemoryRuntimeDoctorReport

PendingBootstrapCommit

PendingBootstrapCommit

PolicyIdentity

PolicyIdentity

RecoveredAllocationMetadata

RecoveredAllocationMetadata

RecoveredLedger

RecoveredLedger

RuntimeMemory

RuntimeMemory

SchemaMetadata

SchemaMetadata

SealedDeclarationFingerprint

SealedDeclarationFingerprint

SealedDeclarationSnapshot

SealedDeclarationSnapshot

StableCellLedgerRecord

StableCellLedgerRecord

StableKey

StableKey

StableKeyError

StableKeyError

StaticMemoryDeclaration

StaticMemoryDeclaration

StaticMemoryRangeDeclaration

StaticMemoryRangeDeclaration

ValidatedAllocations

ValidatedAllocations

Enums

AllocationBinding

AllocationBinding

AllocationReservationError

AllocationReservationError

AllocationRetirementError

AllocationRetirementError

AllocationStageError

AllocationStageError

AllocationState

AllocationState

AllocationValidationError

AllocationValidationError

BootstrapAdmissionError

BootstrapAdmissionError

BootstrapError

BootstrapError

BootstrapReservationError

BootstrapReservationError

BootstrapRetirementError

BootstrapRetirementError

CommitRecoveryError

CommitRecoveryError

CommitSlotDiagnostic

CommitSlotDiagnostic

DeclarationSnapshotError

DeclarationSnapshotError

DiagnosticCheck

DiagnosticCheck

DiagnosticCode

DiagnosticCode

DiagnosticStableCellStatus

DiagnosticStableCellStatus

LedgerCommitError

LedgerCommitError

LedgerIntegrityError

LedgerIntegrityError

LedgerPayloadEnvelopeError

LedgerPayloadEnvelopeError

MemoryManagerLayoutError

MemoryManagerLayoutError

MemoryManagerRangeAuthorityError

MemoryManagerRangeAuthorityError

MemoryManagerRangeError

MemoryManagerRangeError

MemoryManagerRangeMode

MemoryManagerRangeMode

MemoryManagerSlotError

MemoryManagerSlotError

MemoryResolutionError

MemoryResolutionError

PolicyIdentityError

PolicyIdentityError

RuntimeAdoptionError

RuntimeAdoptionError

RuntimeBootstrapError

RuntimeBootstrapError

RuntimeConstructionError

RuntimeConstructionError

RuntimeDiagnosticError

RuntimeDiagnosticError

RuntimeGrowError

RuntimeGrowError

RuntimeOpenError

RuntimeOpenError

RuntimePolicyError

RuntimePolicyError

RuntimeStateError

RuntimeStateError

SchemaMetadataError

SchemaMetadataError

StableCellLedgerError

StableCellLedgerError

StableCellPayloadError

StableCellPayloadError

StaticMemoryDeclarationError

StaticMemoryDeclarationError

Constants

IC_MEMORY_AUTHORITY_OWNER

Diagnostic owner label for ic-memory allocation-governance infrastructure.

IC_MEMORY_AUTHORITY_PURPOSE

Diagnostic purpose for the ic-memory allocation-governance authority range.

IC_MEMORY_LEDGER_LABEL

Diagnostic label of the allocation ledger when backed by the current MemoryManager substrate.

IC_MEMORY_LEDGER_STABLE_KEY

Stable key of the allocation ledger when backed by the current MemoryManager substrate.

IC_MEMORY_STABLE_KEY_PREFIX

Stable-key namespace prefix reserved for ic-memory allocation-governance infrastructure.

LEDGER_PAYLOAD_FORMAT_VERSION

Current durable ledger payload format version.

MAX_LEDGER_BYTES

Maximum encoded ownership ledger size (64 KiB for at most 255 records).

MAX_LEDGER_NESTING

Maximum CBOR container nesting on maintained decode paths.

MAX_LEDGER_RECORD_BYTES

Two bounded CBOR byte strings plus record metadata (128 KiB + 4 KiB).

MEMORY_MANAGER_GOVERNANCE_MAX_ID

Last MemoryManager ID reserved for ic-memory governance in the current substrate.

MEMORY_MANAGER_INVALID_ID

MemoryManager unallocated-bucket sentinel. This is not a usable slot.

MEMORY_MANAGER_LEDGER_ID

MemoryManager ID used by the allocation ledger in the current MemoryManager substrate.

MEMORY_MANAGER_MAX_ID

Last usable MemoryManager virtual memory ID.

MEMORY_MANAGER_MIN_ID

First usable MemoryManager virtual memory ID.

STABLE_CELL_HEADER_SIZE

Stable-cell header byte length.

STABLE_CELL_LAYOUT_VERSION

Stable-cell layout version supported by this adapter.

STABLE_CELL_MAGIC

Stable-cell magic prefix written by ic-stable-structures::Cell.

STABLE_CELL_VALUE_OFFSET

Byte offset where the stable-cell value payload starts.

WASM_PAGE_SIZE_BYTES

WebAssembly page size used by ic-stable-structures memory implementations.

Traits

AllocationPolicy

AllocationPolicy

RuntimeBootstrapPolicy

RuntimeBootstrapPolicy

Functions

bootstrap_default_memory_manager

Bootstrap this thread’s default runtime using generic range policy.

bootstrap_default_memory_manager_with_config

Bootstrap the default runtime with an explicit bucket setting and allocation policy.

bootstrap_default_memory_manager_with_policy

Bootstrap this thread’s default runtime with caller-supplied policy.

committed_allocations

Return this thread’s default runtime committed allocation capability.

decode_stable_cell_ledger_record

Decode a StableCellLedgerRecord from stable-cell value bytes.

decode_stable_cell_ledger_record_from_memory

Fallibly decode a ledger record from its stable-cell envelope and value.

decode_stable_cell_payload

Decode the raw value payload from an ic-stable-structures::Cell memory.

default_memory_manager_commit_recovery_diagnostic

Diagnose protected commit recovery for this thread’s default runtime.

default_memory_manager_diagnostic_export

Export this thread’s default runtime ledger and live memory sizes.

default_memory_manager_doctor_report

Build preflight and lifecycle diagnostics for this thread’s default runtime.

default_memory_manager_doctor_report_with_policy

Build diagnostics for this thread’s default runtime under one explicit policy.

default_memory_manager_memory_allocation_summary

Measure numeric allocation totals in the existing default runtime.

default_memory_manager_memory_allocations

Measure the existing default runtime without constructing a manager or initializing backing memory.

default_memory_manager_memory_id

Resolve an application key’s committed ID in the existing default runtime. Does not construct a manager, open memory, or choose a bucket configuration.

is_default_memory_manager_bootstrapped

Return whether this thread’s default runtime has completed bootstrap.

is_ic_memory_stable_key

Return true when stable_key belongs to the ic-memory namespace.

memory_manager_governance_range

MemoryManager range reserved for ic-memory governance in the current substrate.

open_default_memory_manager_memory

Open a committed memory from this thread’s default runtime. Does not construct an absent runtime or select its bucket configuration.

open_default_memory_manager_memory_by_key

Open a key already committed by the host’s default runtime without changing policy. Does not construct an absent runtime or select its bucket configuration.

register_memory_request

Register a key-only request before the linked snapshot seals.

register_static_memory_declaration

Register one allocation declaration before bootstrap seals the snapshot.

register_static_memory_manager_declaration

Register one MemoryManager declaration before bootstrap seals the snapshot.

register_static_memory_manager_declaration_with_schema

Register one MemoryManager declaration with schema metadata.

register_static_memory_manager_range

Register one MemoryManager authority range before bootstrap seals the snapshot.

register_static_memory_range_declaration

Register one authority range declaration before bootstrap seals the snapshot.

sealed_declaration_snapshot

Seal and return the canonical linked-program declaration snapshot.

validate_allocations

Validate a committed ledger and current declarations before opening.

validate_memory_manager_id

Validate that a MemoryManager ID is usable as an allocation slot.

verify_default_memory_manager_authority

Verify one consumer’s allocation requirements against the existing host runtime. Does not construct, bootstrap, replay admission or change configuration.