Expand description
An immutable key-value store for ordered data with a minimal memory footprint.
Data is stored in a crate::freezer::Freezer and a crate::ordinal::Ordinal to enable lookups by both index and key with minimal memory overhead.
§Uniqueness
Archive assumes all stored indexes and keys are unique. If the same key is associated with
multiple indices, there is no guarantee which value will be returned. If the key is written to
an existing index, Archive will return an error.
§Compression
Archive supports compressing data before storing it on disk. This can be enabled by setting
the compression field in the Config struct to a valid zstd compression level. This setting
can be changed between initializations of Archive, however, it must remain populated if any
data was written with compression enabled.
§Querying for Gaps
Archive tracks gaps in the index space to enable the caller to efficiently fetch unknown keys
using next_gap. This is a very common pattern when syncing blocks in a blockchain.
§Example
use commonware_runtime::{Spawner, Runner, deterministic, buffer::PoolRef};
use commonware_cryptography::{Hasher as _, Sha256};
use commonware_storage::{
archive::{
Archive as _,
immutable::{Archive, Config},
},
};
use commonware_utils::{NZUsize, NZU64};
let executor = deterministic::Runner::default();
executor.start(|context| async move {
// Create an archive
let cfg = Config {
metadata_partition: "metadata".into(),
freezer_table_partition: "table".into(),
freezer_table_initial_size: 65_536,
freezer_table_resize_frequency: 4,
freezer_table_resize_chunk_size: 16_384,
freezer_journal_partition: "journal".into(),
freezer_journal_target_size: 1024,
freezer_journal_compression: Some(3),
freezer_journal_buffer_pool: PoolRef::new(NZUsize!(1024), NZUsize!(10)),
ordinal_partition: "ordinal".into(),
items_per_section: NZU64!(1024),
write_buffer: NZUsize!(1024),
replay_buffer: NZUsize!(1024),
codec_config: (),
};
let mut archive = Archive::init(context, cfg).await.unwrap();
// Put a key
archive.put(1, Sha256::hash(b"data"), 10).await.unwrap();
// Close the archive (also closes the freezer and ordinal)
archive.close().await.unwrap();
});