Struct RemoteStorageLayer 

pub struct RemoteStorageLayer { /* private fields */ }

Remote storage layer that lazily fetches state from a live chain.

Provides a cache-through abstraction: reads check the local cache first, and only fetch from the remote RPC when the value isn’t cached. Fetched values are automatically cached for subsequent reads.

Cloning

RemoteStorageLayer is cheap to clone. Both ForkRpcClient and StorageCache use internal reference counting (connection pools/Arc), so cloning just increments reference counts.

Thread Safety

The layer is Send + Sync and can be shared across async tasks. The underlying cache handles concurrent access safely.

Implementations

impl RemoteStorageLayer

pub fn new(rpc: ForkRpcClient, cache: StorageCache) -> Self

Create a new remote storage layer.

Arguments

• rpc - RPC client connected to the live chain
• cache - Storage cache for persisting fetched values

pub fn rpc(&self) -> &ForkRpcClient

Get a reference to the underlying RPC client.

pub fn cache(&self) -> &StorageCache

Get a reference to the underlying cache.

pub fn endpoint(&self) -> &Url

Get the RPC endpoint URL this layer is connected to.

pub fn stats(&self) -> StorageStatsSnapshot

Take a snapshot of the current storage access counters.

pub fn reset_stats(&self)

Reset all storage access counters to zero.

pub async fn get( &self, block_hash: H256, key: &[u8], ) -> Result<Option<Vec<u8>>, RemoteStorageError>

Get a storage value, fetching from RPC if not cached.

Returns

• Ok(Some(value)) - Storage exists with value
• Ok(None) - Storage key doesn’t exist (empty)
• Err(_) - RPC or cache error

Caching Behavior

• If the key is in cache, returns the cached value immediately
• If not cached and the key is >= 32 bytes, speculatively prefetches the first page of keys sharing the same 32-byte prefix (pallet hash + storage item hash). This converts hundreds of individual RPCs into a handful of bulk fetches without risking a full scan of large maps.
• Falls back to individual RPC fetch if the key is short or the speculative prefetch didn’t cover it (key beyond first page).
• Empty storage (key exists but has no value) is cached as None

pub async fn get_batch( &self, block_hash: H256, keys: &[&[u8]], ) -> Result<Vec<Option<Vec<u8>>>, RemoteStorageError>

Get multiple storage values in a batch, fetching uncached keys from RPC.

Arguments

• block_hash - The hash of the block being queried.
• keys - Slice of storage keys to fetch (as byte slices to avoid unnecessary allocations)

Returns

A vector of optional values, in the same order as the input keys.

Caching Behavior

• Checks cache for all keys first
• Only fetches uncached keys from RPC
• Caches all fetched values (including empty ones)
• Returns results in the same order as input keys

pub async fn prefetch_prefix( &self, block_hash: H256, prefix: &[u8], page_size: u32, ) -> Result<usize, RemoteStorageError>

Prefetch a range of storage keys by prefix (resumable).

Fetches all keys matching the prefix and caches their values. This operation is resumable - if interrupted, calling it again will continue from where it left off.

Arguments

• block_hash.
• prefix - Storage key prefix to match
• page_size - Number of keys to fetch per RPC call

Returns

The total number of keys for this prefix (including previously cached).

pub async fn prefetch_prefix_single_page( &self, block_hash: H256, prefix: &[u8], page_size: u32, ) -> Result<usize, RemoteStorageError>

Fetch a single page of keys for a prefix and cache their values.

Unlike prefetch_prefix, this fetches only the first page of keys (up to page_size) without looping through subsequent pages. This keeps the cost bounded regardless of how many keys exist under the prefix.

Records scan progress so that subsequent calls to prefetch_prefix can resume from where this left off.

pub async fn get_keys( &self, block_hash: H256, prefix: &[u8], ) -> Result<Vec<Vec<u8>>, RemoteStorageError>

Get all keys for a prefix, fetching from RPC if not fully cached.

This is a convenience method that:

1. Ensures the prefix is fully scanned (calls Self::prefetch_prefix if needed)
2. Returns all cached keys matching the prefix

Useful for enumerating all entries in a storage map (e.g., all accounts in a balances pallet).

Arguments

• block_hash - Block hash to query at
• prefix - Storage key prefix to match (typically a pallet + storage item prefix)

Returns

All keys matching the prefix at the specified block hash.

Performance

First call may be slow if the prefix hasn’t been scanned yet. Subsequent calls return cached data immediately.

pub async fn fetch_and_cache_block_by_number( &self, block_number: u32, ) -> Result<Option<BlockRow>, RemoteStorageError>

Fetch a block by number from the remote RPC and cache it.

This method fetches the block data for the given block number and caches the block metadata in the cache.

Arguments

• block_number - The block number to fetch and cache

Returns

• Ok(Some(block_row)) - Block was fetched and cached successfully
• Ok(None) - Block number doesn’t exist
• Err(_) - RPC or cache error

Caching Behavior

• Fetches block hash and data from block number using chain_getBlockHash and chain_getBlock
• Caches block metadata (hash, number, parent_hash, header) in the cache
• If block is already cached, this will update the cache entry

pub async fn next_key( &self, block_hash: H256, prefix: &[u8], key: &[u8], ) -> Result<Option<Vec<u8>>, RemoteStorageError>

Get the next key after the given key that starts with the prefix.

This method is used for key enumeration during runtime execution. Before hitting the RPC, it checks whether a complete prefix scan exists in the cache for the queried prefix (or parent prefixes at 32 or 16 bytes). If so, the answer is served from the local SQLite cache, avoiding an RPC round-trip entirely.

Arguments

• block_hash - Block hash to query at
• prefix - Storage key prefix to match
• key - The current key; returns the next key after this one

Returns

• Ok(Some(key)) - The next key after key that starts with prefix
• Ok(None) - No more keys with this prefix

pub async fn block_body( &self, hash: H256, ) -> Result<Option<Vec<Vec<u8>>>, RemoteStorageError>

Get block body (extrinsics) by hash from the remote chain.

Returns

• Ok(Some(extrinsics)) - Block found, returns list of encoded extrinsics
• Ok(None) - Block not found

pub async fn block_header( &self, hash: H256, ) -> Result<Option<Vec<u8>>, RemoteStorageError>

Get block header by hash from the remote chain.

Returns

• Ok(Some(header_bytes)) - Encoded header bytes
• Ok(None) - Block not found on the remote chain
• Err(..) - Transport/connection error (caller should retry or reconnect)

pub async fn block_hash_by_number( &self, block_number: u32, ) -> Result<Option<H256>, RemoteStorageError>

Get block hash by block number from the remote chain.

Returns

• Ok(Some(hash)) - Block hash at the given number
• Ok(None) - Block number not found

pub async fn block_number_by_hash( &self, hash: H256, ) -> Result<Option<u32>, RemoteStorageError>

Get block number by hash from the remote chain.

This method checks the persistent SQLite cache first before hitting RPC. Results are cached for future lookups.

Returns

• Ok(Some(number)) - Block number for the given hash
• Ok(None) - Block not found

pub async fn parent_hash( &self, hash: H256, ) -> Result<Option<H256>, RemoteStorageError>

Get parent hash of a block from the remote chain.

This method checks the persistent SQLite cache first before hitting RPC. Results are cached for future lookups.

Returns

• Ok(Some(parent_hash)) - Parent hash of the block
• Ok(None) - Block not found

pub async fn block_by_number( &self, block_number: u32, ) -> Result<Option<(H256, Block<SubstrateConfig>)>, RemoteStorageError>

Get full block data (hash and block) by number from the remote chain.

Returns

• Ok(Some((hash, block))) - Block found
• Ok(None) - Block number not found

pub async fn finalized_head(&self) -> Result<H256, RemoteStorageError>

Get the latest finalized block hash from the remote chain.

pub async fn metadata( &self, block_hash: H256, ) -> Result<Metadata, RemoteStorageError>

Get decoded metadata at a specific block from the remote chain.

Trait Implementations

impl Clone for RemoteStorageLayer

fn clone(&self) -> RemoteStorageLayer

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RemoteStorageLayer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.