Crate sc_statement_store 

Statement Store

The statement store is an off-chain, decentralized data store for cryptographically signed statements in the Polkadot SDK. It lets accounts publish arbitrary data that can be queried and propagated across the network without consuming on-chain storage.

Key characteristics:

• Off-chain storage — keeps the data off-chain to avoid blockchain state bloat.
• Gossip-based distribution — statements propagate node-to-node over a notification protocol.
• Cryptographic verification — every statement is signed and verified before acceptance.
• Topic-based indexing — up to four topics per statement for efficient querying.
• Account-based limits — per-account quotas bound how much an account may store.
• Automatic eviction — lower-priority statements are evicted when limits are reached.
• Multiple access methods — reachable over JSON-RPC for applications and via the runtime API (offchain workers) for on-chain logic.

Design principles

The service is built around two pillars — scalability and graceful degradation — and deliberately avoids the properties of a centralized data structure, prioritizing privacy, scalability, and decentralization by reducing other guarantees:

• No centralized-system guarantees. The store is weakly coherent: it does not guarantee message delivery or specific delivery times, operating on a best-effort basis. Under saturation, quality of service (latency, propagation speed) degrades gracefully rather than the system halting.
• Generic transport layer. It does not natively guarantee data confidentiality, integrity, or authenticity; applications must layer those properties on top (e.g. their own encryption).
• Local enforcement. Global quotas cannot be enforced instantly in a decentralized system, so nodes enforce quotas locally and global consistency is only eventual. Node operators may also apply their own storage limits or quality-of-service.

Because delivery is not guaranteed — notably under full network saturation, malicious nodes, or network partitioning/ISP failures — the store is best thought of as closer to UDP than TCP, and applications should use fail-safe mechanisms that confirm a message was delivered before proceeding.

Architecture

The implementation is split across several crates:

• sp-statement-store — the Statement structure, core types, the StatementStore runtime and client interfaces, and cryptographic primitives.
• sc-statement-store — the disk-backed (ParityDB) store: constraint management, the on-disk indexes with their bounded in-memory caches, and subscription handling.
• sc-network-statement — gossip-based propagation, per-peer state, and topic affinity.
• pallet-statement — the runtime pallet: it turns on-chain statement events into statements (via an offchain worker) and defines the bounds used to compute account allowances.
• sc-rpc-api / sc-rpc — the JSON-RPC API (a statement subscription and a submit method) used by external clients; see StatementApiClient.

Account quota

An account’s budget is a StatementAllowance { max_count, max_size }, enforced on two axes at the same time:

• max_count — the maximum number of statements the account may have stored.
• max_size — the maximum total data size in bytes (the length of each statement’s data field, not its full SCALE-encoded size).

Because both are hard limits, an account may spend its budget as a few large statements or many small ones, up to whichever limit it reaches first. For example, with max_size = 50 KiB and max_count = 50: an account could store two 25 KiB statements (filling the size budget but using only 2 of 50 slots), or fifty 1 KiB statements (reaching both limits at once).

When a submission would exceed either limit, the account’s lowest-priority statements (those with the lowest expiry) are evicted to make room. If a single statement’s data exceeds max_size it is rejected outright; and if, even after evicting every lower-priority statement, the new one still does not fit, it is rejected too. Global limits apply on top of per-account limits: the store holds at most DEFAULT_MAX_TOTAL_STATEMENTS statements and DEFAULT_MAX_TOTAL_SIZE of data.

Allowances are not fixed in code: they are held in chain state (keyed under STATEMENT_ALLOWANCE_PREFIX) and granted or revoked by the runtime; an account with no allowance — or a depleted one — cannot store statements. The bounds within which a runtime may set allowances are configured by pallet-statement.

Channels

A channel is an optional per-account identifier (Option<Channel>, 32 bytes) used for message replacement: only one statement per (account, channel) pair is stored at a time.

Submitting a new statement on a channel that already has one replaces the previous statement from the same account only if the new statement has a strictly higher expiry (priority). Replacing frees the old statement’s size; if the new statement is larger and replacing the old one is not enough to stay within the account quota, additional lowest-priority statements from the same account are evicted, and if it still does not fit the submission is rejected.

A statement with no channel is never subject to channel uniqueness — it is only ever removed by priority-based eviction or expiry. The store does not prescribe how a channel id is generated.

Topics

Topics are the primary attribute used to query and filter statements. A statement carries up to MAX_TOPICS (4) topics, each a 32-byte identifier, and the store maintains a topic -> statements index for fast lookups.

Subscriptions filter by topic in one of three ways:

• Any — match every statement.
• MatchAll(topics) — match statements that include all of the given topics (up to MAX_TOPICS = 4 topics; an empty set matches everything).
• MatchAny(topics) — match statements that include any of the given topics (up to MAX_ANY_TOPICS = 128 topics).

Topics are neither private nor reserved: any statement may use any topic, so it is the application’s responsibility to filter or validate them. At the network layer, peers can additionally advertise a topic affinity (a bloom filter) so that only statements matching their interests are propagated to them.

Best practices

• Do not use the store for synchronous communication. It is designed for asynchronous hand-offs and signaling, not real-time exchange between nodes or apps.
• Do not route peer-to-peer data through the store. High-bandwidth or continuous peer-to-peer traffic should go over a direct channel (e.g. WebRTC); use the statement store only to signal the setup of such a connection, not to carry its traffic.
• Do not rely on timely or guaranteed delivery. Network congestion can delay or drop propagation — treat the store as closer to UDP than TCP.

Integrating the statement store

This guide covers what an application developer needs to know to integrate the statement store. It focuses on integration-specific guidance; for the concepts and mechanics it refers to the crate overview above and the API docs (StatementStore, Statement, and the RPC client StatementApiClient) rather than repeating them.

Authorization and signing

There is no separate authentication step. Every statement must be signed by the submitting account’s key, and the store authorizes submission by checking that the account has a non-zero allowance (StatementAllowance); an account with no allowance cannot store statements.

Channels and topics in practice

Channels act as a single-slot mailbox per (account, channel) pair: only one statement per channel is kept, and a new one replaces the old one only if it has strictly higher priority (a greater expiry). This makes them ideal for replaceable state, keeping only the latest value on a channel. A lower-or-equal-priority submission on an existing channel is rejected with RejectionReason::ChannelPriorityTooLow. There is no built-in way to fetch statements by channel — retrieval is by topic only.

Topics are network-wide, public, and unreserved: anyone may post to any topic, and every subscriber listening on a topic receives matching statements. They are the main element for grouping related statements. The store does not prescribe how a TopicID (32 bytes) is chosen; two common patterns and their trade-offs:

• hash(app_name + public_key) — convenient, but since the public key is known this lets an observer correlate all of an application’s statements for that key (metadata leakage).
• hash(public_key + entropy) — avoids that correlation where no metadata leakage is acceptable.

It is the application’s responsibility to filter and validate topics (see Privacy and security).

Expiry and priority

A statement’s expiry is a single u64 that packs an expiration timestamp (seconds since the UNIX epoch) in the high 32 bits and a sequence number in the low 32 bits; a higher value is higher priority. Eviction is purely priority-based and treats channel and non-channel statements uniformly — there is no separate per-channel eviction.

For example, to keep only the latest value on a channel, use a fixed future expiry timestamp with an incrementing sequence number:

// Two successive updates share the same channel and the same future expiry timestamp, but the
// sequence number increments so the newer one outranks the older.
let expires_at: u64 = 0x69F5_4B54; // ~30 days in the future, seconds since the UNIX epoch

let first: u64 = (expires_at << 32) | 1;
let second: u64 = (expires_at << 32) | 2;

// Higher priority, so submitting `second` on the same channel replaces the `first` statement.
assert!(second > first);
assert_eq!(second, 0x69F5_4B54_0000_0002);

A caveat for client design: within a shared application namespace, a statement with a very high expiry can crowd out others, and one with a very low expiry is easily evicted — choose priorities deliberately.

Submission outcomes and errors

Submitting returns a SubmitResult: new (accepted), known / knownExpired (already seen), or a failure. Failures are of two kinds.

Rejections (RejectionReason) — the statement is valid but does not fit:

• channelPriorityTooLow (RejectionReason::ChannelPriorityTooLow) — a statement already exists on this (account, channel) and the new one is not strictly higher priority. Because a channel is a single-slot mailbox, replacement requires a strictly higher expiry; this stops an attacker from cheaply overwriting an existing channel message.
• dataTooLarge — the data exceeds the account’s remaining size allowance.
• accountFull — the account is at its statement limit and the new expiry is too low to evict an existing one.
• storeFull — the global store is full.
• noAllowance — the account has no allowance to use the store.

Validation failures (InvalidReason) — the statement is malformed or unusable:

• noProof / badProof — missing or invalid signature.
• encodingTooLarge — the encoded statement exceeds the network size limit.
• alreadyExpired — the expiry timestamp is already in the past.

Duplicates, retries, and delivery

The store does not guarantee delivery or timing (see the overview’s design principles). In practice:

• Deduplicate on the client. On reconnecting with a filter you receive all currently matching statements again, so applications must handle duplicates.
• Handle retries and fail-safes. Under network saturation a statement may not propagate; applications should retry and confirm delivery according to their own logic before proceeding.

Privacy and security

As a transport layer the store provides no privacy or integrity guarantees of its own; applications build those on top. The main considerations:

• Metadata leakage (IP). Submitting statements to a node reveals network metadata such as the client’s IP address. Mitigation: end users should use a VPN.
• No confidentiality or integrity. The store neither encrypts nor verifies application data. Mitigation: the application must encrypt and sign its payloads.
• Identity correlation. A signing key does not directly identify a person, but reusing one keypair across interactions (including with other services) allows correlation. Mitigation: use a voucher-based privacy mechanism provided by the identity layer.
• Unverified statement injection. A client trusts the node it queries to return only valid statements, and because topics are open, a malicious node can inject spam — even spam encrypted to the user’s public key. The application must therefore filter unwelcome statements and validate that topics are relevant and senders are trusted.

For general usage do’s and don’ts (asynchronous use only, not routing peer-to-peer traffic through the store), see the best-practices section of the overview.

Implementation notes

This crate contains a disk-backed implementation of sp_statement_store::StatementStore.

Constraint management

The statement store validates statements using node-side signature verification and static runtime allowance limits. The following constraints are then checked:

• For a given account id, there may be at most max_count statements with max_size total data size. To satisfy this, statements for this account ID are removed from the store starting with the lowest priority until a constraint is satisfied.
• There may not be more than MAX_TOTAL_STATEMENTS total statements with MAX_TOTAL_SIZE size. To satisfy this, statements are removed from the store starting with the lowest global_priority until a constraint is satisfied.

When a new statement is inserted that would not satisfy constraints in the first place, no statements are deleted and a Rejected result is returned. The order in which statements with the same priority are deleted is unspecified.

Statement expiration

Each time a statement is removed from the store (Either evicted by higher priority statement or explicitly with the remove function) the statement is marked as expired. Expired statements can’t be added to the store for Options::purge_after_sec seconds. This is to prevent old statements from being propagated on the network.

Structs

Config

Statement store and network handler configuration.

MultiFilterEventStream

Stream of multi-filter subscription events

Store

Statement store.

SubscriptionHandle

A handle that attaches, removes, and inspects filters for one multi-filter subscription

Enums

AddFilterError

Error returned when attaching a filter to a multi-filter subscription fails

Error

Statement store error.

MultiFilterSubscriptionEvent

Event emitted by a multi-filter subscription

Constants

DEFAULT_MAX_TOTAL_SIZE

The maximum amount of data the statement store can hold, regardless of the number of statements from which the data originates.

DEFAULT_MAX_TOTAL_STATEMENTS

The maximum number of statements the statement store can hold.

DEFAULT_NETWORK_WORKERS

Default number of concurrent workers for statement validation.

DEFAULT_PURGE_AFTER_SEC

The amount of time an expired statement is kept before it is removed from the store entirely.

DEFAULT_RATE_LIMIT

Default maximum statements per second per peer before rate limiting kicks in. Default maximum statements per second before rate limiting kicks in.

MAX_FILTERS_PER_SUBSCRIPTION

Maximum number of active filters attached to one statement subscription.

MAX_STATEMENT_SIZE

The maximum size of a single statement in bytes. Accounts for the 1-byte vector length prefix when statements are gossiped as Vec<Statement>.

MAX_TOPICS

Total number of topic fields allowed in a statement and in MatchAll filters.

Traits

MultiFilterSubscriptionApi

Creates multi-filter subscriptions for the RPC module

StatementStore

Statement store API.

StatementStoreSubscriptionApi

Trait for initiating statement store subscriptions from the RPC module.