Expand description
This crate is a parser for Substrate chain data. It could be used to decode signable transactions, calls, events, storage items etc. with chain metadata. Decoded data could be pattern matched or represented in readable form.
Key trait AsMetadata
describes the metadata suitable for parsing of
signable transactions and other encoded chain items, for example, the data
from chain storage. Trait AsCompleteMetadata
is AsMetadata
with few
additional properties, it describes the metadata suitable for parsing of
unchecked extrinsics.
Traits AsMetadata
and AsCompleteMetadata
could be applied as well to
metadata addressable in external memory. As metadata typically is typically
a few hundred kB, this is useful for hardware devices with limited memory
capacity.
AsMetadata
and AsCompleteMetadata
are implemented for RuntimeMetadata
versions V14
and V15
, both of which have conveniently in-built types
registry allowing to track types using metadata itself without any
additional information.
§Assumptions
Chain data is SCALE-encoded. Data blobs entering decoder are expected to be decoded completely: all provided bytes must be used in decoding with no data remaining unparsed.
For decoding either the entry type (such as the type of particular storage item) or the data internal structure used to find the entry type in metadata (as is the case for signable transactions or unchecked extrinsics) must be known.
Entry type gets resolved into constituting types with metadata in-built
types registry and appropriate bytes chunks are selected from input blob
and decoded. The process follows what the decode
from the
SCALE codec does, except the types that go into the
decoder are found dynamically during the decoding itself.
§Signable transactions
Signable transaction consist of the call part and extensions part.
Call part may or may not be double SCALE-encoded, i.e. SCALE-encoded call data may or may not be preceded by compact of the encoded call length.
Function parse_transaction
is used for signable transactions with double
SCALE-encoded call data. Call length allows to separate encoded call data
and extensions, and decode them independently, extensions first. This
approach is preferable if multiple metadata entries (same chain, different
spec_version
) are tried for transaction parsing, because extensions must
contain metadata spec_version
, thus allowing to check if the correct one
is being used for decoding before call decoding even starts.
Signable transactions without length prefix are parsed with function
parse_transaction_unmarked
, call first. Similarly, found in extensions
chain spec_version
is checked to assure the correct metadata was used.
Call parsing entry point is call_ty
. This is the type describing all calls
available on chain. Effectively, call_ty
leads to enum with variants
corresponsing to all pallets, first u8
of the data is the pallet index.
Each variant is expected to have only one field, the type of which is also
an enum. This second enum represents all calls available in the selected
pallet, with second u8
of the data being enum variant index, i.e. exact
call contained in transaction. Further data is just the set of fields for
this selected variant.
Remaining data is SCALE-encoded set of signable extensions, as declared, for
example, in
v14::ExtrinsicMetadata
for V14
and in v15::ExtrinsicMetadata
for V15
. Chain genesis hash must be found among the decoded extensions and
must match the genesis hash known for the chain, if the one was provided for
parser. spec_version
must be found among the decoded extensions and must
match the spec_version
derived from the provided metadata.
§Storage items
Storage items could be queried from chain via rpc calls, and the retrieved
SCALE-encoded data has a type declared in corresponding chain metadata
StorageEntryType
.
Storage items (combination of both a key and a value) is parsed using
decode_as_storage_entry
function.
§Unchecked extrinsics
Unchecked extrinsics could be decoded with metadata implementing
AsCompleteMetadata
trait, using function
decode_as_unchecked_extrinsic
.
§Other items
Any part of a bytes blob could be decoded if the corresponding type is known
using function decode_as_type_at_position
.
For cases when whole blob corresponds to a single known type, function
decode_all_as_type
is suggested.
§Parsed data and cards
Parsing data with a given type results in ExtendedData
. Parsing data as
a call results in Call
. Both types are complex and may (and usually
do) contain layered parsed data inside. During parsing itself as much as
possible of internal data structure, identifiers and docs are preserved so
that it is easier to find information in parsed items or pattern-match them.
All parsed results may be carded. Cards are flat formatted elements,
with type-associated information, that could be printed or otherwise
displayed to user. Each Call
and ExtendedData
gets carded into
Vec<ExtendedCard>
.
§Special types
Types, as stored in the metadata types registry, have associated
Path
information. The ident
segment of the Path
is
used to detect the special types.
Some Path
identifiers are used without further checking, such as
well-known array-based types (AccountId32
, hashes, public keys, signatures
etc) or other types with known or easily determined encoded size, such as
Era
, PerThing
items etc.
Other Path
identifiers are checked first, and used only if the further
discovered type information matches the expected one, this is the case for
Call
and Event
. If it does not match, the data is parsed as is, i.e.
without fitting into specific item format.
Enums and structs contain sets of Field
s. Field
name
and type_name
may also hint at type specialty information, although
less reliably than the Path
. Such hints do not cause errors in parser flow
if appear unexpectedly, and just get ignored.
Field could contain currency-related data. When carded and displayed, currency is displayed with chain decimals and units only if encountered in a one of balance-displaying pallets or in extensions.
Some types (AccountId32
, public key types, signature types) are used as
re-defined in lightweight crate substrate_crypto_light
, Era
is
re-defined in module additional_types
. These types are similar to their
original counterparts in sp_core
and sp_runtime
crates. Re-defining
is done to ensure no_std
compatibility and simplify dependencies tree.
Internal content of these types could be seamlessly transferred into
original types if need be.
§Features
Crate supports no_std
in default-features = false
mode.
§Examples
use frame_metadata::v14::RuntimeMetadataV14;
use parity_scale_codec::Decode;
use primitive_types::H256;
use scale_info::{IntoPortable, Path, Registry};
use std::str::FromStr;
use substrate_crypto_light::common::AccountId32;
use substrate_parser::{
parse_transaction,
AddressableBuffer,
AsMetadata,
additional_types::Era,
cards::{
Call, ExtendedData, FieldData, Info,
PalletSpecificData, ParsedData, VariantData,
},
special_indicators::SpecialtyUnsignedInteger,
};
// A simple signable transaction: Alice sends some cash by `transfer_keep_alive` method
let signable_data = hex::decode("9c0403008eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a480284d717d5031504025a62029723000007000000e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e98a8ee9e389043cd8a9954b254d822d34138b9ae97d3b7f50dc6781b13df8d84").unwrap();
// Hexadecimal metadata, such as one fetched through rpc query
let metadata_westend9111_hex = std::fs::read_to_string("for_tests/westend9111").unwrap();
// SCALE-encoded `V14` metadata, first 5 elements cut off here are `META` prefix and
// `V14` enum index
let metadata_westend9111_vec = hex::decode(&metadata_westend9111_hex.trim()).unwrap()[5..].to_vec();
// `RuntimeMetadataV14` decoded and ready to use.
let metadata_westend9111 = RuntimeMetadataV14::decode(&mut &metadata_westend9111_vec[..]).unwrap();
// Chain genesis hash, typically well-known. Could be fetched through a separate rpc query.
let westend_genesis_hash = H256::from_str("e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e").unwrap();
let parsed = parse_transaction(
&signable_data.as_ref(),
&mut (),
&metadata_westend9111,
Some(westend_genesis_hash),
).unwrap();
let call_data = parsed.call_result.unwrap();
// Pallet name.
assert_eq!(call_data.0.pallet_name, "Balances");
// Call name within the pallet.
assert_eq!(call_data.0.variant_name, "transfer_keep_alive");
// Call contents are the associated `Field` data.
let expected_field_data = vec![
FieldData {
field_name: Some(String::from("dest")),
type_name: Some(String::from("<T::Lookup as StaticLookup>::Source")),
field_docs: String::new(),
data: ExtendedData {
data: ParsedData::Variant(VariantData {
variant_name: String::from("Id"),
variant_docs: String::new(),
fields: vec![
FieldData {
field_name: None,
type_name: Some(String::from("AccountId")),
field_docs: String::new(),
data: ExtendedData {
data: ParsedData::Id(AccountId32(hex::decode("8eaf04151687736326c9fea17e25fc5287613693c912909cb226aa4794f26a48").unwrap().try_into().unwrap())),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"sp_core",
"crypto",
"AccountId32"
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
}
}
]
}),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"sp_runtime",
"multiaddress",
"MultiAddress"
])
.unwrap()
.into_portable(&mut Registry::new()),
}
],
}
},
FieldData {
field_name: Some(String::from("value")),
type_name: Some(String::from("T::Balance")),
field_docs: String::new(),
data: ExtendedData {
data: ParsedData::PrimitiveU128{
value: 100000000,
specialty: SpecialtyUnsignedInteger::Balance,
},
info: Vec::new()
}
}
];
assert_eq!(call_data.0.fields, expected_field_data);
// Parsed extensions. Note that many extensions are empty.
let expected_extensions_data = vec![
ExtendedData {
data: ParsedData::Composite(Vec::new()),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"frame_system",
"extensions",
"check_spec_version",
"CheckSpecVersion",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Composite(Vec::new()),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"frame_system",
"extensions",
"check_tx_version",
"CheckTxVersion",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Composite(Vec::new()),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"frame_system",
"extensions",
"check_genesis",
"CheckGenesis",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Composite(vec![
FieldData {
field_name: None,
type_name: Some(String::from("Era")),
field_docs: String::new(),
data: ExtendedData {
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"sp_runtime",
"generic",
"era",
"Era",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
],
data: ParsedData::Era(Era::Mortal(64, 61)),
}
}
]),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"frame_system",
"extensions",
"check_mortality",
"CheckMortality",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Composite(vec![
FieldData {
field_name: None,
type_name: Some(String::from("T::Index")),
field_docs: String::new(),
data: ExtendedData {
data: ParsedData::PrimitiveU32 {
value: 261,
specialty: SpecialtyUnsignedInteger::Nonce,
},
info: Vec::new()
}
}
]),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"frame_system",
"extensions",
"check_nonce",
"CheckNonce",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Composite(Vec::new()),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"frame_system",
"extensions",
"check_weight",
"CheckWeight",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Composite(vec![
FieldData {
field_name: None,
type_name: Some(String::from("BalanceOf<T>")),
field_docs: String::new(),
data: ExtendedData {
data: ParsedData::PrimitiveU128 {
value: 10000000,
specialty: SpecialtyUnsignedInteger::Tip
},
info: Vec::new()
}
}
]),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"pallet_transaction_payment",
"ChargeTransactionPayment",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::PrimitiveU32 {
value: 9111,
specialty: SpecialtyUnsignedInteger::SpecVersion
},
info: Vec::new()
},
ExtendedData {
data: ParsedData::PrimitiveU32 {
value: 7,
specialty: SpecialtyUnsignedInteger::TxVersion
},
info: Vec::new()
},
ExtendedData {
data: ParsedData::GenesisHash(H256::from_str("e143f23803ac50e8f6f8e62695d1ce9e4e1d68aa36c1cd2cfd15340213f3423e").unwrap()),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"primitive_types",
"H256",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::BlockHash(H256::from_str("98a8ee9e389043cd8a9954b254d822d34138b9ae97d3b7f50dc6781b13df8d84").unwrap()),
info: vec![
Info {
docs: String::new(),
path: Path::from_segments(vec![
"primitive_types",
"H256",
])
.unwrap()
.into_portable(&mut Registry::new()),
}
]
},
ExtendedData {
data: ParsedData::Tuple(Vec::new()),
info: Vec::new()
},
ExtendedData {
data: ParsedData::Tuple(Vec::new()),
info: Vec::new()
},
ExtendedData {
data: ParsedData::Tuple(Vec::new()),
info: Vec::new()
}
];
assert_eq!(parsed.extensions, expected_extensions_data);
Parsed data could be transformed into set of flat and formatted
ExtendedCard
cards using card
method.
Cards could be printed using show
or show_with_docs
methods into
readable Strings.
Re-exports§
pub use decoding_sci::decode_as_call;
pub use decoding_sci::decode_as_call_unmarked;
pub use decoding_sci::ResolvedTy;
pub use storage_data::decode_as_storage_entry;
pub use traits::AsCompleteMetadata;
pub use traits::AsMetadata;
pub use traits::ResolveType;
pub use unchecked_extrinsic::decode_as_unchecked_extrinsic;
Modules§
- additional_
types - Types and functions from
sp_core
andsp_runtime
. - cards
- Types for output: parsed data (nested) and parser cards (flat and formatted).
- compacts
Compact
search and processing.- decoding_
sci - Decode types and calls using metadata with in-built type descriptors.
- error
- Errors.
- printing_
balance - Display balance-representing integers properly using chain-specific decimals and units.
- propagated
- Data that can propagate hierarchically during parsing.
- special_
indicators - Special decoding triggers and indicators.
- storage_
data - Interpret storage data.
- traits
- Traits for generalized metadata.
- unchecked_
extrinsic - Decode unchecked extrinsics, signed or unsigned.
Structs§
- Marked
Data - Marked signable transaction data, with associated start positions for call and extensions data.
- Short
Specs - Chain data necessary to display decoded data correctly.
- Transaction
Carded - Signable transaction parsing outcome represented as formatted flat cards.
- Transaction
Parsed - Signable transaction parsing outcome.
- Transaction
Unmarked Carded - Signable transaction parsing outcome represented as formatted flat cards, for unmarked transaction.
- Transaction
Unmarked Parsed - Signable transaction parsing outcome, for unmarked transaction.
Traits§
- Addressable
Buffer - Bytes access through
ExternalMemory
. - External
Memory - External addressable memory.
Functions§
- decode_
all_ as_ type - Decode whole bytes slice as a known type.
- decode_
as_ type_ at_ position - Decode part of bytes slice starting at a given position as a known type.
- decode_
extensions - Parse extensions part of the signable transaction
MarkedData
using provided metadata. - decode_
extensions_ unmarked - Parse extensions part of the signable transaction using provided metadata.
- parse_
transaction - Parse a signable transaction.
- parse_
transaction_ unmarked - Parse a signable transaction when call is not prefixed by call length.