Crate sc_network

Substrate-specific P2P networking.

Important: This crate is unstable and the API and usage may change.

Node identities and addresses

In a decentralized network, each node possesses a network private key and a network public key. In Substrate, the keys are based on the ed25519 curve.

From a node's public key, we can derive its identity. In Substrate and libp2p, a node's identity is represented with the PeerId struct. All network communications between nodes on the network use encryption derived from both sides's keys, which means that identities cannot be faked.

A node's identity uniquely identifies a machine on the network. If you start two or more clients using the same network key, large interferences will happen.

Substrate's network protocol

Substrate's networking protocol is based upon libp2p. It is at the moment not possible and not planned to permit using something else than the libp2p network stack and the rust-libp2p library. However the libp2p framework is very flexible and the rust-libp2p library could be extended to support a wider range of protocols than what is offered by libp2p.

Discovery mechanisms

In order for our node to join a peer-to-peer network, it has to know a list of nodes that are part of said network. This includes nodes identities and their address (how to reach them). Building such a list is called the discovery mechanism. There are three mechanisms that Substrate uses:

• Bootstrap nodes. These are hard-coded node identities and addresses passed alongside with the network configuration.
• mDNS. We perform a UDP broadcast on the local network. Nodes that listen may respond with their identity. More info here. mDNS can be disabled in the network configuration.
• Kademlia random walk. Once connected, we perform random Kademlia FIND_NODE requests in order for nodes to propagate to us their view of the network. More information about Kademlia can be found on Wikipedia.

Connection establishment

When node Alice knows node Bob's identity and address, it can establish a connection with Bob. All connections must always use encryption and multiplexing. While some node addresses (eg. addresses using /quic) already imply which encryption and/or multiplexing to use, for others the multistream-select protocol is used in order to negotiate an encryption layer and/or a multiplexing layer.

The connection establishment mechanism is called the transport.

As of the writing of this documentation, the following base-layer protocols are supported by Substrate:

• TCP/IP for addresses of the form /ip4/1.2.3.4/tcp/5. Once the TCP connection is open, an encryption and a multiplexing layer are negotiated on top.
• WebSockets for addresses of the form /ip4/1.2.3.4/tcp/5/ws. A TCP/IP connection is open and the WebSockets protocol is negotiated on top. Communications then happen inside WebSockets data frames. Encryption and multiplexing are additionally negotiated again inside this channel.
• DNS for addresses of the form /dns4/example.com/tcp/5 or /dns4/example.com/tcp/5/ws. A node's address can contain a domain name.

On top of the base-layer protocol, the Noise protocol is negotiated and applied. The exact handshake protocol is experimental and is subject to change.

The following multiplexing protocols are supported:

• Mplex. Support for mplex will likely be deprecated in the future.
• Yamux.

Substreams

Once a connection has been established and uses multiplexing, substreams can be opened. When a substream is open, the multistream-select protocol is used to negotiate which protocol to use on that given substream. In practice, Substrate opens the following substreams:

• We periodically open an ephemeral substream in order to ping the remote and check whether the connection is still alive. Failure for the remote to reply leads to a disconnection. This uses the libp2p ping protocol.
• We periodically open an ephemeral substream in order to ask information from the remote. This is called the identify protocol.
• We periodically open ephemeral substreams for Kademlia random walk queries. Each Kademlia query is done in a new separate substream. This uses the standard libp2p Kademlia protocol.
• We optionally keep a substream alive for all Substrate-based communications. The name of the protocol negotiated is based on the protocol ID passed as part of the network configuration. This protocol ID should be unique for each chain and prevents nodes from different chains from connecting to each other. More information below.

The Substrate substream

Substrate uses a component named the peerset manager (PSM). Through the discovery mechanism, the PSM is aware of the nodes that are part of the network and decides which nodes we should perform Substrate-based communications with. For these nodes, we open a connection if necessary and open a unique substream for Substrate-based communications. If the PSM decides that we should disconnect a node, then that substream is closed.

For more information about the PSM, see the sc-peerset crate.

Note that at the moment there is no mechanism in place to solve the issues that arise where the two sides of a connection open the unique substream simultaneously. In order to not run into issues, only the dialer of a connection is allowed to open the unique substream. When the substream is closed, the entire connection is closed as well. This is a bug, and should be fixed by improving the protocol.

Within the unique Substrate substream, messages encoded using parity-scale-codec are exchanged. The detail of theses messages is not totally in place, but they can be found in the message.rs file.

Once the substream is open, the first step is an exchange of a status message from both sides, containing information such as the chain root hash, head of chain, and so on.

Communications within this substream include:

• Syncing. Blocks are announced and requested from other nodes.
• Light-client requests. When a light client requires information, a random node we have a substream open with is chosen, and the information is requested from it.
• Gossiping. Used for example by grandpa.

It is intended that in the future each of these components gets more isolated, so that they are free to open and close their own substreams, and so that syncing and light client requests are able to communicate with nodes outside of the range of the PSM.

Usage

Using the sc-network crate is done through the NetworkWorker struct. Create this struct by passing a config::Params, then poll it as if it was a Future. You can extract an Arc<NetworkService> from the NetworkWorker, which can be shared amongst multiple places in order to give orders to the networking.

See the config module for more information about how to configure the networking.

After the NetworkWorker has been created, the important things to do are:

• Calling NetworkWorker::poll in order to advance the network. This can be done by dispatching a background task with the NetworkWorker.
• Calling on_block_import whenever a block is added to the client.
• Calling on_block_finalized whenever a block is finalized.
• Calling trigger_repropagate when a transaction is added to the pool.

More precise usage details are still being worked on and will likely change in the future.

Modules

config	Configuration of the networking layer.
error	Substrate network possible errors.
multiaddr
network_state	Information about the networking, for diagnostic purposes.

Structs

Multiaddr	Representation of a Multiaddr.
NetworkService	Substrate network service. Handles network IO and manages connectivity.
NetworkWorker	Main network worker. Must be polled in order for the network to advance.
PeerId	Identifier of a peer of the network.
PeerInfo	Info about a peer's known state.
ReportHandle	A cloneable handle for reporting cost/benefits of peers.
ReputationChange	Description of a reputation adjustment for a node.

Enums

DhtEvent	Events generated by DHT as a response to get_value and put_value requests.
Event	Type for events generated by networking layer.
SyncState	Reported sync state.

Traits

ExHashT	Minimum Requirements for a Hash within Networking
NetworkStateInfo	Trait for providing information about the local network state