Crate p2panda_net 

p2panda-net is a data-type-agnostic p2p networking layer offering robust, direct communication to any device, no matter where they are.

It provides a stream-based API for higher layers: Applications subscribe to any “topic” they are interested in and p2panda-net will automatically discover similar peers and transport raw bytes between them.

Additionally p2panda-net can be extended with custom sync protocols for all data types, allowing applications to “catch up on past data”, eventually converging to the same state.

Features

Most of the lower-level networking of p2panda-net is made possible by the work of iroh utilising well-established and known standards, like QUIC for transport, (self-certified) TLS for transport encryption, STUN for establishing direct connections between devices, Tailscale’s DERP (Designated Encrypted Relay for Packets) for relay fallbacks, PlumTree and HyParView for broadcast-based gossip overlays.

p2panda adds crucial functionality on top of iroh for peer-to-peer application development, without tying developers too closely to any pre-defined data types and allowing plenty of space for customisation:

1. Data of any kind can be exchanged efficiently via gossip broadcast (“live mode”) or via sync protocols between two peers (“catching up on past state”)
2. Custom network-wide queries to express interest in certain data of applications
3. Ambient peer discovery: Learning about new, previously unknown peers in the network
4. Ambient topic discovery: Learning what peers are interested in, automatically forming overlay networks per topic
5. Sync protocol API, providing an eventual-consistency guarantee that peers will converge on the same state over time
6. Manages connections, automatically syncs with discovered peers and re-tries on faults
7. Extension for networks to handle efficient sync of large files

Offline-First

This networking crate is designed to run on top of bi-directional, ordered connections on the IP layer (aka “The Internet”), with robustness to work in environments with unstable connectivity or offline time-periods.

While this IP-based networking implementation should provide for many “modern” use-cases, p2panda data-types are designed for more extreme scenarios where connectivity can never be assumed and data transmission needs to be highly “delay tolerant”: For example “broadcast-only” topologies on top of BLE (Bluetooth Low Energy), LoRa or even Digital Radio Communication infrastructure.

Extensions

p2panda-net is agnostic to any data type (sending and receiving raw byte streams) and can seamlessly be extended with external or official p2panda implementations for different parts of the application:

1. Custom Data types exchanged over the network
2. Optional relay nodes to aid connection establishment when peers are behind firewalls etc.
3. Custom sync protocol for any data types, with managed re-attempts on connection failures and optional re-sync schedules
4. Custom peer discovery strategies (multiple approaches can be used at the same time)
5. Sync and storage of (very) large blobs
6. Fine-tune gossipping behaviour
7. Additional custom protocol handlers

Integration with other p2panda solutions

We provide p2panda’s fork-tolerant and prunable append-only logs in p2panda-core, offering single-writer and multi-writer streams, authentication, deletion, ordering and more. This can be further extended with an efficient sync implementation in p2panda-sync and validation and fast stream-based ingest solutions in p2panda-streams.

For discovery of peers on the local network, we provide an mDNS-based implementation in p2panda-discovery, planned next are additional techniques like “rendesvouz” nodes and random walk algorithms.

Lastly we maintain persistance layer APIs in p2panda-store for in-memory storage or embeddable, SQL-based databases.

In the future we will provide additional implementations for managing access control and group encryption.

Example

use p2panda_core::{PrivateKey, Hash};
use p2panda_discovery::mdns::LocalDiscovery;
use p2panda_net::{NetworkBuilder, TopicId};
use p2panda_sync::TopicQuery;
use serde::{Serialize, Deserialize};

// Peers using the same "network id" will eventually find each other. This is the most global
// identifier to group peers into multiple networks when necessary.
let network_id = [1; 32];

// The network can be used to automatically find and ask other peers about any data the
// application is interested in. This is expressed through "network-wide queries" over topics.
//
// In this example we would like to be able to query messages from each chat group, identified
// by a BLAKE3 hash.
#[derive(Clone, Debug, PartialEq, Eq, Hash, Deserialize, Serialize)]
struct ChatGroup(Hash);

impl ChatGroup {
    pub fn new(name: &str) -> Self {
        Self(Hash::new(name.as_bytes()))
    }
}

impl TopicQuery for ChatGroup {}

impl TopicId for ChatGroup {
    fn id(&self) -> [u8; 32] {
        self.0.into()
    }
}

// Generate an Ed25519 private key which will be used to authenticate your peer towards others.
let private_key = PrivateKey::new();

// Use mDNS to discover other peers on the local network.
let mdns_discovery = LocalDiscovery::new();

// Establish the p2p network which will automatically connect you to any discovered peers.
let network = NetworkBuilder::new(network_id)
    .private_key(private_key)
    .discovery(mdns_discovery)
    .build()
    .await?;

// Subscribe to network events.
let mut event_rx = network.events().await?;

// From now on we can send and receive bytes to any peer interested in the same chat.
let my_friends_group = ChatGroup::new("me-and-my-friends");
let (tx, mut rx, ready) = network.subscribe(my_friends_group).await?;

Re-exports

pub use config::Config;

pub use network::FromNetwork;

pub use network::Network;

pub use network::NetworkBuilder;

pub use network::RelayMode;

pub use network::ToNetwork;

Modules

config

Configuration for network nodes and gossip.

network

Node implementation for p2p networking and data streaming, extensible with discovery strategies, sync protocols, blob sync and more.

Structs

LogSyncProtocol

Efficient sync protocol for append-only log data types.

NodeAddress

Node address including public key, socket address(es) and an optional relay URL.

RelayUrl

URL identifying a relay server.

ResyncConfiguration

Configuration parameters for resync behaviour.

SyncConfiguration

Configuration parameters for data synchronisation between peers.

Enums

SystemEvent

Network system events.

Traits

ProtocolHandler

Interface to accept incoming connections for custom protocol implementations.

TopicId

Topic ids are announced on the network and used to identify peers with overlapping interests.

Type Aliases

NetworkId

Unique 32 byte identifier for a network.