ipfrs-transport

TensorSwap: High-performance data exchange protocol for IPFRS.

Overview

ipfrs-transport implements the data exchange layer for IPFRS, featuring:

• TensorSwap Protocol: Custom Bitswap variant optimized for tensor streaming
• GraphSync: Dependency-aware block fetching for computation graphs
• QUIC First: Low-latency transport with quinn
• Backpressure Handling: Flow control for heterogeneous network conditions

Key Components

TensorSwap Protocol (Custom Bitswap)

Evolution of IPFS Bitswap with neural-symbolic optimizations:

• Priority Scheduling: Fetch blocks in computation graph order
• Tensor Streaming: Chunked transfer of large model weights
• Gradient Aware: Bidirectional gradient exchange for federated learning
• Debiting System: Fair resource allocation across peers

GraphSync Integration

• Request entire DAG subgraphs in single round-trip
• Selector-based traversal (IPLD selectors)
• Incremental verification during transfer
• Resume support for interrupted transfers

Transport Stack

• QUIC (Primary): 0-RTT connection, multiplexing, built-in encryption
• TCP: Fallback for restricted networks
• WebTransport: Browser compatibility
• WebSocket: Legacy IPFS gateway support

Architecture

TensorSwap Engine
├── Request Manager    # Outgoing block requests
├── Response Handler   # Incoming block provides
├── Want List Manager  # Priority queue of needed blocks
└── Peer Selector      # Choose best peer for each block

         ↓
    QUIC Stream (quinn)
         ↓
   ipfrs-network (libp2p)

Design Principles

• Latency Sensitive: Prioritize time-to-first-block for inference
• Throughput Optimized: Bulk transfer for model distribution
• Graph Aware: Understand dependencies in computation graphs
• Fair: Prevent resource monopolization

Performance Characteristics

Usage Example

use ipfrs_transport::{TensorSwap, Config};
use ipfrs_core::Cid;

// Initialize TensorSwap
let swapper = TensorSwap::new(config).await?;

// Request blocks with priority
swapper.want_block(cid, Priority::High).await?;

// Stream large tensor
let stream = swapper.stream_tensor(cid).await?;
while let Some(chunk) = stream.next().await {
    // Process chunk...
}

Dependencies

• quinn - QUIC implementation
• libp2p - P2P networking primitives
• tokio - Async runtime
• futures - Stream processing

References

• IPFRS v0.3.0 Whitepaper (TensorSwap Protocol)
• Bitswap Specification: https://github.com/ipfs/specs/blob/master/BITSWAP.md
• GraphSync Spec: https://github.com/ipld/specs/blob/master/block-layer/graphsync/graphsync.md