Crate runsync_transfer 

runsync-transfer

A peer-to-peer file transfer engine: the sender compresses and seals chunks, the receiver opens and decompresses them, and both ends run the work across all available cores while the network stays saturated.

What it does

• Parallel everywhere. Chunks are compressed, encrypted, and written concurrently across N streams and N CPU workers. Nothing is serialised on a file cursor or a reassembly buffer.
• Adaptive compression. Per-chunk entropy probing plus an extension table, so .flac and .mp4 are shipped raw while .wav and text get compressed. Incompressible data is never inflated.
• End-to-end encryption. X25519 + HKDF + AES-256-GCM or ChaCha20-Poly1305, layered inside the transport’s own TLS so a relay cannot read payloads.
• Large files. 64-bit offsets throughout, positional I/O, constant memory. A 100 GB file costs the same resident bytes as a 100 MB one.
• Many files at once. One manifest, one connection, chunks from different files interleaved across streams.
• Resume. A crashed or cancelled transfer restarts from the chunks already on disk.
• Verification. BLAKE3 over every chunk, folded into a per-file root.

Transport

The engine talks to a Transport, not to a socket. The bundled QUIC adapter wraps a quinn::Connection, so an application that already has a QUIC endpoint hands over its live connection rather than dialling a second one:

use runsync_transfer::{send, Config, Source, QuicTransport};
use std::sync::Arc;

let transport = Arc::new(QuicTransport::from_connection(conn));
let stats = send(transport, &[Source::new("/data/album")], &Config::default(), None).await?;
println!("{stats}");

The receiving side:

use runsync_transfer::{receive, Config, QuicTransport};
use std::sync::Arc;

let transport = Arc::new(QuicTransport::from_connection(conn));
let stats = receive(transport, "/dest", &Config::default(), None).await?;

Encryption

Secrecy::TransportOnly is the default and relies on QUIC/TLS. For payload confidentiality that survives a relay, give both ends the same pre-shared key or pin each other’s static X25519 identities:

use runsync_transfer::{Config, Secrecy, crypto};

let psk = crypto::random_key();          // share this out of band
let cfg = Config::default().with_secrecy(Secrecy::Psk(psk));

Tuning

Config::default is a reasonable middle. Config::throughput favours a fast link (4 MiB chunks, LZ4); Config::bandwidth_saving favours a slow one (zstd level 9). The knob that matters most on a long fat path is streams, and the one that bounds memory is streams × queue_depth × chunk_size — see Config::memory_budget.

Re-exports

pub use codec::compress::Algorithm;

pub use codec::crypto;

pub use config::Cipher;

pub use config::CompressionConfig;

pub use config::CompressionMode;

pub use config::Config;

pub use config::Secrecy;

pub use error::Error;

pub use error::Result;

pub use manifest::Source;

pub use metrics::human_bytes;

pub use metrics::Metrics;

pub use metrics::Progress;

pub use metrics::ProgressFn;

pub use recv::receive;

pub use resume::ChunkBitmap;

pub use send::send;

pub use transport::mem::MemTransport;

pub use transport::Transport;

pub use wire::EntryKind;

pub use wire::FileEntry;

pub use transport::quic::bulk_transport_config;

pub use transport::quic::QuicTransport;

Modules

codec

Chunk-level codecs: compression, then authenticated encryption.

config

error

index

A cache of per-chunk hashes, so a file that has not changed is never read.

io

Positional file I/O.

manifest

Building the file list on the sender, and validating it on the receiver.

metrics

Progress and throughput accounting.

pool

Chunk buffer recycling.

recv

Receiver pipeline.

resume

Resume support.

send

Sender pipeline.

transport

Transport abstraction.

wire

Wire format.