1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
//! # runsync-transfer
//!
//! A peer-to-peer file transfer engine: the sender compresses and seals chunks,
//! the receiver opens and decompresses them, and both ends run the work across
//! all available cores while the network stays saturated.
//!
//! ## What it does
//!
//! - **Parallel everywhere.** Chunks are compressed, encrypted, and written
//! concurrently across N streams and N CPU workers. Nothing is serialised on
//! a file cursor or a reassembly buffer.
//! - **Adaptive compression.** Per-chunk entropy probing plus an extension
//! table, so `.flac` and `.mp4` are shipped raw while `.wav` and text get
//! compressed. Incompressible data is never inflated.
//! - **End-to-end encryption.** X25519 + HKDF + AES-256-GCM or
//! ChaCha20-Poly1305, layered inside the transport's own TLS so a relay
//! cannot read payloads.
//! - **Large files.** 64-bit offsets throughout, positional I/O, constant
//! memory. A 100 GB file costs the same resident bytes as a 100 MB one.
//! - **Many files at once.** One manifest, one connection, chunks from
//! different files interleaved across streams.
//! - **Resume.** A crashed or cancelled transfer restarts from the chunks
//! already on disk.
//! - **Verification.** BLAKE3 over every chunk, folded into a per-file root.
//!
//! ## Transport
//!
//! The engine talks to a [`Transport`], not to a socket. The bundled QUIC
//! adapter wraps a `quinn::Connection`, so an application that already has a
//! QUIC endpoint hands over its live connection rather than dialling a second
//! one:
//!
//! ```no_run
//! # #[cfg(feature = "quic")]
//! # async fn f(conn: quinn::Connection) -> Result<(), runsync_transfer::Error> {
//! use runsync_transfer::{send, Config, Source, QuicTransport};
//! use std::sync::Arc;
//!
//! let transport = Arc::new(QuicTransport::from_connection(conn));
//! let stats = send(transport, &[Source::new("/data/album")], &Config::default(), None).await?;
//! println!("{stats}");
//! # Ok(()) }
//! ```
//!
//! The receiving side:
//!
//! ```no_run
//! # #[cfg(feature = "quic")]
//! # async fn f(conn: quinn::Connection) -> Result<(), runsync_transfer::Error> {
//! use runsync_transfer::{receive, Config, QuicTransport};
//! use std::sync::Arc;
//!
//! let transport = Arc::new(QuicTransport::from_connection(conn));
//! let stats = receive(transport, "/dest", &Config::default(), None).await?;
//! # Ok(()) }
//! ```
//!
//! ## Encryption
//!
//! [`Secrecy::TransportOnly`] is the default and relies on QUIC/TLS. For
//! payload confidentiality that survives a relay, give both ends the same
//! pre-shared key or pin each other's static X25519 identities:
//!
//! ```
//! use runsync_transfer::{Config, Secrecy, crypto};
//!
//! let psk = crypto::random_key(); // share this out of band
//! let cfg = Config::default().with_secrecy(Secrecy::Psk(psk));
//! ```
//!
//! ## Tuning
//!
//! [`Config::default`] is a reasonable middle. [`Config::throughput`] favours a
//! fast link (4 MiB chunks, LZ4); [`Config::bandwidth_saving`] favours a slow
//! one (zstd level 9). The knob that matters most on a long fat path is
//! `streams`, and the one that bounds memory is
//! `streams × queue_depth × chunk_size` — see [`Config::memory_budget`].
pub use Algorithm;
pub use crypto;
pub use ;
pub use ;
pub use Source;
pub use ;
pub use receive;
pub use ChunkBitmap;
pub use send;
pub use ;
pub use ;
pub use ;