Crate remote

Expand description

Remote copy protocol and networking for distributed file operations

This crate provides the networking layer and protocol definitions for remote file copying in the RCP tools suite. It enables efficient distributed copying between remote hosts using SSH for orchestration and QUIC for high-performance data transfer.

§Overview

The remote copy system uses a three-node architecture:

Master (rcp)
├── SSH → Source Host (rcpd)
│   └── QUIC → Master (control)
│   └── QUIC Server (waits for Destination)
└── SSH → Destination Host (rcpd)
    └── QUIC → Master (control)
    └── QUIC Client → Source (data transfer)

§Connection Flow

Initialization: Master starts rcpd processes on source and destination via SSH
Control Connections: Both rcpd processes connect back to Master via QUIC
Address Exchange: Source starts QUIC server and sends its address to Master
Direct Connection: Master forwards address to Destination, which connects to Source
Data Transfer: Files flow directly from Source to Destination (not through Master)

This design ensures efficient data transfer while allowing the Master to coordinate operations and monitor progress.

§Key Components

§SSH Session Management

The SshSession type represents an SSH connection to a remote host and is used to:

Launch rcpd daemons on remote hosts
Configure connection parameters (user, host, port)

§QUIC Networking

QUIC protocol provides:

Multiplexed streams over a single connection
Built-in encryption and authentication
Efficient data transfer with congestion control

Key functions:

get_server_with_port_ranges - Create QUIC server endpoint with optional port restrictions
get_client_with_port_ranges_and_pinning - Create secure QUIC client with certificate pinning
get_endpoint_addr - Get the local address of an endpoint

§Port Range Configuration

The port_ranges module allows restricting QUIC to specific port ranges, useful for firewall-restricted environments:

// bind to ports in the 8000-8999 range with default timeouts
// idle_timeout: 10 seconds, keep_alive: 1 second
let (endpoint, cert_fingerprint) = get_server_with_port_ranges(
    Some("8000-8999"),
    10,  // idle_timeout_sec
    1,   // keep_alive_interval_sec
)?;

§Protocol Messages

The protocol module defines the message types exchanged between nodes:

MasterHello - Master → rcpd configuration
SourceMasterHello - Source → Master address information
RcpdResult - rcpd → Master operation results
TracingHello - rcpd → Master tracing initialization

§Stream Communication

The streams module provides high-level abstractions over QUIC streams:

Bidirectional streams for request/response communication
Unidirectional streams for tracing and logging
Object serialization/deserialization using bincode

§Remote Tracing

The tracelog module enables distributed logging and progress tracking:

Forward tracing events from remote rcpd processes to Master
Aggregate progress information across multiple remote operations
Display unified progress for distributed operations

§Security Model

The remote copy system implements a defense-in-depth security model using SSH for authentication and certificate pinning for QUIC connection integrity. This provides protection against man-in-the-middle (MITM) attacks while maintaining ease of deployment.

§Authentication & Authorization

SSH is the security perimeter: All remote operations begin with SSH authentication.

Initial access control is handled entirely by SSH
Users must be authenticated and authorized via SSH before any QUIC connections are established
SSH configuration (keys, permissions, etc.) determines who can initiate remote copies

§Transport Encryption & Integrity

QUIC with TLS 1.3: All data transfer uses QUIC protocol built on TLS 1.3

Provides encryption for data confidentiality
Ensures data integrity through cryptographic authentication
Built-in protection against replay attacks

§Trust Bootstrap via Certificate Pinning

Two secured QUIC connections in every remote copy operation:

§1. Master ← rcpd (Control Connection)

Master (rcp)                    Remote Host (rcpd)
   |                                   |
   | 1. SSH connection established     |
   |<--------------------------------->|
   | 2. Master generates self-signed   |
   |    cert, computes SHA-256         |
   |    fingerprint                    |
   |                                   |
   | 3. Launch rcpd via SSH with       |
   |    fingerprint as argument        |
   |---------------------------------->|
   |                                   |
   | 4. rcpd validates Master's cert   |
   |    against received fingerprint   |
   |<---(QUIC + cert pinning)----------|

Master generates ephemeral self-signed certificate at startup
Certificate fingerprint (SHA-256) is passed to rcpd via SSH command-line arguments
rcpd validates Master’s certificate by computing its fingerprint and comparing
Connection fails if fingerprints don’t match (MITM protection)

§2. Source → Destination (Data Transfer Connection)

Source (rcpd)                   Destination (rcpd)
   |                                   |
   | 1. Source generates self-signed   |
   |    cert, computes SHA-256         |
   |    fingerprint                    |
   |                                   |
   | 2. Send fingerprint + address     |
   |    to Master via secure channel   |
   |---------------------------------->|
   |                    Master         |
   |                      |            |
   | 3. Master forwards   |            |
   |    to Destination    |            |
   |                      |----------->|
   |                                   |
   | 4. Destination validates Source's |
   |    cert against received          |
   |    fingerprint                    |
   |<---(QUIC + cert pinning)----------|

Source generates ephemeral self-signed certificate
Fingerprint is sent to Master over already-secured Master←Source connection
Master forwards fingerprint to Destination over already-secured Master←Destination connection
Destination validates Source’s certificate against fingerprint
Direct Source→Destination connection established only after successful validation

§SSH as Secure Out-of-Band Channel

Key insight: SSH provides a secure, authenticated channel for bootstrapping QUIC trust

Certificate fingerprints are transmitted through SSH (Master→rcpd command-line arguments)
SSH connection is already authenticated and encrypted
This creates a “chain of trust”:
1. User trusts SSH (proven by successful authentication)
2. SSH carries the certificate fingerprint securely
3. QUIC connection validates against that fingerprint
4. Therefore, QUIC connection is trustworthy

§Attack Resistance

§✅ Protected Against

Man-in-the-Middle (MITM): Certificate pinning prevents attackers from impersonating endpoints
Replay Attacks: TLS 1.3 in QUIC provides built-in replay protection
Eavesdropping: All data encrypted with TLS 1.3
Tampering: Cryptographic integrity checks prevent data modification
Unauthorized Access: SSH authentication is required before any operations

§⚠️ Threat Model Assumptions

SSH is secure: The security model depends on SSH being properly configured and uncompromised
Certificate fingerprints are short-lived: Ephemeral certificates are generated per-session
Trusted network for Master: The machine running Master (rcp) should be trusted

§Best Practices

Secure SSH Configuration: Use key-based authentication, disable password auth
Keep Systems Updated: Ensure SSH, TLS libraries, and QUIC implementations are current
Network Segmentation: Run remote copies on trusted network segments when possible
Monitor Logs: Certificate validation failures indicate potential security issues

§Network Troubleshooting

Common failure scenarios and their handling:

§SSH Connection Fails

Cause: Host unreachable, authentication failure
Timeout: ~30s (SSH default)
Error: Standard SSH error messages

§rcpd Cannot Connect to Master

Cause: Firewall blocks QUIC, network routing issue
Timeout: Configurable via --remote-copy-conn-timeout-sec (default: 15s)
Solution: Check firewall rules for QUIC ports

§Destination Cannot Connect to Source

Cause: Firewall blocks direct connection between hosts
Timeout: Configurable (default: 15s)
Solution: Use --quic-port-ranges to specify allowed ports, configure firewall

For detailed troubleshooting, see the repository’s docs/network_connectivity.md.

§Examples

§Starting a Remote Copy Daemon

use remote::{SshSession, protocol::RcpdConfig, start_rcpd};
use std::net::SocketAddr;

let session = SshSession {
    user: Some("user".to_string()),
    host: "example.com".to_string(),
    port: None,
};

let config = RcpdConfig {
    verbose: 0,
    fail_early: false,
    max_workers: 4,
    max_blocking_threads: 512,
    max_open_files: None,
    ops_throttle: 0,
    iops_throttle: 0,
    chunk_size: 1024 * 1024,
    dereference: false,
    overwrite: false,
    overwrite_compare: String::new(),
    debug_log_prefix: None,
    quic_port_ranges: None,
    quic_idle_timeout_sec: 10,
    quic_keep_alive_interval_sec: 1,
    progress: false,
    progress_delay: None,
    remote_copy_conn_timeout_sec: 15,
    master_cert_fingerprint: Vec::new(),
};
let master_addr: SocketAddr = "192.168.1.100:5000".parse()?;
let server_name = "master-server";

let process = start_rcpd(&config, &session, &master_addr, server_name).await?;

§Creating a QUIC Server with Port Ranges

use remote::{get_server_with_port_ranges, get_endpoint_addr};

// create server restricted to ports 8000-8999
// timeouts: 10s idle, 1s keep-alive (CLI defaults)
let (endpoint, _cert_fingerprint) = get_server_with_port_ranges(
    Some("8000-8999"),
    10,  // idle_timeout_sec
    1,   // keep_alive_interval_sec
)?;
let addr = get_endpoint_addr(&endpoint)?;
println!("Server listening on: {}", addr);

§Module Organization

port_ranges - Port range parsing and UDP socket binding
protocol - Protocol message definitions and serialization
streams - QUIC stream wrappers with typed message passing
tracelog - Remote tracing and progress aggregation

Modules§

port_ranges
protocol
streams
tracelog

Structs§

QuicConfig: Configuration for QUIC connections
SshSession

Functions§

get_client_with_port_ranges_and_pinning
get_endpoint_addr
get_random_server_name
get_server_with_port_ranges
start_rcpd
wait_for_rcpd_process

Crate remote

Crate remote Copy item path

§Overview

§Connection Flow

§Key Components

§SSH Session Management

§QUIC Networking

§Port Range Configuration

§Protocol Messages

§Stream Communication

§Remote Tracing

§Security Model

§Authentication & Authorization

§Transport Encryption & Integrity

§Trust Bootstrap via Certificate Pinning

§1. Master ← rcpd (Control Connection)

§2. Source → Destination (Data Transfer Connection)

§SSH as Secure Out-of-Band Channel

§Attack Resistance

§✅ Protected Against

§⚠️ Threat Model Assumptions

§Best Practices

§Network Troubleshooting

§SSH Connection Fails

§rcpd Cannot Connect to Master

§Destination Cannot Connect to Source

§Examples

§Starting a Remote Copy Daemon

§Creating a QUIC Server with Port Ranges

§Module Organization

Modules§

Structs§

Functions§

Crate remote