rusttorch 0.2.0

Eager PyTorch-style neural networks and graph execution for Rust, backed by LibTorch
Documentation

RustTorch

easy to use and easy to implement, but crazy fast

An unofficial, eager-first Rust frontend over LibTorch.

crates.io docs.rs license MSRV CI

Installation · Capabilities · Runtime setup · Documentation · Contributing · License

[!IMPORTANT] RustTorch is early-stage software with a deliberately small API. It does not provide full PyTorch parity, and the 0.x series does not promise API or graph format stability. Use the canonical compatibility ledger and its generated coverage page to evaluate implemented scope.

RustTorch offers fallible Rust APIs for common eager model code while LibTorch owns tensor storage, kernels, automatic differentiation, and backend execution. The project credo guides API and implementation choices; it is not an unqualified performance claim.

Installation and Example

Install the 0.2 release from crates.io:

cargo install rusttorch-cli --version 0.2.0
cargo new rusttorch-demo
cd rusttorch-demo
cargo add rusttorch@0.2
rusttorch setup --backend auto

Replace src/main.rs with this eager example:

use rusttorch::nn::Sequential;
use rusttorch::{DeviceSpec, Kind, Result, Tensor};

fn main() -> Result<()> {
    let model = Sequential::builder()
        .linear(2, 4)
        .relu()
        .linear(4, 1)
        .build(DeviceSpec::Auto)?;
    let input = Tensor::f_zeros([8, 2], (Kind::Float, model.device()))?;
    let output = model.forward(&input)?;
    assert_eq!(output.size(), [8, 1]);
    Ok(())
}

Then run it:

cargo run

The rusttorch facade is the seamless default for data loading:

use std::convert::Infallible;

use rusttorch::data::{DataLoader, Dataset, SequentialSampler};

struct Rows([i64; 3]);

impl Dataset for Rows {
    type Sample = i64;
    type Error = Infallible;

    fn len(&self) -> usize {
        self.0.len()
    }

    fn get(&self, index: usize) -> Result<Self::Sample, Self::Error> {
        Ok(self.0[index])
    }
}

fn main() {
    let rows = Rows([2, 3, 5]);
    let batches = DataLoader::new(&rows, SequentialSampler::new(rows.len()), 2, false)
        .expect("batch size is nonzero")
        .collect::<Result<Vec<_>, _>>()
        .expect("rows are infallible");

    assert_eq!(batches, vec![vec![2, 3], vec![5]]);
}

Use the separate rusttorch-data package when an application wants the data layer without the facade; it provides the same loader surface directly.

Owned, ordered, zero-worker map loaders also support versioned exact resume at consumer-visible batch boundaries. Opt in with an explicit replay-safe or transactional dataset wrapper and .dataset_identity(...), persist the typed serde LoaderState in the format your application chooses, then pass it to .resume_from(...). Built-in samplers (including distributed and weighted sampling), automatic or explicit batching, no-batch conversion, component state, deterministic seed metadata, and effective pinning status are validated before any restored state is applied. Positive-worker, prefetched, and stream resume are not yet supported; see the rusttorch-data guide for the complete contract and example.

Owned map and explicitly sharded stream loaders support strict post-transform queue budgets through .prefetch_bytes(...) and recursive post-collation batch pinning through .pin_memory() or .pin_memory_for(Device). Automatic pinning uses CUDA device zero when CUDA is available and otherwise reports a typed no-accelerator no-op. The byte estimate is a conservative logical payload bound for prefetched values, not a claim about whole-process RSS; the coordinator's active item-bounded collation batch is outside it.

The first setup may download a large official LibTorch artifact into Cargo build storage. RustTorch links LibTorch dynamically, so the platform loader must also be able to find its shared libraries at runtime.

Features

The default download-libtorch feature lets tch acquire its compatible LibTorch 2.13.0 runtime. For checks and rustdoc without a runtime, use doc-only with default features disabled:

[dependencies]
rusttorch = { version = "0.2", default-features = false, features = ["doc-only"] }

Native runtime

Executables need LibTorch/PyTorch 2.13.0, matching tch 0.26.0. The default feature uses the downloaded runtime; alternatively, disable default features and build with LIBTORCH_USE_PYTORCH=1 against Python torch 2.13.0 or set LIBTORCH=/absolute/path/to/libtorch. The platform dynamic loader must find the selected shared libraries at runtime.

Current capabilities

Area Implemented scope
Eager models Linear, Identity, ReLU, GELU, Dropout, Flatten, and Sequential
Training LibTorch autograd, MSE and cross-entropy losses, Adam, and SGD
Devices Explicit CPU, CUDA, and MPS requests plus checked automatic selection
State interchange Strict, non-strict, mapped, and dry-run SafeTensors loading
Graphs Optional named-input graph API with branching, validation, summaries, and DOT output
Data loading Fallible map datasets and explicitly sharded streams, bounded workers, seeded transforms, batching, and custom collation
Runtime Project-local managed CPU or CUDA 12.6 setup over official LibTorch artifacts

The machine-readable compatibility ledger is the canonical inventory; API coverage is its generated human-readable view. Every entry has an exact scope and one of supported, partial, planned, python_only, or not_supported. supported applies only to the written scope. Rows cover differently sized capabilities, so their count is not converted into a support percentage.

Backend availability depends on the linked LibTorch build and host; explicit unavailable device requests return errors rather than silently falling back.

Runtime setup

Run one of the three supported commands inside a Cargo project:

rusttorch setup --backend auto
rusttorch setup --backend cpu
rusttorch setup --backend cuda-12.6
  • auto preserves an active Python, system LibTorch, or CUDA selector. Otherwise it chooses CUDA 12.6 on a compatible Linux or Windows NVIDIA host and CPU on the remaining supported hosts.
  • cpu selects the managed CPU distribution. On supported macOS systems that distribution can expose MPS.
  • cuda-12.6 selects cu126 on Linux or Windows after checking the NVIDIA driver. It never installs or changes drivers or CUDA toolkits.

Setup locates the Cargo workspace root, keeps managed CPU and CUDA artifacts separate, writes project-local Cargo settings, and runs cargo check. Existing Python, system, and offline LibTorch workflows remain available. See platform setup and CUDA support for selectors, driver floors, target isolation, retry behavior, and dynamic-loader requirements.

Documentation

Guide What it covers
API documentation Public Rust types and functions
Compatibility ledger Canonical machine-readable scopes and evidence
Compatibility coverage Generated status view of PyTorch API areas
rusttorch-core compatibility Generated core-package compatibility scope
rusttorch-data compatibility Generated data-package compatibility scope
Architecture Eager frontend and LibTorch boundary
rusttorch-data loader guide Borrowed, threaded, sharded-stream, resource, and checkpoint modes
Platform support Runtime, devices, and system/Python setup
Backend evidence Hardware-specific validation and parity scope
Graph system Optional graph construction, validation, and execution
Model interoperability SafeTensors exchange with PyTorch
Porting policy Source attribution and compatibility rules
Release provenance Exact package subjects and verification procedure
Governance Roles, decisions, security, and release authority
Support Usage questions and issue routing
Security Supported versions and private vulnerability reporting

Limitations

RustTorch currently has no convolutional, recurrent, or transformer module surface; distributed training; quantization; replacement autograd; torch.compile; or custom-kernel framework. SafeTensors is the supported model-state format. Python pickle models, TorchScript, torch.export, and cross-language optimizer checkpoint resume are not exposed by the current API. Map datasets and explicitly sharded stream factories support bounded Rust worker threads, cooperative per-batch timeouts and cancellation, loader-owned persistent worker pools, and ordered or completion-order delivery. Stream records merge globally before batching, so drop_last removes at most one global tail. Rust cannot force-cancel a blocking foreign or native call, so drop waits for non-cooperative work to return. Exact loader checkpoint/resume supports owned ordered map loaders and explicitly checkpointable sharded streams. Exact worker replay requires nonpersistent workers, no custom worker initializer, no timeout, and no byte budget. Streams retain paired source and transform states and validate all shards before applying a resume. Distributed sampling is available without distributed training orchestration.

Contributing

Contributions are welcome. Start with the contribution guide, keep claims within tested scope, and add the narrowest evidence that proves a change. Report vulnerabilities through the project's security policy, never a public issue.

License

Original RustTorch code is available under either the MIT License or the Apache License 2.0, at your option. PyTorch, LibTorch, tch, and other dependencies retain their own licenses and attribution; see third-party notices.

RustTorch is not affiliated with or endorsed by the PyTorch Foundation.