rusttorch 0.4.0

Build, train, and run neural networks in Rust with LibTorch
Documentation

RustTorch

Build, train, and run neural networks in Rust.

crates.io docs.rs license CI

Get started · Train a model · Guides · Roadmap · Contribute

RustTorch brings tensor computation, automatic differentiation, neural-network layers, optimizers, and typed data pipelines to Rust. Write ordinary Rust functions, compose layers, and train on CPU or an available CUDA or MPS device. LibTorch supplies the numerical kernels; RustTorch supplies model construction, validation, data loading, and weight management.

Use it for image classification, time-series models, token embeddings, attention and sequence-to-sequence models, streaming training data, and inference inside a Rust application. The API is evolving in the 0.x series. The roadmap separates available functionality from future work.

Installation

Install the 0.4 release from crates.io. Rust 1.88 or newer is required:

cargo install rusttorch-cli --version 0.4.0
cargo new rusttorch-demo
cd rusttorch-demo
cargo add rusttorch@0.4
rusttorch setup --backend auto

Setup selects a compatible native runtime and checks your project. The first setup can download a large LibTorch archive. Use --backend cpu to select CPU or --backend cuda-12.6 for the managed CUDA distribution on a supported host. See platform setup for prerequisites and runtime library paths.

These examples use the 0.4 API. For an older application, see the 0.2 documentation. To use a development checkout, run cargo add rusttorch --path /absolute/path/to/RustTorch.

Example: train a regressor

Put this in src/main.rs, then run cargo run. Each row contains two numeric features; the model learns one target value per row.

use rusttorch::{DeviceSpec, Result, Tensor, no_grad};
use rusttorch::nn::{Sequential, functional};
use rusttorch::optim::AdamW;

fn main() -> Result<()> {
    let mut model = Sequential::builder()
        .linear(2, 16)
        .relu()
        .linear(16, 1)
        .build(DeviceSpec::Auto)?;
    let inputs = Tensor::from_slice(&[0_f32, 0., 0., 1., 1., 0., 1., 1.])
        .f_reshape([4, 2])?.f_to_device(model.device())?;
    let targets = Tensor::from_slice(&[0_f32, 1., 1., 2.])
        .f_reshape([4, 1])?.f_to_device(model.device())?;
    let mut optimizer = AdamW::builder()
        .learning_rate(0.01)
        .build(model.var_store())?;

    model.train();
    for _ in 0..200 {
        let predictions = model.forward(&inputs)?;
        let loss = functional::mse_loss(&predictions, &targets)?;
        optimizer.backward_step(&loss)?;
    }

    model.eval();
    let predictions = no_grad(|| model.forward(&inputs))?;
    println!("predictions: {predictions:?}");
    model.save_weights("regressor.safetensors")?;
    Ok(())
}

For classification, make the final layer output one logit per class and use functional::cross_entropy with Int64 class indices. For inference, rebuild the same architecture, call load_weights("regressor.safetensors"), switch to eval(), and run the forward pass inside no_grad. Evaluation mode controls layers such as dropout; no_grad controls gradient recording.

The training guide covers custom models, optimizer choice, gradient accumulation and clipping, parameter groups, loss selection, mixed precision, and resumable training. The sequence guide builds a recurrent classifier and masked Transformer models.

Example: batch a dataset

A dataset returns owned samples and its own error type. The borrowed loader is useful for a small dataset or for debugging a preprocessing pipeline.

use std::convert::Infallible;
use rusttorch::data::{DataLoader, Dataset, SequentialSampler};

struct Rows(Vec<f32>);
impl Dataset for Rows {
    type Sample = f32;
    type Error = Infallible;
    fn len(&self) -> usize { self.0.len() }
    fn get(&self, index: usize) -> Result<f32, Infallible> {
        Ok(self.0[index])
    }
}

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let data = Rows(vec![2.0, 3.0, 5.0]);
    let batches = DataLoader::new(&data, SequentialSampler::new(data.len()), 2, false)?
        .collect::<Result<Vec<_>, _>>()?;
    assert_eq!(batches, [vec![2.0, 3.0], vec![5.0]]);
    Ok(())
}

Use DataLoader::builder(dataset) for reusable epochs, shuffling, transforms, worker threads, and prefetching. Use StreamDataLoader for explicitly sharded streams. See the data guide for worker configuration, distributed sampling, memory budgets, and the supported exact checkpoint/resume combinations.

Features

Need RustTorch API
Tensor math and gradients Tensor, Kind, no_grad, and fallible tensor operations
Dense models nn::Linear, Sequential, ReLU, GELU, Dropout, and Flatten
Spatial features and upsampling Conv1d/2d/3d, transposed convolution, max/average/adaptive pooling
Feature normalization LayerNorm, BatchNorm, InstanceNorm, and GroupNorm
Recurrent models RNN, LSTM with optional projections, and GRU with explicit state
Numerical workflows Indexing, views, reductions, linalg, FFT, sparse inference and validated quantization
Sensitivity and probability Functional derivatives, Jacobians, Hessians, surrogate gradients and three distribution families
Attention and sequence models MultiheadAttention, Transformer layers, encoder/decoder stacks
Learned categorical or token features nn::Embedding
Model training Configurable regression/classification losses and seven optimizer families
Training controls Parameter groups, five schedulers, optimizer checkpoints, gradient clipping/scaling
Mixed precision CUDA autocast with strict device selection and unwind-safe scopes
Input pipelines Datasets, samplers, collation, bounded workers, streams, and checkpoints
Weight persistence SafeTensors with strict validation and explicit name mappings
Model inspection Named graph inputs, validation, summaries, and DOT diagrams

download-libtorch is enabled by default and acquires the compatible native runtime. For documentation builds without a native runtime, disable defaults and enable doc-only. doc-only cannot run a model.

[dependencies]
rusttorch = { version = "0.4", default-features = false, features = ["doc-only"] }

Optional features enable data formats without changing existing tensor/model imports:

Feature Use case Facade API
vision Images, annotations, classification and detection batches rusttorch::vision
audio WAV/FLAC, resampling, spectral/Mel/MFCC features rusttorch::audio
text Local tokenizer files, padded masks and token-budget batches rusttorch::text
tabular CSV/JSONL with immutable fitted preprocessing rusttorch::tabular
columnar Add Arrow IPC/Parquet to tabular pipelines rusttorch::tabular
codec Timestamped CPU audio/video from installed FFmpeg 8 rusttorch::codec
full All of the above, including native codec linking All domain APIs

For example, cargo add rusttorch@0.4 --features vision,text enables image and text pipelines. Each package also works directly with rusttorch-data and rusttorch-core. Follow the domain guide for complete examples, supported formats, finite input limits and native linking choices.

Native runtime

RustTorch currently uses tch 0.26.0 and LibTorch 2.13.0. LibTorch is the C++ numerical library also used by PyTorch; your Rust application does not need a Python training loop. You can use the managed download, an installed LibTorch selected with LIBTORCH, or Python torch 2.13.0 selected with LIBTORCH_USE_PYTORCH=1. The platform loader must find the selected shared libraries when your executable runs.

DeviceSpec::Auto selects CUDA, then MPS, then CPU according to runtime availability. An explicit unavailable device returns an error. Consult the device guide, CUDA guide, and MPS guide for backend requirements.

Documentation

Guide Start here when you want to…
Rust API reference Find types, methods, defaults, and Rust examples
Tensor workflows Index, reshape, normalize, solve systems, filter signals and use sparse data
Differentiation and probability Compute sensitivities, higher derivatives, surrogate gradients and sample latent variables
Models and training Build a classifier, choose losses, schedule updates and resume training
Sequence models Train recurrent and Transformer models with state and masks
Checkpoint example Restore model, optimizer, scheduler and scaler together
Distributed training Synchronize CPU processes and train with rank-local parameter/optimizer shards
Deployment Import/export PT2 and ONNX or compile and execute TorchScript artifacts
Framework tools Check gradients, benchmark, export traces and atomically save a complete training boundary
Domain data Load image annotations, audio features, padded tokens, timestamped media and fitted tabular records
Data pipelines Load, transform, batch, and resume training data
Graph guide Inspect named inputs, branches, and execution order
Model interoperability Exchange weights with another model implementation
Platform setup Configure the native runtime for your machine
Architecture Understand ownership and execution boundaries
Roadmap See implementation priorities and remaining capabilities
API census maintenance Refresh the pinned inventory and validate scoped compatibility claims
Compatibility evidence Check exact tested scopes and source references

PyTorch is the behavioral reference for compatibility tests and weight exchange. It appears in provenance and interoperability guides for that reason. The long-term goal is to provide its framework functionality through RustTorch APIs; current coverage is narrower. Python runtime features need explicit Rust equivalents, and planned functionality is not a support claim.

Contributing

Read CONTRIBUTING.md before proposing an API change. It covers issue-first discussion, focused tests, examples, compatibility evidence, source attribution, and DCO sign-off. The maintainer guide describes review and release checks.

Code of conduct · Governance · Support · Security

License

RustTorch is available under the MIT License or Apache License 2.0, at your option. Dependencies and upstream behavioral references retain their own terms; see third-party notices. RustTorch is independent of the PyTorch Foundation.