Skip to main content

Variable

Struct Variable 

Source
pub struct Variable { /* private fields */ }
Expand description

A differentiable variable wrapping a Tensor.

Variables use libtorch’s native autograd. When a tensor has requires_grad=true, all standard operations build a C++ computation graph automatically. Calling backward() runs libtorch’s backward engine — no Rust-side graph walking.

Internally uses Rc<RefCell<>> for shared ownership — the same parameter can be referenced by both a Module and an Optimizer.

ⓘ
let w = Variable::new(Tensor::randn(&[3, 2], opts)?, true);
let x = Variable::new(Tensor::from_f32(&[1.0, 2.0, 3.0], &[1, 3], Device::CPU)?, false);
let loss = x.matmul(&w)?.sum()?;
loss.backward()?;
println!("{:?}", w.grad()); // gradient of loss w.r.t. w

Implementations§

Source§

impl Variable

Source

pub fn add(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise addition.

Source

pub fn sub(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise subtraction.

Source

pub fn mul(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise multiplication (Hadamard product).

Source

pub fn div(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise division.

Source

pub fn matmul(&self, other: &Variable) -> Result<Variable, TensorError>

Matrix multiplication (torch.matmul). Supports batched matmul.

Source

pub fn mul_scalar(&self, scalar: f64) -> Result<Variable, TensorError>

Multiply all elements by a scalar.

Source

pub fn div_scalar(&self, scalar: f64) -> Result<Variable, TensorError>

Divide all elements by a scalar.

Source

pub fn add_scalar(&self, scalar: f64) -> Result<Variable, TensorError>

Add a scalar to all elements.

Source

pub fn neg(&self) -> Result<Variable, TensorError>

Element-wise negation.

Source

pub fn relu(&self) -> Result<Variable, TensorError>

ReLU activation: max(0, x).

Source

pub fn sigmoid(&self) -> Result<Variable, TensorError>

Sigmoid activation: 1 / (1 + exp(-x)).

Source

pub fn tanh(&self) -> Result<Variable, TensorError>

Tanh activation: element-wise hyperbolic tangent.

Source

pub fn gelu(&self) -> Result<Variable, TensorError>

GELU activation (Gaussian Error Linear Unit, erf form): 0.5 * x * (1 + erf(x / sqrt(2))).

Source

pub fn gelu_tanh(&self) -> Result<Variable, TensorError>

Tanh-approximation GELU. Matches HuggingFace hidden_act="gelu_new" and PyTorch F.gelu(x, approximate="tanh"). Used by ALBERT, GPT-2, and any HF checkpoint that ships hidden_act in {gelu_new, gelu_pytorch_tanh}.

Source

pub fn silu(&self) -> Result<Variable, TensorError>

SiLU activation (Sigmoid Linear Unit, aka Swish): x * sigmoid(x).

Source

pub fn leaky_relu(&self, negative_slope: f64) -> Result<Variable, TensorError>

Leaky ReLU: max(0, x) + negative_slope * min(0, x).

Source

pub fn elu(&self, alpha: f64) -> Result<Variable, TensorError>

ELU: max(0, x) + min(0, alpha * (exp(x) - 1)).

Source

pub fn softplus( &self, beta: f64, threshold: f64, ) -> Result<Variable, TensorError>

Softplus: (1/beta) * log(1 + exp(beta * x)).

Source

pub fn mish(&self) -> Result<Variable, TensorError>

Mish: x * tanh(softplus(x)).

Source

pub fn selu(&self) -> Result<Variable, TensorError>

SELU: self-normalizing ELU with fixed alpha/lambda.

Source

pub fn hardswish(&self) -> Result<Variable, TensorError>

Hardswish: x * clamp(x + 3, 0, 6) / 6.

Source

pub fn hardsigmoid(&self) -> Result<Variable, TensorError>

Hardsigmoid: clamp(x + 3, 0, 6) / 6.

Source

pub fn prelu(&self, weight: &Variable) -> Result<Variable, TensorError>

PReLU: max(0, x) + weight * min(0, x).

Source

pub fn sum(&self) -> Result<Variable, TensorError>

Sum of all elements, returning a scalar.

Source

pub fn mean(&self) -> Result<Variable, TensorError>

Mean of all elements, returning a scalar.

Source

pub fn sum_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Sum along a dimension. If keepdim, the reduced dimension is retained with size 1.

Source

pub fn mean_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Mean along a dimension. If keepdim, the reduced dimension is retained with size 1.

Source

pub fn prod(&self) -> Result<Variable, TensorError>

Product of all elements, returning a scalar.

Source

pub fn prod_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Product along a dimension.

Source

pub fn cumsum(&self, dim: i32) -> Result<Variable, TensorError>

Cumulative sum along a dimension.

Source

pub fn logsumexp( &self, dim: i32, keepdim: bool, ) -> Result<Variable, TensorError>

Log of summed exponentials along a dimension (numerically stable).

Source

pub fn min(&self) -> Result<Variable, TensorError>

Minimum of all elements, returning a scalar.

Source

pub fn max(&self) -> Result<Variable, TensorError>

Maximum of all elements, returning a scalar.

Source

pub fn min_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Minimum values along a dimension.

Source

pub fn max_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Maximum values along a dimension.

Source

pub fn var(&self) -> Result<Variable, TensorError>

Variance of all elements (Bessel-corrected).

Source

pub fn std(&self) -> Result<Variable, TensorError>

Standard deviation of all elements (Bessel-corrected).

Source

pub fn var_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Variance along a dimension (Bessel-corrected).

Source

pub fn std_dim(&self, dim: i32, keepdim: bool) -> Result<Variable, TensorError>

Standard deviation along a dimension (Bessel-corrected).

Source

pub fn softmax(&self, dim: i32) -> Result<Variable, TensorError>

Softmax along dim (output sums to 1).

Source

pub fn log_softmax(&self, dim: i32) -> Result<Variable, TensorError>

Log-softmax along dim (numerically stable log(softmax(x))).

Source

pub fn exp(&self) -> Result<Variable, TensorError>

Element-wise exponential.

Source

pub fn log(&self) -> Result<Variable, TensorError>

Element-wise natural logarithm.

Source

pub fn sqrt(&self) -> Result<Variable, TensorError>

Element-wise square root.

Source

pub fn abs(&self) -> Result<Variable, TensorError>

Element-wise absolute value.

Source

pub fn pow_scalar(&self, exponent: f64) -> Result<Variable, TensorError>

Raise each element to a scalar power.

Source

pub fn triu(&self, diagonal: i64) -> Result<Variable, TensorError>

Upper triangular part. diagonal=0 keeps the main diagonal; positive shifts up.

Source

pub fn tril(&self, diagonal: i64) -> Result<Variable, TensorError>

Lower triangular part. diagonal=0 keeps the main diagonal; negative shifts down.

Source

pub fn sin(&self) -> Result<Variable, TensorError>

Element-wise sine.

Source

pub fn cos(&self) -> Result<Variable, TensorError>

Element-wise cosine.

Source

pub fn sign(&self) -> Result<Variable, TensorError>

Element-wise sign (-1, 0, or 1).

Source

pub fn floor(&self) -> Result<Variable, TensorError>

Element-wise floor (round toward negative infinity).

Source

pub fn ceil(&self) -> Result<Variable, TensorError>

Element-wise ceil (round toward positive infinity).

Source

pub fn round(&self) -> Result<Variable, TensorError>

Element-wise round to nearest integer.

Source

pub fn reciprocal(&self) -> Result<Variable, TensorError>

Element-wise reciprocal (1/x).

Source

pub fn clamp(&self, min: f64, max: f64) -> Result<Variable, TensorError>

Clamp all elements to [min, max].

Source

pub fn clamp_min(&self, min: f64) -> Result<Variable, TensorError>

Clamp all elements to be at least min.

Source

pub fn clamp_max(&self, max: f64) -> Result<Variable, TensorError>

Clamp all elements to be at most max.

Source

pub fn log1p(&self) -> Result<Variable, TensorError>

Element-wise log(1 + x), numerically stable for small x.

Source

pub fn expm1(&self) -> Result<Variable, TensorError>

Element-wise exp(x) - 1, numerically stable for small x.

Source

pub fn log2(&self) -> Result<Variable, TensorError>

Element-wise base-2 logarithm.

Source

pub fn log10(&self) -> Result<Variable, TensorError>

Element-wise base-10 logarithm.

Source

pub fn atan2(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise atan2 (arc tangent of y/x).

Source

pub fn maximum(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise maximum of two variables.

Source

pub fn minimum(&self, other: &Variable) -> Result<Variable, TensorError>

Element-wise minimum of two variables.

Source

pub fn masked_fill( &self, mask: &Tensor, value: f64, ) -> Result<Variable, TensorError>

Fill elements where mask is true (non-zero) with value.

Source

pub fn normalize(&self, p: f64, dim: i32) -> Result<Variable, TensorError>

L_p normalize along a dimension.

Source

pub fn cosine_similarity( &self, other: &Variable, dim: i64, eps: f64, ) -> Result<Variable, TensorError>

Cosine similarity between two variables along a dimension.

Source

pub fn reshape(&self, shape: &[i64]) -> Result<Variable, TensorError>

Return a view with a different shape. Use -1 for one inferred dimension.

Source

pub fn transpose(&self, dim0: i32, dim1: i32) -> Result<Variable, TensorError>

Swap two dimensions.

Source

pub fn permute(&self, dims: &[i64]) -> Result<Variable, TensorError>

Reorder dimensions according to the given permutation.

Source

pub fn squeeze(&self, dim: i32) -> Result<Variable, TensorError>

Remove a dimension of size 1.

Source

pub fn unsqueeze(&self, dim: i32) -> Result<Variable, TensorError>

Insert a dimension of size 1 at dim.

Source

pub fn unsqueeze_many(&self, dims: &[i32]) -> Result<Variable, TensorError>

Insert size-1 dimensions at multiple positions.

Source

pub fn flatten( &self, start_dim: i32, end_dim: i32, ) -> Result<Variable, TensorError>

Flatten dimensions from start_dim to end_dim (inclusive) into one.

Source

pub fn expand(&self, shape: &[i64]) -> Result<Variable, TensorError>

Broadcast to shape without copying data. Use -1 to keep a dimension.

Source

pub fn narrow( &self, dim: i32, start: i64, length: i64, ) -> Result<Variable, TensorError>

Slice along dim from start for length elements.

Source

pub fn select(&self, dim: i32, index: i64) -> Result<Variable, TensorError>

Select a single index along dim, removing that dimension.

Source

pub fn index_select( &self, dim: i32, index: &Tensor, ) -> Result<Variable, TensorError>

Select multiple indices along dim using a 1-D index tensor.

Source

pub fn gather(&self, dim: i32, index: &Tensor) -> Result<Variable, TensorError>

Gather values along dim using an index tensor of the same rank.

Source

pub fn cat(&self, other: &Variable, dim: i32) -> Result<Variable, TensorError>

Concatenate two variables along dim.

Source

pub fn cat_many(vars: &[&Variable], dim: i32) -> Result<Variable, TensorError>

Concatenate multiple variables along dim (torch.cat).

Source

pub fn stack(vars: &[Variable], dim: i32) -> Result<Variable, TensorError>

Stack variables along a new dimension (torch.stack).

Source

pub fn chunk(&self, chunks: i32, dim: i32) -> Result<Vec<Variable>, TensorError>

Split into chunks pieces along dim (torch.chunk).

Source

pub fn repeat(&self, repeats: &[i64]) -> Result<Variable, TensorError>

Repeat the tensor along each dimension by the given counts.

Source

pub fn pad(&self, padding: &[i64], value: f64) -> Result<Variable, TensorError>

Constant-value padding. padding is [left, right, ...] from the last dim inward.

Source

pub fn topk( &self, k: i64, dim: i32, largest: bool, sorted: bool, ) -> Result<(Variable, Tensor), TensorError>

Return the k largest (or smallest) values and their indices along dim.

Source

pub fn sort( &self, dim: i32, descending: bool, ) -> Result<(Variable, Tensor), TensorError>

Sort along dim. Returns (sorted_values, indices).

Source§

impl Variable

Source

pub fn new(data: Tensor, requires_grad: bool) -> Variable

Create a leaf variable (parameter or input data). If requires_grad is true, libtorch will track operations for autodiff.

§Panics

Panics if gradient tracking cannot be enabled — e.g. on integer dtypes (libtorch: only floating-point tensors can require gradients). PyTorch raises the same error; silently returning a non-tracking variable would train nothing for this parameter.

Source

pub fn data(&self) -> Tensor

Get the underlying tensor data (shallow clone sharing storage).

The returned Tensor shares the same memory as the variable’s data — the same aliasing semantics as PyTorch’s .data. In-place mutations on either side will be visible to both. If you need an independent copy, call .data().copy() instead. Because the share counts as the same tensor for concurrency purposes, the thread-safety rules on Tensor apply across it: never mutate in place while any other thread accesses the variable.

Source

pub fn id(&self) -> usize

Stable identity of the shared autograd cell: clones of the same Variable return the same id, independent copies differ. This is what parameter collection dedups on (a module graph can reach the same shared parameter through several paths). Valid only while some clone is alive — ids can be recycled after the last clone drops.

Source

pub fn grad(&self) -> Option<Tensor>

Get the accumulated gradient, if any. Reads from the C++ tensor’s .grad() field.

Source

pub fn set_grad(&self, grad: Tensor)

Replace the gradient tensor directly (e.g. for gradient clipping or unscaling). Equivalent to param.grad = grad in PyTorch.

Panics if the FFI write fails (broken tensor handle): dropping the write silently would let the optimizer step with the old gradient.

Source

pub fn requires_grad(&self) -> bool

Whether this variable tracks gradients.

Source

pub fn set_requires_grad(&self, requires_grad: bool) -> Result<(), TensorError>

Change whether this variable tracks gradients. Replaces the inner data handle (the FFI returns a new handle sharing storage). All clones of this Variable share the same Rc<RefCell>, so the change is visible everywhere (module, optimizer, etc.).

Source

pub fn is_leaf(&self) -> bool

Whether this is a leaf variable (no grad_fn in libtorch). A leaf tensor is one created by the user, not by an operation.

Source

pub fn ensure_grad_accumulator( &self, ) -> Result<Option<GradAccumulatorHandle>, TensorError>

Force creation of the AccumulateGrad node for a leaf variable (requires_grad=true). The node’s stream is captured from the current CUDA stream at the moment of this call. Returns a handle that keeps the node alive; store it for the lifetime of the worker.

Call this under a StreamGuard on the training compute stream during DDP worker setup so that gradient accumulation runs on the same stream as subsequent forward/backward passes. Without this, the node is created lazily on first backward() — inside the autograd engine’s worker thread, whose current stream is the device default, and libtorch fires the “AccumulateGrad node’s stream does not match” warning on every run.

Returns Ok(None) for non-leaf or non-requires-grad variables.

Source

pub fn autograd_node_count(&self) -> i64

Count unique autograd nodes reachable from this variable’s grad_fn. Returns 0 for leaf variables. Measures graph complexity — compare against Python’s equivalent to detect decomposed-op bloat.

Source

pub fn shape(&self) -> Vec<i64>

Shape of the underlying tensor (e.g. [batch, features]).

Source

pub fn dtype(&self) -> DType

Data type of the underlying tensor (e.g. Float32, Float16).

Source

pub fn device(&self) -> Device

Device where the underlying tensor lives (CPU or CUDA(n)).

Source

pub fn item(&self) -> Result<f64, TensorError>

Extract a scalar value as f64. The tensor must contain exactly one element.

Source

pub fn zero_grad(&self)

Zero out the accumulated gradient (fills .grad() with zeros). See also zero_grad_set_to_none for the faster alternative.

Panics if the FFI call fails (broken tensor handle): a silently skipped zero would leak the previous step’s gradient into the next.

Source

pub fn zero_grad_set_to_none(&self)

Null out the gradient instead of zeroing it. No CUDA kernel.

Source

pub fn detach(&self) -> Variable

Detach from the computation graph. Returns a new leaf variable sharing the same data tensor (detached) with no gradient tracking.

Panics if the FFI call fails (broken tensor handle): falling back to the attached tensor would silently keep gradients flowing where the caller asked to cut them.

Source

pub fn to_device(&self, device: Device) -> Result<Variable, TensorError>

Move to a different device. Returns a new leaf variable.

Source

pub fn set_data(&self, data: Tensor)

Replace the underlying tensor data (used by optimizers). Preserves the requires_grad flag from the current tensor. Bumps data_generation so caches built from the old tensor can detect the replacement.

Source

pub fn data_generation(&self) -> u64

Monotonic generation of the underlying tensor: incremented every time set_data replaces it (checkpoint load, dtype cast, device move). In-place mutation (copy_, optimizer steps) does not change it. Module caches built from parameter tensors key their validity on this.

Source

pub fn numel(&self) -> i64

Total number of elements in the tensor (product of all dimensions).

Source

pub fn backward(&self) -> Result<(), TensorError>

Run backward pass from this scalar variable. Populates .grad() on all leaf variables in the computation graph.

After backward completes, the tensor is detached in-place to immediately release the C++ grad_fn chain. Without this, the autograd Node objects stay alive until the Variable is dropped.

Trait Implementations§

Source§

impl Clone for Variable

Source§

fn clone(&self) -> Variable

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for Variable

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result<(), Error>

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self> ⓘ

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self> ⓘ

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = !

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, !>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self> ⓘ
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self> ⓘ

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more