Struct dfdx::nn::Linear

pub struct Linear<const I: usize, const O: usize> {
    pub weight: Tensor2D<O, I, NoTape>,
    pub bias: Tensor1D<O, NoTape>,
}

A linear transformation of the form x * transpose(W) + b, where W is a matrix, x is a vector or matrix, and b is a vector. If x is a matrix this does matrix multiplication.

Implements:

• Module for vectors like Tensor1D
• Module for matrices like Tensor2D<B, I>, where B is batch size
• ResetParams to set weight & bias to uniform random numbers from a distribution based on I.
• CanUpdateWithGradients
• SaveToNpz
• LoadFromNpz

Generics:

• I The input size of vectors & matrices.
• O The output size of vectors & matrices.

Example usage: Linear<5, 2> can act on vectors with 5 elements, and results in vectors with 2 elements.

let model: Linear<5, 2> = Default::default();
assert_eq!(model.weight.data(), &[[0.0; 5]; 2]);
assert_eq!(model.bias.data(), &[0.0; 2]);
let x: Tensor1D<5> = Default::default();
let y: Tensor1D<2> = model.forward(x);
assert_eq!(y.data(), &[0.0; 2]);

Fields

weight: Tensor2D<O, I, NoTape>

Transposed weight matrix, shape (O, I)

bias: Tensor1D<O, NoTape>

Bias vector, shape (O, )

Trait Implementations

impl<const I: usize, const O: usize> CanUpdateWithGradients for Linear<I, O>

fn update<G: GradientProvider>(&mut self, grads: &mut G)

impl<const I: usize, const O: usize> Clone for Linear<I, O>

fn clone(&self) -> Linear<I, O>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<const I: usize, const O: usize> Debug for Linear<I, O>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<const I: usize, const O: usize> Default for Linear<I, O>

fn default() -> Linear<I, O>

Returns the “default value” for a type.

impl<const I: usize, const O: usize> LoadFromNpz for Linear<I, O>

fn read<R>(
    &mut self,
    pre: &String,
    r: &mut ZipArchive<R>
) -> Result<(), NpzError> where
    R: Read + Seek, 

Reads self.weight from {pre}weight.npy and self.bias from {pre}bias.npy using [numpy::read()].

fn load<P: AsRef<Path>>(&mut self, path: P) -> Result<(), NpzError>

Loads data from a .npz zip archive at the specified path.

impl<const I: usize, const O: usize, H: Tape> Module<Tensor1D<I, H>> for Linear<I, O>

fn forward(&self, x: Tensor1D<I, H>) -> Self::Output

1d forward using vecmat_mul() and add().

type Output = Tensor1D<O, H>

impl<const B: usize, const I: usize, const O: usize, H: Tape> Module<Tensor2D<B, I, H>> for Linear<I, O>

fn forward(&self, x: Tensor2D<B, I, H>) -> Self::Output

Batched 2d forward using matmul() and add_broadcast_rhs_first()

type Output = Tensor2D<B, O, H>

impl<const I: usize, const O: usize> ResetParams for Linear<I, O>

fn reset_params<R: Rng>(&mut self, rng: &mut R)

Initializes self.weight and self.bias from a Uniform distribution between [-1 / sqrt(I), 1 / sqrt(I)].

This uses Randomize::randomize() to set the values of the tensor.

impl<const I: usize, const O: usize> SaveToNpz for Linear<I, O>

fn write<W>(&self, pre: &String, w: &mut ZipWriter<W>) -> ZipResult<()> where
    W: Write + Seek, 

Saves self.weight to {pre}weight.npy and self.bias to {pre}bias.npy using [numpy::write()].

fn save<P: AsRef<Path>>(&self, path: P) -> ZipResult<()>

Save this object into the .npz file determined located at path.