Struct dfdx::nn::Linear

pub struct Linear<const I: usize, const O: usize> {
    pub weight: Tensor2D<O, I, NoneTape>,
    pub bias: Tensor1D<O, NoneTape>,
}

A linear transformation of the form weight * x + bias, where weight is a matrix, x is a vector or matrix, and bias is a vector.

Generics

• I The “input” size of vectors & matrices.
• O The “output” size of vectors & matrices.

Examples

Linear<5, 2> can act on vectors with 5 elements, and results in vectors with 2 elements.

let model: Linear<5, 2> = Default::default();
assert_eq!(model.weight.data(), &[[0.0; 5]; 2]);
assert_eq!(model.bias.data(), &[0.0; 2]);
let x: Tensor1D<5> = Default::default();
let y: Tensor1D<2> = model.forward(x);
assert_eq!(y.data(), &[0.0; 2]);

Fields

weight: Tensor2D<O, I, NoneTape>

Transposed weight matrix, shape (O, I)

bias: Tensor1D<O, NoneTape>

Bias vector, shape (O, )

Trait Implementations

impl<const I: usize, const O: usize> CanUpdateWithGradients for Linear<I, O>

fn update<G: GradientProvider>(
    &mut self,
    grads: &mut G,
    unused: &mut UnusedTensors
)

Updates self given the GradientProvider. When any parameters that are NOT present in G, then this function should add the tensor’s UniqueId to UnusedTensors.

impl<const I: usize, const O: usize> Clone for Linear<I, O>

fn clone(&self) -> Linear<I, O>

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<const I: usize, const O: usize> Debug for Linear<I, O>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<const I: usize, const O: usize> Default for Linear<I, O>

fn default() -> Linear<I, O>

Returns the “default value” for a type.

impl<const I: usize, const O: usize> LoadFromNpz for Linear<I, O>

fn read<R>(&mut self, pre: &str, r: &mut ZipArchive<R>) -> Result<(), NpzError> where
    R: Read + Seek, 

Reads Self::weight from {pre}weight.npy and Self::bias from {pre}bias.npy using npz_fread().

fn load<P: AsRef<Path>>(&mut self, path: P) -> Result<(), NpzError>

Loads data from a .npz zip archive at the specified path.

impl<const I: usize, const O: usize, H: Tape> Module<Tensor1D<I, H>> for Linear<I, O>

fn forward(&self, x: Tensor1D<I, H>) -> Self::Output

1d forward using vecmat_mul() and add().

type Output = Tensor1D<O, H>

The type that this unit produces given Input.

fn forward_mut(&mut self, input: Input) -> Self::Output

Pass an Input through the unit and produce Self::Output. Can be implemented for multiple Input types.

impl<const B: usize, const I: usize, const O: usize, H: Tape> Module<Tensor2D<B, I, H>> for Linear<I, O>

fn forward(&self, x: Tensor2D<B, I, H>) -> Self::Output

Batched 2d forward using matmul() and add()

type Output = Tensor2D<B, O, H>

The type that this unit produces given Input.

fn forward_mut(&mut self, input: Input) -> Self::Output

Pass an Input through the unit and produce Self::Output. Can be implemented for multiple Input types.

impl<const B: usize, const S: usize, const I: usize, const O: usize, H: Tape> Module<Tensor3D<B, S, I, H>> for Linear<I, O>

fn forward(&self, x: Tensor3D<B, S, I, H>) -> Self::Output

Batched 3d forward using matmul() and add()

type Output = Tensor3D<B, S, O, H>

The type that this unit produces given Input.

fn forward_mut(&mut self, input: Input) -> Self::Output

Pass an Input through the unit and produce Self::Output. Can be implemented for multiple Input types.

impl<const I: usize, const O: usize> ResetParams for Linear<I, O>

fn reset_params<R: Rng>(&mut self, rng: &mut R)

Initializes Self::weight and Self::bias from a Uniform distribution between [-1 / sqrt(I), 1 / sqrt(I)].

This uses Randomize::randomize() to set the values of the tensor.

impl<const I: usize, const O: usize> SaveToNpz for Linear<I, O>

fn write<W>(&self, pre: &str, w: &mut ZipWriter<W>) -> ZipResult<()> where
    W: Write + Seek, 

Saves Self::weight to {pre}weight.npy and Self::bias to {pre}bias.npy using npz_fwrite().

fn save<P: AsRef<Path>>(&self, path: P) -> ZipResult<()>

Save this object into the .npz file determined located at path.