Struct Tape 

pub struct Tape { /* private fields */ }

Implementations

impl Tape

pub fn new() -> Tape

pub fn leaf(&mut self, t: Tensor) -> TVar

Register a leaf (parameter or input); its gradient is what backward reports.

pub fn add(&mut self, a: &TVar, b: &TVar) -> Result<TVar>

pub fn matmul( &mut self, a: &TVar, b: &TVar, bias: Option<&TVar>, spec: MatmulSpec, ) -> Result<TVar>

Matmul with the specs the training forward uses: trans_a == false, b_rows == None, operands of equal rank (2x2 or batched 3x3).

pub fn gelu(&mut self, x: &TVar) -> Result<TVar>

pub fn layernorm( &mut self, x: &TVar, gamma: &TVar, beta: &TVar, eps: f32, ) -> Result<TVar>

pub fn softmax(&mut self, x: &TVar, causal: bool, off: usize) -> Result<TVar>

Softmax over the last dim (optionally causal). Saves the output for backward; the causal mask needs no bookkeeping there because masked outputs are exact zeros.

pub fn split_heads( &mut self, qkv: &TVar, n_head: usize, ) -> Result<(TVar, TVar, TVar)>

pub fn merge_heads(&mut self, x: &TVar) -> Result<TVar>

pub fn embedding( &mut self, ids: &Tensor, wte: &TVar, wpe: &TVar, pos: usize, ) -> Result<TVar>

Fused token + positional embedding over a single-chunk wte. dwte comes from scatter-add over the token ids; dwpe is the upstream gradient placed at rows pos..pos+t.

pub fn dropout(&mut self, x: &TVar, p: f32, seed: u32) -> Result<TVar>

Inverted dropout; the deterministic counter RNG regenerates the same mask in backward, so nothing is saved.

pub fn backward( &mut self, root: &TVar, seed_grad: Tensor, ) -> Result<Vec<Option<Tensor>>>

Reverse sweep from root seeded with seed_grad (dL/droot). Returns per-node gradients; leaves keep theirs, interior gradients are freed as soon as they have been consumed.

Trait Implementations

impl Default for Tape

fn default() -> Tape

Returns the “default value” for a type.