Enum Op 

pub enum Op {
    Embed {
        table: W,
        out: Val,
    },
    LayerNorm {
        x: Val,
        w: W,
        b: Option<W>,
        eps: f64,
        out: Val,
    },
    Gemm {
        a: Val,
        w: W,
        b: Option<W>,
        epilogue: Epilogue,
        out: Val,
    },
    Rope {
        qkv: Val,
        theta: f64,
    },
    Attention {
        qkv: Val,
        window: Option<usize>,
        out: Val,
    },
    GeGlu {
        x: Val,
        out: Val,
    },
    AddType {
        h: Val,
        table: W,
    },
    GatherMarkers {
        h: Val,
        out: Val,
    },
    ActFeatures {
        h: Val,
        logits: Val,
        out: Val,
    },
}

One step of a graph. Every op runs on the rows the batch has, not the bucket’s padded count.

Variants

Embed

out[i] = table[ids[i]], token rows.

Fields

table: W

[vocab, width].

out: Val

Token rows.

LayerNorm

LayerNorm over each row, statistics in f64.

Fields

x: Val

Input.

w: W

Scale.

b: Option<W>

Shift, if the norm has one.

eps: f64

Epsilon.

out: Val

Output, the same shape as x.

Gemm

out = epilogue(a wᵀ + b) with w as [n, k].

Fields

a: Val

Input, k wide.

w: W

[n, k].

b: Option<W>

[n], if there is one.

epilogue: Epilogue

What happens to each result.

out: Val

Output, n wide, with as many rows as a.

Rope

Rotary embedding applied in place to the q and k parts of fused [q | k | v] rows, with positions counted from the start of each sequence.

Fields

qkv: Val

Token rows, 3 heads 64 wide.

theta: f64

The base.

Attention

Self attention within each sequence over fused [q | k | v] rows.

Fields

qkv: Val

Token rows, 3 heads 64 wide.

window: Option<usize>

Keys at most this far away on either side, or all of them.

out: Val

Token rows, heads 64 wide.

GeGlu

out = gelu(x[:, ..n]) * x[:, n..] with x twice as wide as out.

Fields

x: Val

Input.

out: Val

Output.

AddType

Adds table[qtype[s]] to every row of sequence s, in place.

Fields

h: Val

Token rows.

table: W

[types, width].

GatherMarkers

The row of each marker.

Fields

h: Val

Token rows.

out: Val

Marker rows.

ActFeatures

Per sequence, its first row followed by [top1, top1 - top2, entropy / ln k, k / 255] over the softmax of its markers’ logits, Laya’s act head input.

Fields

h: Val

Token rows.

logits: Val

Marker rows, one wide.

out: Val

Sequence rows, h width plus four.

Implementations

impl Op

pub fn uses(&self) -> (Vec<Val>, Vec<Val>)

Values the op reads, then values it writes. An in place op lists the value in both.

Trait Implementations

impl Clone for Op

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Op

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for Op

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Op