Struct Dash 

pub struct Dash {

    pub layers: Vec<DashLayer>,
    pub block: usize,
    pub local_blocks: usize,
    pub local_window: usize,
    pub index_dim: usize,
    pub head_dim: usize,
    pub inv_tok: Vec<f32>,
    pub image_id: u32,
    pub page_id: u32,
    pub rope: Option<PageRope>,
    pub guard: Option<PageGuard>,
    pub cs: Vec<(f32, f32)>,
    pub angles: Vec<f32>,
    pub allow: Vec<bool>,
    pub cands: Vec<u32>,
    pub coff: Vec<u32>,
    pub k_bar: Vec<Vec<f32>>,
    pub open: Vec<Vec<f32>>,
    pub on_device: bool,
    /* private fields */
}

The router: weights, geometry and the sequence state on the host.

Fields

layers: Vec<DashLayer>

block: usize

local_blocks: usize

local_window: usize

index_dim: usize

head_dim: usize

inv_tok: Vec<f32>

image_id: u32

page_id: u32

rope: Option<PageRope>

guard: Option<PageGuard>

The answer’s page bookkeeping (_uocr_page_count_guard).

cs: Vec<(f32, f32)>

(sin, cos) of the current position, one per frequency pair.

angles: Vec<f32>

The angles of the current rows, one half per row.

allow: Vec<bool>

Candidate blocks of the current query.

cands: Vec<u32>

The blocks the device scores per row: remote and candidates, or every remote block when none is both. coff[row]..coff[row + 1].

coff: Vec<u32>

k_bar: Vec<Vec<f32>>

Per layer: completed block summaries [n, index_dim].

open: Vec<Vec<f32>>

Per layer: the keys of the open block.

on_device: bool

Whether the graph holds the router state instead of the host.

Implementations

impl Dash

pub fn from_model(model: &CmfModel) -> Result<Self, String>

pub fn begin(&mut self, ids: &[u32], eos_id: u32) -> Result<(), String>

A new sequence over the prompt ids.

pub fn at_prompt(&mut self, pos: usize)

Set the coordinates and candidates of a prompt position.

pub fn at_prompt_chunk(&mut self, pos: usize, k: usize)

The angles and candidates of the prompt rows pos..pos + k. allow is left at the last row.

pub fn at_decode(&mut self, id: u32, pos: usize)

Advance over decoded token id at pos and set its candidates.