Struct GpuGrammar 

pub struct GpuGrammar { /* private fields */ }

One request’s device-side JSON-schema grammar: the compiled schema, the live 64-word FSM state, and bind groups over the SHARED GrammarKernels + GrammarVocab. Exposes Self::mask (per-token accept bits), Self::advance (commit a chosen token), and Self::snap_advance/Self::restore (the speculative seam) — all byte-exact with the CPU FSM.

Implementations

impl GpuGrammar

pub const DEFAULT_SLOTS: u32 = 16

Default snapshot slots (the speculative draft depth the seam supports).

pub fn new( ctx: &GpuCtx, kernels: &GrammarKernels, vocab: &GrammarVocab, schema: &GpuSchema, ) -> Result<Self>

Build a per-request grammar over the shared kernels + vocab (uploads only the schema).

pub fn new_with_slots( ctx: &GpuCtx, kernels: &GrammarKernels, vocab: &GrammarVocab, schema: &GpuSchema, slots: u32, ) -> Result<Self>

Self::new with an explicit snapshot-slot count (speculative draft depth).

pub fn set_state(&self, ctx: &GpuCtx, words: &[u32; 64])

Overwrite the device FSM state (the 64 words of super::FsmState::as_words).

pub fn read_state(&self, ctx: &GpuCtx) -> Result<[u32; 64]>

Read the live device FSM state back (the same 64 words the CPU threads).

pub fn mask_dispatch(&self, ctx: &GpuCtx) -> Result<()>

Run mask_main and read back the per-token accept bits (1 = the grammar accepts stepping this token from the current state, 0 = rejected). Compute the mask on-device WITHOUT reading it back — the GPU-resident path, where the mask feeds a mask-apply kernel instead of the host. Blocks until the work completes so callers can time the kernel alone.

pub fn mask(&self, ctx: &GpuCtx) -> Result<Vec<u32>>

pub fn advance(&self, ctx: &GpuCtx, token: u32) -> Result<bool>

Advance the live state by token. Returns whether the grammar accepted it; on rejection the device state is left untouched (a bad token can never corrupt the live automaton).

pub fn snap_advance(&self, ctx: &GpuCtx, slot: u32, token: u32) -> Result<bool>

Snapshot the live state into slot, then advance by the draft token — the speculative seam. Pair with Self::restore to roll back a rejected draft.

pub fn restore(&self, ctx: &GpuCtx, slot: u32) -> Result<()>

Restore the live state from snapshot slot (a device-side buffer copy; no kernel).