Struct WgpuContext 

pub struct WgpuContext {
    pub device: Device,
    pub queue: Queue,
    pub adapter_info: AdapterInfo,
    /* private fields */
}

Owns the wgpu device/queue and a cache of compiled compute pipelines.

Fields

device: Device

queue: Queue

adapter_info: AdapterInfo

Implementations

impl WgpuContext

pub fn new() -> Result<Arc<Self>>

Available on non-WebAssembly only.

Sync device creation — native only. On wasm use Self::new_async.

pub async fn new_async() -> Result<Arc<Self>>

Async device creation (works on native and wasm32; roadmap v4, pitfall 14: the async form is primary, the sync API is the facade).

pub fn scope(self: &Arc<Self>) -> DispatchScope

Open a recording scope: every WgpuContext::dispatch made while the returned guard is alive is recorded into one command encoder and submitted once, when the outermost guard drops.

This is the difference between ~100 submits per decoded token and one. Scopes nest — only the outermost submits — so a caller can wrap a whole decode step without knowing whether its callees also wrap their bodies.

Reads are safe across a scope: any readback flushes first (see WgpuContext::flush), and compute passes within one encoder execute in the order they were recorded.

pub fn flush(&self)

Submit whatever a scope has recorded so far, without closing it.

Every readback path calls this first. Making it the readback’s job rather than the caller’s is deliberate: “remember to flush before you read” is exactly the kind of rule that silently rots into a stale-bytes bug, and the cost is one uncontended mutex on a path that is about to wait on a fence anyway.

pub fn create_pooled(self: &Arc<Self>, size_bytes: usize) -> PooledBuffer

A storage buffer, from the free list when one of this exact size is waiting. See PooledBuffer.

pub fn stats(&self) -> Stats

Snapshot of this device’s cumulative counters. Subtract two snapshots with Stats::since to measure one region.

pub fn create_storage(&self, size_bytes: usize) -> Buffer

pub fn create_zeroed(self: &Arc<Self>, size_bytes: usize) -> PooledBuffer

A storage buffer that never came from the pool, and never returns to it — so it is guaranteed zero-filled, which WebGPU promises for a newly created buffer and a recycled one obviously cannot.

For the kernels that write only part of their output and rely on the rest being zero. There is exactly one today (unsplit_heads, which fills one third of a [t, 3c] gradient), and it is worth a dedicated allocator rather than zeroing every recycled buffer: zeroing through queue.write_buffer would reintroduce the ordering hazard described on PooledBuffer, and clearing through the encoder would force a pass boundary on a path that has no other reason for one.

pub fn upload(self: &Arc<Self>, bytes: &[u8]) -> PooledBuffer

A fresh storage buffer holding bytes.

Deliberately outside the pool in both directions — it neither takes a recycled buffer nor returns this one. See PooledBuffer: a queue.write_buffer into a recycled buffer can land ahead of recorded dispatches that still expect its old contents.

pub fn readback( &self, buf: &Buffer, offset_bytes: usize, size_bytes: usize, ) -> Result<Vec<u8>>

Available on non-WebAssembly only.

Read size_bytes starting at offset_bytes back to the host. Sync facade — native only (device.poll(Wait) cannot exist on wasm).

pub async fn readback_async( &self, buf: &Buffer, offset_bytes: usize, size_bytes: usize, ) -> Result<Vec<u8>>

Async readback — the primary form; on wasm the browser event loop drives the mapping.

pub async fn readback_many_async( &self, regions: &[(&Buffer, usize, usize)], ) -> Result<Vec<Vec<u8>>>

Read several regions back in one submit and one fence wait.

WgpuContext::readback_async costs a submit and a wait each, which dominates when a single logical step wants several small tensors: the attention probe reads n_layer + 1 per generated token, and one at a time that cost more than the decode itself. Staged into one encoder they cost one round trip regardless of how many there are.

Regions are returned in the order given.

pub fn dispatch( &self, name: &'static str, params: &[u32], buffers: &[(&Buffer, usize, usize)], workgroups: (u32, u32, u32), )

Dispatch name with binding 0 = params (uniform, raw words) and bindings 1.. = buffers (storage). Each buffer entry is (buffer, offset_bytes, size_bytes).

Trait Implementations

impl Debug for WgpuContext

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.