pub struct CacheSnapshot {
pub kv_len: Vec<Option<usize>>,
pub conv: Vec<Option<CudaSlice<f32>>>,
pub ssm: Vec<Option<CudaSlice<f32>>>,
pub pos: usize,
}Expand description
Snapshot of the dual cache taken BEFORE a spec-decode draft+verify round (MTP-PLAN §C/§D.4).
- Full-attn KV: only the per-layer
lenis recorded; rollback truncates (append-only, position-addressed — no copy). C.1. - Linear-attn conv/ssm: real device-to-device COPIES of the recurrent state, because those buffers are mutated IN PLACE by the verify pass and have no position index to truncate. C.2. (CudaSlice::clone is an Arc refcount, NOT a buffer copy — so we alloc fresh + memcpy_dtod.)
Fields§
§kv_len: Vec<Option<usize>>§conv: Vec<Option<CudaSlice<f32>>>§ssm: Vec<Option<CudaSlice<f32>>>§pos: usizeAuto Trait Implementations§
impl Freeze for CacheSnapshot
impl RefUnwindSafe for CacheSnapshot
impl Send for CacheSnapshot
impl Sync for CacheSnapshot
impl Unpin for CacheSnapshot
impl UnsafeUnpin for CacheSnapshot
impl UnwindSafe for CacheSnapshot
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more