pub struct PrefillMirror {
pub kv_id: u64,
pub layer: usize,
pub limit: usize,
}Expand description
The decode graph’s device K/V mirror a prefill chunk of one layer also
writes to (wgpu): kv_id is the pipeline’s graph id, layer the
virtual layer index, limit the cache’s max_seq_len — the same key
and capacity ceiling the token graph uses, so the mirror the prefill
fills is the one the first decode token reads.
Fields§
§kv_id: u64§layer: usize§limit: usizeTrait Implementations§
Source§impl Clone for PrefillMirror
impl Clone for PrefillMirror
impl Copy for PrefillMirror
Source§impl Debug for PrefillMirror
impl Debug for PrefillMirror
impl Eq for PrefillMirror
Source§impl PartialEq for PrefillMirror
impl PartialEq for PrefillMirror
impl StructuralPartialEq for PrefillMirror
Auto Trait Implementations§
impl Freeze for PrefillMirror
impl RefUnwindSafe for PrefillMirror
impl Send for PrefillMirror
impl Sync for PrefillMirror
impl Unpin for PrefillMirror
impl UnsafeUnpin for PrefillMirror
impl UnwindSafe for PrefillMirror
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Compare self to
key and return true if they are equal.