pub struct PageTableHost {
pub block_table: Vec<u32>,
pub context_lens: Vec<u32>,
pub batch: usize,
pub max_blocks: usize,
}Expand description
A materialized Op::PagedAttn page table — the host-side block_table +
context_lens for a batch of B sessions, ready to upload as Op::Const
operands. block_table is row-major [B, max_blocks]; context_lens is
[B]. Both are u32 (the op’s declared operand dtype).
Fields§
§block_table: Vec<u32>Row-major [B, max_blocks] logical→physical map. Rows shorter than
max_blocks are zero-padded: index 0 is always an in-bounds physical
block, and every padded key position is ≥ context_len, so the op’s
variable-length mask (key_pos ≥ context_len → −inf) neutralizes it
before softmax. Padding is therefore never read, only never invalid.
context_lens: Vec<u32>Per-sequence context length (filled_tokens), [B].
batch: usizeB — number of sessions (rows).
max_blocks: usizemax_blocks — the padded row width (the widest session’s block count,
at least 1 so the tensor stays a well-formed rank-2 [B, max_blocks]).
Implementations§
Source§impl PageTableHost
impl PageTableHost
Sourcepub fn block_table_shape(&self) -> Shape
pub fn block_table_shape(&self) -> Shape
Shape of the block_table Op::Const — [B, max_blocks].
Sourcepub fn context_lens_shape(&self) -> Shape
pub fn context_lens_shape(&self) -> Shape
Shape of the context_lens Op::Const — [B].
Trait Implementations§
Source§impl Clone for PageTableHost
impl Clone for PageTableHost
Source§impl Debug for PageTableHost
impl Debug for PageTableHost
impl Eq for PageTableHost
Source§impl PartialEq for PageTableHost
impl PartialEq for PageTableHost
impl StructuralPartialEq for PageTableHost
Auto Trait Implementations§
impl Freeze for PageTableHost
impl RefUnwindSafe for PageTableHost
impl Send for PageTableHost
impl Sync for PageTableHost
impl Unpin for PageTableHost
impl UnsafeUnpin for PageTableHost
impl UnwindSafe for PageTableHost
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
key and return true if they are equal.impl<T> ErasedDestructor for Twhere
T: 'static,
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more