pub struct BlockTable {
pub physical_blocks: Vec<usize>,
pub prefix_hashes: Vec<u64>,
pub seq_len: usize,
pub block_size: usize,
}Expand description
Virtual-to-Physical block mapping table for a single sequence or request.
Fields§
§physical_blocks: Vec<usize>Ordered list of physical blocks allocated to this sequence.
prefix_hashes: Vec<u64>Prefix hash of each committed block in this sequence.
seq_len: usizeNumber of tokens currently registered in the sequence.
block_size: usizeSize of each block in tokens.
Implementations§
Source§impl BlockTable
impl BlockTable
pub fn new(block_size: usize) -> BlockTable
Sourcepub fn append_token(
&mut self,
token_id: u32,
allocator: &mut BlockAllocator,
) -> Result<(), CacheError>
pub fn append_token( &mut self, token_id: u32, allocator: &mut BlockAllocator, ) -> Result<(), CacheError>
Appends a new token to this sequence’s KV-cache, allocating a block if needed and performing Copy-on-Write if the target block is shared.
Sourcepub fn append_tokens(
&mut self,
tokens: &[u32],
allocator: &mut BlockAllocator,
) -> Result<(), CacheError>
pub fn append_tokens( &mut self, tokens: &[u32], allocator: &mut BlockAllocator, ) -> Result<(), CacheError>
Appends a slice of tokens in batch with automatic prefix cache matching.
Sourcepub fn rollback(
&mut self,
valid_token_count: usize,
allocator: &mut BlockAllocator,
) -> Result<usize, CacheError>
pub fn rollback( &mut self, valid_token_count: usize, allocator: &mut BlockAllocator, ) -> Result<usize, CacheError>
Speculative Decoding Rollback:
Instantly rewinds the sequence to valid_token_count tokens.
Trailing unneeded blocks are freed, and partial boundary blocks undergo
Copy-on-Write if they were shared.
Sourcepub fn translate_logical_offset(
&self,
logical_index: usize,
) -> Result<(usize, usize), CacheError>
pub fn translate_logical_offset( &self, logical_index: usize, ) -> Result<(usize, usize), CacheError>
Translates a logical token position (0..seq_len) to its physical (block_id, offset) coordinates.
Sourcepub fn free_all(
&mut self,
allocator: &mut BlockAllocator,
) -> Result<(), CacheError>
pub fn free_all( &mut self, allocator: &mut BlockAllocator, ) -> Result<(), CacheError>
Cleans up and frees all physical blocks allocated to this sequence.
Trait Implementations§
Source§impl Clone for BlockTable
impl Clone for BlockTable
Source§fn clone(&self) -> BlockTable
fn clone(&self) -> BlockTable
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more