Skip to main content

Module block

Module block 

Source

Structs§

PhysicalBlock
Represents a physical memory block storing KV-tensors for a fixed number of tokens.

Constants§

DEFAULT_BLOCK_SIZE
Size of a block in number of tokens (typically 16, 32, or 64).

Type Aliases§

PhysicalBlockId
Unique physical block index in the memory pool.