pub struct CacheSignature {
pub format_version: u32,
pub model: String,
pub n_layers: usize,
pub n_kv_heads: usize,
pub head_dim: usize,
pub dtype: KvDtype,
pub tokens: usize,
pub layout: BlockLayout,
}Expand description
What a block’s payload is: the shape any reader must match to use
it. Construct it from a payload with Self::from_payload, or as a
reader’s requirement with Self::expected.
Fields§
§format_version: u32§model: StringIdentifies the weights the KV state was computed under. The one field no payload can prove about itself – which is exactly why it is checked against the reader’s expectation in step 3.
n_layers: usize§n_kv_heads: usize§head_dim: usize§dtype: KvDtype§tokens: usizeToken positions actually stored, per layer. Equal to
layout.block_size() for any block this module will stamp or
verify.
layout: BlockLayoutHow the sequence was cut into blocks, and the sliding window that cut had to line up with. See the module note.
Implementations§
Source§impl CacheSignature
impl CacheSignature
Sourcepub fn from_payload(
model: &str,
layout: BlockLayout,
layers: &[KvCache],
) -> Result<Self, SignatureError>
pub fn from_payload( model: &str, layout: BlockLayout, layers: &[KvCache], ) -> Result<Self, SignatureError>
Derives a signature by measuring layers. There is
deliberately no parameter to fill a gap from: every shape field
except model and the sliding window comes from the tensors
themselves, and even the declared layout’s block size is
checked against the depth the tensors actually have.
Fails if the payload cannot describe itself coherently: no layers at all, layers that disagree with each other, a layer whose buffers do not match its own declared shape, or a token depth that is not the block size the layout claims.
Trait Implementations§
Source§impl Clone for CacheSignature
impl Clone for CacheSignature
Source§fn clone(&self) -> CacheSignature
fn clone(&self) -> CacheSignature
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreSource§impl Debug for CacheSignature
impl Debug for CacheSignature
impl Eq for CacheSignature
Source§impl PartialEq for CacheSignature
impl PartialEq for CacheSignature
impl StructuralPartialEq for CacheSignature
Auto Trait Implementations§
impl Freeze for CacheSignature
impl RefUnwindSafe for CacheSignature
impl Send for CacheSignature
impl Sync for CacheSignature
impl Unpin for CacheSignature
impl UnsafeUnpin for CacheSignature
impl UnwindSafe for CacheSignature
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more