pub struct PreparedCompressedIndex {
pub layer_idx: usize,
pub head_idx: usize,
pub head_dim: usize,
pub num_tokens: usize,
pub num_heads: usize,
pub key_codes: Vec<FibCodeV1>,
pub value_codes: Vec<FibCodeV1>,
pub scorer: FibScorer,
}Expand description
Pre-built compressed index for one layer/head, avoiding per-call reconstruction.
Caches decoded key codes, value codes, and a built FibScorer so that
attention_topk_compressed_prepared only needs to prepare the query
(O(dim)) and score (O(num_tokens)) without rebuilding codec state.
Fields§
§layer_idx: usizeLayer index this index was built for.
head_idx: usizeHead index this index was built for.
head_dim: usizeHead dimension.
num_tokens: usizeNumber of tokens in the pool.
num_heads: usizeNumber of KV heads.
key_codes: Vec<FibCodeV1>Decoded key codes for the entire layer (all heads, all tokens).
value_codes: Vec<FibCodeV1>Decoded value codes for the entire layer (all heads, all tokens).
scorer: FibScorerBuilt FibScorer (includes quantizer).
Auto Trait Implementations§
impl Freeze for PreparedCompressedIndex
impl RefUnwindSafe for PreparedCompressedIndex
impl Send for PreparedCompressedIndex
impl Sync for PreparedCompressedIndex
impl Unpin for PreparedCompressedIndex
impl UnsafeUnpin for PreparedCompressedIndex
impl UnwindSafe for PreparedCompressedIndex
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§impl<T> Pointable for T
impl<T> Pointable for T
Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
The inverse inclusion map: attempts to construct
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
Checks if
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
Use with care! Same as
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
The inclusion map: converts
self to the equivalent element of its superset.Source§impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
impl<SS, SP> SupersetOf<SS> for SPwhere
SS: SubsetOf<SP>,
Source§fn to_subset(&self) -> Option<SS>
fn to_subset(&self) -> Option<SS>
The inverse inclusion map: attempts to construct
self from the equivalent element of its
superset. Read moreSource§fn is_in_subset(&self) -> bool
fn is_in_subset(&self) -> bool
Checks if
self is actually part of its subset T (and can be converted to it).Source§fn to_subset_unchecked(&self) -> SS
fn to_subset_unchecked(&self) -> SS
Use with care! Same as
self.to_subset but without any property checks. Always succeeds.Source§fn from_subset(element: &SS) -> SP
fn from_subset(element: &SS) -> SP
The inclusion map: converts
self to the equivalent element of its superset.