#[repr(u8)]pub enum AttentionProjection {
Full = 0,
SharedKV = 1,
}Expand description
Attention projection configuration. Controls whether K and V projections share weights (Q-K=V tying).
Variants§
Full = 0
Standard Q, K, V (3 projections, full KV cache)
Q-K=V: K and V share projection (2 projections, K-only cache). 50% KV cache reduction, ~3% perplexity cost. Post-hoc weight merging: W_kv = (W_k + W_v) / 2.
Trait Implementations§
Source§impl Clone for AttentionProjection
impl Clone for AttentionProjection
Source§fn clone(&self) -> AttentionProjection
fn clone(&self) -> AttentionProjection
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreimpl Copy for AttentionProjection
Source§impl Debug for AttentionProjection
impl Debug for AttentionProjection
Source§impl Default for AttentionProjection
impl Default for AttentionProjection
Source§fn default() -> AttentionProjection
fn default() -> AttentionProjection
Returns the “default value” for a type. Read more
impl Eq for AttentionProjection
Source§impl PartialEq for AttentionProjection
impl PartialEq for AttentionProjection
impl StructuralPartialEq for AttentionProjection
Auto Trait Implementations§
impl Freeze for AttentionProjection
impl RefUnwindSafe for AttentionProjection
impl Send for AttentionProjection
impl Sync for AttentionProjection
impl Unpin for AttentionProjection
impl UnsafeUnpin for AttentionProjection
impl UnwindSafe for AttentionProjection
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more