pub struct DecoderShapes {
pub vocab_size: usize,
pub hidden_size: usize,
pub intermediate_size: usize,
pub num_layers: usize,
pub q_width: usize,
pub kv_width: usize,
}Expand description
The tensor shapes a decoder-only transformer’s layers must have.
Grouped-query attention means k_proj/v_proj are narrower than
q_proj: their output width is num_key_value_heads * head_dim. Getting
that wrong is the single most common way a “successful” load ends up with
transposed or truncated attention weights, so it is a first-class field here
rather than an ad-hoc expression per model.
Fields§
§vocab_size: usizeVocabulary size — rows of the embedding matrix and the LM head.
Residual-stream width.
intermediate_size: usizeFeed-forward inner width.
num_layers: usizeNumber of decoder layers.
q_width: usizeOutput width of q_proj (num_attention_heads * head_dim).
kv_width: usizeOutput width of k_proj/v_proj (num_key_value_heads * head_dim).
Implementations§
Trait Implementations§
Source§impl Clone for DecoderShapes
impl Clone for DecoderShapes
impl Copy for DecoderShapes
Source§impl Debug for DecoderShapes
impl Debug for DecoderShapes
impl Eq for DecoderShapes
Source§impl PartialEq for DecoderShapes
impl PartialEq for DecoderShapes
impl StructuralPartialEq for DecoderShapes
Auto Trait Implementations§
impl Freeze for DecoderShapes
impl RefUnwindSafe for DecoderShapes
impl Send for DecoderShapes
impl Sync for DecoderShapes
impl Unpin for DecoderShapes
impl UnsafeUnpin for DecoderShapes
impl UnwindSafe for DecoderShapes
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§impl<Q, K> Equivalent<K> for Q
impl<Q, K> Equivalent<K> for Q
Source§fn equivalent(&self, key: &K) -> bool
fn equivalent(&self, key: &K) -> bool
Compare self to
key and return true if they are equal.Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more