pub enum LayerShapes {
Uniform,
PerLayer(Vec<LayerShape>),
}Expand description
Every layer’s shape, or the statement that they all agree.
Uniform is not PerLayer(vec![same; n]): the fused Metal stacks,
the CUDA resident KV and the slot-file format each hold ONE
geometry, and “is this model uniform” is a question they ask
through Self::is_uniform rather than by comparing entries.
Variants§
Uniform
Every layer is ModelConfig::{n_heads, n_kv_heads, moe.expert_ffn_dim}.
PerLayer(Vec<LayerShape>)
One entry per layer, at least two of which differ.
Implementations§
Source§impl LayerShapes
impl LayerShapes
pub fn is_uniform(&self) -> bool
Sourcepub fn replicated(self, n_loops: usize) -> Self
pub fn replicated(self, n_loops: usize) -> Self
The same shapes for n_loops passes over the layers, as
nanbeige.cpp:24-26 copies each physical layer’s arrays to every
logical slot (crate::layer_loops). Uniform stays uniform.
Sourcepub fn resolve(
arch: &str,
heads: &[u64],
kv_heads: &[u64],
ffn: Option<&[u64]>,
expert_ffn_dim: usize,
recurrent: Option<&RecurrentMask>,
) -> Result<Self, LoadError>
pub fn resolve( arch: &str, heads: &[u64], kv_heads: &[u64], ffn: Option<&[u64]>, expert_ffn_dim: usize, recurrent: Option<&RecurrentMask>, ) -> Result<Self, LoadError>
Builds the table from the three per-layer arrays, collapsing to
Uniform when nothing varies and refusing a varying file for an
architecture llama.cpp itself reads at layer 0.
ffn may be absent (a MoE file that declares only
expert_feed_forward_length); then every layer takes
expert_ffn_dim.
recurrent is crate::gdn::recurrent_layers’ answer: the layers
that run a recurrent block AND which block, on an architecture
whose head counts are uniform and say nothing about either.
Trait Implementations§
Source§impl Clone for LayerShapes
impl Clone for LayerShapes
Source§impl Debug for LayerShapes
impl Debug for LayerShapes
Source§impl Default for LayerShapes
impl Default for LayerShapes
impl Eq for LayerShapes
Source§impl PartialEq for LayerShapes
impl PartialEq for LayerShapes
impl StructuralPartialEq for LayerShapes
Auto Trait Implementations§
impl Freeze for LayerShapes
impl RefUnwindSafe for LayerShapes
impl Send for LayerShapes
impl Sync for LayerShapes
impl Unpin for LayerShapes
impl UnsafeUnpin for LayerShapes
impl UnwindSafe for LayerShapes
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more