pub enum RopeLayers {
All,
SlidingOnly,
NoRopeEvery {
step: NonZeroUsize,
phase: NoRopePhase,
},
}Expand description
llama.cpp’s per-layer use_rope, as a value a ModelConfig can
carry.
The default is RopeLayers::All and it is what every audited
architecture but four gets. A variant is only ever added by reading
a use_rope in src/models/.
Variants§
All
Every layer rotates: llama.cpp writes no gate at all.
SlidingOnly
Only the SLIDING-WINDOW layers rotate; the full-attention layers
get no rotation. exaone4 (when its SWA is on) and exaone-moe.
Note what this means when a model’s SWA pattern windows nothing
(llama.cpp’s degenerate set_swa_pattern(1)): NO layer rotates.
That is upstream’s answer too – is_swa(il) is false everywhere
while swa_type is still STANDARD – and it is why this is not
written as “sliding layers, or everything if nothing slides”.
NoRopeEvery
One layer in every step does not rotate,
hparams.n_no_rope_layer_step.
Fields
step: NonZeroUsizellama.cpp’s n_no_rope_layer_step. Never zero here: the two
architectures that guard on step > 0 inherit the
llama-hparams.h:203 default of 4 and never assign it.
phase: NoRopePhaseWhich layer of the period is skipped.
Implementations§
Source§impl RopeLayers
impl RopeLayers
Sourcepub fn rotates(self, layer_idx: usize, layer_slides: bool) -> bool
pub fn rotates(self, layer_idx: usize, layer_slides: bool) -> bool
Does layer layer_idx rotate?
layer_slides is ModelConfig::layer_sliding_window(il).is_some(),
i.e. llama.cpp’s hparams.is_swa(il). It is a parameter rather
than something this enum works out, because the SWA layout is
the ModelConfig’s answer and restating it here would be two
structures that must agree about one thing.
Sourcepub fn any_layer_unrotated(
self,
n_layers: usize,
slides: impl Fn(usize) -> bool,
) -> bool
pub fn any_layer_unrotated( self, n_layers: usize, slides: impl Fn(usize) -> bool, ) -> bool
True when this rule leaves at least one layer of an n_layers
model unrotated, i.e. when it is anything other than “rotate
everything” IN PRACTICE rather than in name.
The distinction matters: NoRopeEvery { step: 64, .. } on a
30-layer model is Self::All by any observable test, and a
caller asking “must I implement this?” wants the observable
answer.
Trait Implementations§
Source§impl Clone for RopeLayers
impl Clone for RopeLayers
Source§fn clone(&self) -> RopeLayers
fn clone(&self) -> RopeLayers
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreimpl Copy for RopeLayers
Source§impl Debug for RopeLayers
impl Debug for RopeLayers
Source§impl Default for RopeLayers
impl Default for RopeLayers
Source§fn default() -> RopeLayers
fn default() -> RopeLayers
impl Eq for RopeLayers
Source§impl PartialEq for RopeLayers
impl PartialEq for RopeLayers
impl StructuralPartialEq for RopeLayers
Auto Trait Implementations§
impl Freeze for RopeLayers
impl RefUnwindSafe for RopeLayers
impl Send for RopeLayers
impl Sync for RopeLayers
impl Unpin for RopeLayers
impl UnsafeUnpin for RopeLayers
impl UnwindSafe for RopeLayers
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more