pub struct BankLocation {
pub path: PathBuf,
pub offset: u64,
pub num_experts: usize,
pub bytes_per_expert: usize,
}Expand description
Where an expert bank’s bytes live, and how they are cut into experts.
bytes_per_expert is the bank’s byte length divided by its expert count.
That is exact for the block-quantized formats this targets provided each
expert’s rows are a whole number of quant blocks — true for GGUF K-quants,
whose block is 256 elements and whose in_dim is a multiple of it.
ExpertPager::register checks it rather than trusting it, because getting
it wrong reads a misaligned window and silently produces garbage weights.
Fields§
§path: PathBufCheckpoint file holding the bank.
offset: u64Absolute byte offset of expert 0 within that file.
num_experts: usize§bytes_per_expert: usizeImplementations§
Source§impl BankLocation
impl BankLocation
Sourcepub fn total_bytes(&self) -> u64
pub fn total_bytes(&self) -> u64
Total bytes of the whole bank.
Trait Implementations§
Source§impl Clone for BankLocation
impl Clone for BankLocation
Source§fn clone(&self) -> BankLocation
fn clone(&self) -> BankLocation
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreAuto Trait Implementations§
impl Freeze for BankLocation
impl RefUnwindSafe for BankLocation
impl Send for BankLocation
impl Sync for BankLocation
impl Unpin for BankLocation
impl UnsafeUnpin for BankLocation
impl UnwindSafe for BankLocation
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
Converts
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more