Enum WeightBytes 

pub enum WeightBytes {
    Owned(Vec<u8>),
    Mapped {
        mmap: Arc<Mmap>,
        range: Range<usize>,
    },
    Shared {
        buf: Arc<Vec<u8>>,
        range: Range<usize>,
    },
}

Backing storage for a quantized weight matrix’s raw bytes: either an owned buffer (synthetic/test weights, or any tensor that had to be copied for some other reason) or a zero-copy view into a shared memory-mapped GGUF file. This is the fix for the “loader read everything into a fresh Vec” inefficiency: a real checkpoint’s resident memory should be the mmap itself, not a second copy of it, which is how llama.cpp’s mmap-based loader both avoid doubling a multi-hundred-gigabyte checkpoint’s memory footprint.

Variants

Owned(Vec<u8>)

Mapped

Fields

mmap: Arc<Mmap>

range: Range<usize>

Shared

A sub-range of a shared, lease-style buffer (e.g. one matrix inside an ferrox_core::expert_store::ExpertLease’s combined gate/up/down bytes). Holding the Arc here is exactly what makes the store’s lease pinning structural: as long as any WeightMatrix built over these bytes is alive, the cache entry’s strong count stays >1 and eviction cannot reuse it.

Fields

buf: Arc<Vec<u8>>

range: Range<usize>

Implementations

impl WeightBytes

pub fn as_slice(&self) -> &[u8]

pub fn len(&self) -> usize

pub fn is_empty(&self) -> bool

pub fn address_is_stable(&self) -> bool

May a repack cache key on this buffer’s ADDRESS?

Only for Mapped, whose address is a fixed point in a process-lifetime mmap. Shared is a lease over an expert store’s recycled buffer: two different experts routinely land at the same address, and an address-keyed cache then serves one of them the other’s repacked bytes. That produced fluent garbage on OLMoE with expert streaming on, while the raw weight bytes compared equal, because the corruption was in the CACHE and not in the weights.

Owned is excluded for the same reason: a freed Vec’s address can be reused.

pub fn is_mapped(&self) -> bool

True if this is a zero-copy mmap view rather than an owned heap allocation – useful for tests/diagnostics asserting that the loader actually took the zero-copy path.