Trait ModelSource

Source

pub trait ModelSource: Send + Sync {
    // Required methods
    fn metadata(&self) -> &ModelMetadata;
    fn tensor_names(&self) -> Vec<String>;
    fn open_tensor(&self, name: &str) -> Result<TensorReader<'_>>;
    fn tokenizer(&self) -> Result<TokenizerSpec>;
    fn sampler_defaults(&self) -> Option<SamplerConfig>;

    // Provided method
    fn open_tensor_quant(&self, _name: &str) -> Result<Option<QuantTensor<'_>>> { ... }
}

Expand description

The central adapter trait: a source of model weights + config, independent of the on-disk format (LiteRT-LM ModelResources equivalent).

Implementations must be cheap to query for metadata and names, and lazy / zero-copy (e.g. mmap-backed) when opening tensors.

Required Methods§

Source

fn metadata(&self) -> &ModelMetadata

Architecture + hyperparameter metadata.

Source

fn tensor_names(&self) -> Vec<String>

Names of all tensors available in this source.

Source

fn open_tensor(&self, name: &str) -> Result<TensorReader<'_>>

Opens a tensor by name, returning a lazy reader over the raw bytes.

Source

fn tokenizer(&self) -> Result<TokenizerSpec>

Tokenizer specification (path to tokenizer.json + added tokens).

Source

fn sampler_defaults(&self) -> Option<SamplerConfig>

Sampler defaults from generation_config.json, if present.

Provided Methods§

Source

fn open_tensor_quant(&self, _name: &str) -> Result<Option<QuantTensor<'_>>>

Raw packed quantized bytes for a tensor, when this source stores it in a quant format that has a device kernel (GGUF Q4_0/Q4_K/Q6_K). None means “no packed representation” — the caller falls back to ModelSource::open_tensor, which dequantizes to float. Sources without packed formats keep this default.