Struct QuantizedEmbedding 

pub struct QuantizedEmbedding {
    pub weight: Array,
    pub scales: Array,
    pub biases: Array,
    pub group_size: usize,
    pub bits: u32,
}

An Embedding with its table quantized (MLX’s nn.QuantizedEmbedding): rows are gathered packed and dequantized; the packed table, scales and biases are frozen.

Fields

weight: Array

The packed table (uint32), [num_embeddings, dims · bits / 32].

scales: Array

One scale per group of values, [num_embeddings, dims / group_size].

biases: Array

One bias per group of values, shaped like scales.

group_size: usize

Values per scale and bias.

bits: u32

Bits per value.

Implementations

impl QuantizedEmbedding

pub fn from_embedding( embedding: &Embedding, group_size: usize, bits: u32, ) -> Result<Self>

Quantizes embedding’s table in groups of group_size values with bits bits.

pub fn forward(&self, x: &Array) -> Result<Array>

The rows for the ids in x, dequantized: shape x.shape + [dims].

pub fn as_linear(&self, x: &Array) -> Result<Array>

x Wᵀ with the quantized table (tied embeddings).

Trait Implementations

impl Clone for QuantizedEmbedding

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for QuantizedEmbedding

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Module for QuantizedEmbedding

fn visit(&mut self, p: &mut Parameters<'_>)

Reports every parameter and submodule to p, frozen ones included.

fn parameters(&self) -> Vec<(String, Array)>

Every parameter with its path (layers.0.weight), in visiting order.

fn trainable_parameters(&self) -> Vec<(String, Array)>

The parameters that are trained (not frozen), with their paths.

fn update_trainable(&mut self, values: &[Array]) -> Result<()>

Replaces the trainable parameters, given in the order of Module::trainable_parameters (as optimizers return them).

fn load_weights( &mut self, weights: impl IntoIterator<Item = (String, Array)>, strict: bool, ) -> Result<()>

Replaces parameters by path, as MLX’s load_weights: each weight must have its parameter’s shape (the dtype is kept as loaded). With strict, every parameter must be given and no unknown names are allowed.