Enum FileType

pub enum FileType {
    F32,
    MostlyF16,
    MostlyQ4_0,
    MostlyQ4_1,
    MostlyQ4_1SomeF16,
    MostlyQ4_2,
    MostlyQ8_0,
    MostlyQ5_0,
    MostlyQ5_1,
}

How the tensors are stored in GGML LLM models.

Variants

F32

All tensors are stored as f32.

MostlyF16

All tensors are mostly stored as f16, except for the 1D tensors (32-bit).

MostlyQ4_0

All tensors are mostly stored as Q4_0, except for the 1D tensors (32-bit).

MostlyQ4_1

All tensors are mostly stored as Q4_1, except for the 1D tensors (32-bit)

MostlyQ4_1SomeF16

All tensors are mostly stored as Q4_1, except for the 1D tensors (32-bit) and the tok_embeddings.weight (f16) and output.weight tensors (f16).

MostlyQ4_2

All tensors are mostly stored as Q4_2, except for the 1D tensors (32-bit).

MostlyQ8_0

All tensors are mostly stored as Q8_0, except for the 1D tensors (32-bit).

MostlyQ5_0

All tensors are mostly stored as Q5_0, except for the 1D tensors (32-bit).

MostlyQ5_1

All tensors are mostly stored as Q5_1, except for the 1D tensors (32-bit).

Trait Implementations

impl Clone for FileType

fn clone(&self) -> FileType

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for FileType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for FileType

fn default() -> FileType

Returns the “default value” for a type.

impl Display for FileType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<FileType> for i32

fn from(value: FileType) -> Self

Converts to this type from the input type.

impl PartialEq for FileType

fn eq(&self, other: &FileType) -> bool

Tests for self and other values to be equal, and is used by ==.

const fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl TryFrom<i32> for FileType

type Error = ()

The type returned in the event of a conversion error.

fn try_from(value: i32) -> Result<Self, Self::Error>

Performs the conversion.

impl Copy for FileType

impl Eq for FileType

impl StructuralPartialEq for FileType