Crate ggml 

ggml is a semi-idiomatic wrapper for the ggml C library.

It exposes a subset of operations (currently used to implement the llm library). Note that it does not expose a fully-idiomatic safe Rust interface; operations that could be potentially unsafe are marked as such.

ggml operates on a computational graph; no values will be computed until Context::graph_compute is executed. All Tensors are nodes in this computational graph, and values cannot be retrieved until computation is completed.

Modules

format

Loading and saving of GGML files.

util

Utilities for reading and writing.

Structs

Buffer

A buffer of memory that can be used as a scratch buffer for a Context.

ComputationGraph

A ggml computation graph. Keeps track of all state during computation.

Context

Acts as a RAII-guard over a sys::ggml_context, allocating via ggml_init and dropping via ggml_free.

QuantizationResult

Contains the result of a quantization operation.

Tensor

Tensors are owned by the context. A tensor is alive as long as the underlying context it was created with is alive.

Enums

ContainerType

The format of the file containing the model.

Type

The type of a value in ggml.

Constants

FILE_MAGIC_GGJT

Magic constant for ggml files (versioned, ggjt).

FILE_MAGIC_GGMF

Magic constant for ggml files (versioned, ggmf).

FILE_MAGIC_UNVERSIONED

Magic constant for ggml files (unversioned).

FORMAT_VERSION

The currently-supported format version for ggml files.

OBJECT_SIZE

The size of a ggml object.

Functions

blck_size

The size of a block for t. Only relevant for quantized types.

quantize_q4_0

Quantizes src into dst using q4_0 quantization.

quantize_q4_1

Quantizes src into dst using q4_1 quantization.

type_size

The size of t as bytes.

type_sizef

type_size/blck_size as float.

Type Aliases

ElementType

The type of a tensor element.