Expand description
ggml
is a semi-idiomatic wrapper for the ggml
C library.
It exposes a subset of operations (currently used to implement the llm library). Note that it does not expose a fully-idiomatic safe Rust interface; operations that could be potentially unsafe are marked as such.
ggml
operates on a computational graph; no values will be computed until Context::graph_compute is executed.
All Tensors are nodes in this computational graph, and values cannot be retrieved until computation is completed.
Modules
- Loading and saving of GGML files.
- Utilities for reading and writing.
Structs
- A buffer of memory that can be used as a scratch buffer for a Context.
- A
ggml
computation graph. Keeps track of all state during computation. - Acts as a RAII-guard over a
sys::ggml_context
, allocating viaggml_init
and dropping viaggml_free
. - Contains the result of a quantization operation.
- Tensors are owned by the context. A tensor is alive as long as the underlying context it was created with is alive.
Enums
- The format of the file containing the model.
- The type of a value in
ggml
.
Constants
- Magic constant for
ggml
files (versioned, ggjt). - Magic constant for
ggml
files (versioned, ggmf). - Magic constant for
ggml
files (unversioned). - The currently-supported format version for
ggml
files. - The size of a
ggml
object.
Functions
- The size of a block for
t
. Only relevant for quantized types. - Quantizes
src
intodst
usingq4_0
quantization. - Quantizes
src
intodst
usingq4_1
quantization. - The size of
t
as bytes.
Type Definitions
- The type of a tensor element.