Crate ggml

ggml is a semi-idiomatic wrapper for the ggml C library.

It exposes a subset of operations (currently used to implement the llm library). Note that it does not expose a fully-idiomatic safe Rust interface; operations that could be potentially unsafe are marked as such.

ggml operates on a computational graph; no values will be computed until Context::graph_compute is executed. All Tensors are nodes in this computational graph, and values cannot be retrieved until computation is completed.

Modules

• format
  Loading and saving of GGML files.
• util
  Utilities for reading and writing.

Structs

• Buffer
  A buffer of memory that can be used as a scratch buffer for a Context.
• ComputationGraph
  A ggml computation graph. Keeps track of all state during computation.
• Context
  Acts as a RAII-guard over a sys::ggml_context, allocating via ggml_init and dropping via ggml_free.
• QuantizationResult
  Contains the result of a quantization operation.
• Tensor
  Tensors are owned by the context. A tensor is alive as long as the underlying context it was created with is alive.

Enums

• ContainerType
  The format of the file containing the model.
• Type
  The type of a value in ggml.

Constants

• FILE_MAGIC_GGJT
  Magic constant for ggml files (versioned, ggjt).
• FILE_MAGIC_GGMF
  Magic constant for ggml files (versioned, ggmf).
• FILE_MAGIC_UNVERSIONED
  Magic constant for ggml files (unversioned).
• FORMAT_VERSION
  The currently-supported format version for ggml files.
• OBJECT_SIZE
  The size of a ggml object.

Functions

• blck_size
  The size of a block for t. Only relevant for quantized types.
• quantize_q4_0
  Quantizes src into dst using q4_0 quantization.
• quantize_q4_1
  Quantizes src into dst using q4_1 quantization.
• type_size
  The size of t as bytes.
• type_sizef
  type_size/blck_size as float.

Type Definitions

• ElementType
  The type of a tensor element.