Crate ggml

source ·
Expand description

ggml is a semi-idiomatic wrapper for the ggml C library.

It exposes a subset of operations (currently used to implement the llm library). Note that it does not expose a fully-idiomatic safe Rust interface; operations that could be potentially unsafe are marked as such.

ggml operates on a computational graph; no values will be computed until Context::graph_compute is executed. All Tensors are nodes in this computational graph, and values cannot be retrieved until computation is completed.

Modules

  • Loading and saving of GGML files.
  • Utilities for reading and writing.

Structs

  • A buffer of memory that can be used as a scratch buffer for a Context.
  • A ggml computation graph. Keeps track of all state during computation.
  • Acts as a RAII-guard over a sys::ggml_context, allocating via ggml_init and dropping via ggml_free.
  • Contains the result of a quantization operation.
  • Tensors are owned by the context. A tensor is alive as long as the underlying context it was created with is alive.

Enums

  • The format of the file containing the model.
  • The type of a value in ggml.

Constants

Functions

Type Definitions