Module ggml 

Safe wrappers around core ggml graph computation APIs.

This module provides the building blocks for creating and executing tensor computation graphs using ggml backends. It’s used for operations like LoRA merging, importance matrix computation, and control vector generation.

Example

use llama_cpp_4::ggml::*;

// Create a backend and context
let backend = GgmlBackend::cpu()?;
let mut ctx = GgmlContext::new(1024 * 1024, true)?;

// Create tensors
let a = ctx.new_tensor_2d(GgmlType::F32, 4, 4);
let b = ctx.new_tensor_2d(GgmlType::F32, 4, 4);

// Build computation graph
let sum = ctx.add(&a, &b);
let mut graph = ctx.new_graph();
graph.build_forward(&sum);

// Allocate and compute
let mut alloc = GgmlAllocr::new(&backend);
alloc.alloc_graph(&mut graph);
// ... set tensor data ...
backend.graph_compute(&mut graph);
// ... get results ...

Structs

GgmlAllocr

A safe wrapper around ggml_gallocr.

GgmlBackend

A safe wrapper around ggml_backend.

GgmlContext

A safe wrapper around ggml_context.

GgmlGraph

A wrapper around ggml_cgraph.

GgmlTensor

A wrapper around a ggml_tensor pointer.

Functions

buffer_free^⚠

Free a backend buffer.

graph_overhead

Get the overhead in bytes for a computation graph.

is_quantized

Check if a type is quantized.

tensor_get^⚠

Get tensor data into a byte slice.

tensor_overhead

Get the overhead in bytes for tensor metadata.

tensor_set^⚠

Set tensor data from a byte slice.

type_name

Get the name of a ggml type.

Type Aliases

ggml_type

Re-export the raw ggml types for advanced usage.