llama-cpp-sys-4

Raw bindgen-generated bindings to llama.cpp, plus the C/C++ build logic that compiles the library.

llama.cpp version: 184f653602 (b11829, incl. v0.7.0) · Crate version: 0.9.0

Unless you need access to a symbol not yet exposed by llama-cpp-4, use that crate instead — it provides a safe API over these raw bindings.

For application code, start with:

use llama_cpp_4::prelude::*;

What's included

• llama_* functions and types from llama.h
• ggml_* functions and types from ggml/include/ggml.h
• LLAMA_* constants
• common_tokenize and common_token_to_piece from common/common.h
• The entire llama.cpp static library (or shared, with dynamic-link)

Feature flags

DFlash2

DFlash2 drafting (grouped dynamic depthwise convolution + candidate selector) merged upstream in #27342 (b10658) and ships in the vendored b11829, so it needs no build feature — the dflash2 feature and patches/0006-dflash2.patch that carried the pre-merge code are gone. Converting a checkpoint uses upstream's own convert_hf_to_gguf.py.

DFlash2 checkpoints are detected from GGUF metadata, so no separate speculative type is needed — select a DFlash draft via Eagle3Session::new_dflash in llama-cpp-4.

Prebuilt libraries

The exact speculative-state, decode-lifecycle, and fail-closed EAGLE-3 process patches require native libraries built from the same patched source. No published release archive carries a patch-identity envelope yet, so prebuilt currently emits a warning and uses the verified source build. LLAMA_PREBUILT_DIR fails closed while those patches are active rather than mixing patched headers with unverifiable native objects.

Once release assets carry an exact patch identity, the intended interface is:

# Automatic download + cache, when a compatible release asset is available
cargo build -p llama-cpp-sys-4 --features prebuilt

# Prefetch manually, then build
./scripts/fetch-prebuilt.sh
export LLAMA_PREBUILT_DIR=target/llama-prebuilt-cache/0.9.0/llama-prebuilt-...
cargo build -p llama-cpp-sys-4

Release assets are named llama-prebuilt-{linux|macos|windows}-{target}-{cpu|vulkan|blas|metal}-{static|dynamic}.tar.gz and published by .github/workflows/prebuilt-llama.yml on version tags.

cuda, hip, webgpu, opencl, and q1 have no published prebuilts yet — those builds always compile locally.

Building

The crate compiles llama.cpp from the vendored submodule at build time using cc + cmake-style flags. No external llama.cpp installation is required.

# CPU only (default)
cargo build -p llama-cpp-sys-4

# Metal (macOS)
cargo build -p llama-cpp-sys-4 --features metal

# CUDA
cargo build -p llama-cpp-sys-4 --features cuda

# OpenMPI (distributed inference)
brew install open-mpi   # or apt install libopenmpi-dev
cargo build -p llama-cpp-sys-4 --features mpi

Configuring llama.cpp's own CMake options

llama.cpp has dozens of LLAMA_* CMake options and gains more every release. Rather than mirror each one as a Cargo feature — which does not scale, and could not carry a value anyway — any LLAMA_* option can be set as data, from three places. Lowest precedence first:

1. llama-cpp.toml, found by walking up from this crate or from OUT_DIR, or named outright by LLAMA_CONFIG. Good for a machine-local or gitignored override:

[cmake]
LLAMA_OPENSSL = false      # or "OFF"; 0/1, on/off, true/false, yes/no all work
LLAMA_CURL = "OFF"

[cmake.target."aarch64-apple-darwin"]
LLAMA_CURL = "ON"          # per-target overrides win over the shared table

2. [package.metadata.llama-cpp-sys-4.cmake] in the consuming project's manifest (or [workspace.metadata.…], since a workspace root is often a virtual manifest). Good for a setting that belongs to the project and should be checked in:

[package.metadata.llama-cpp-sys-4.cmake]
LLAMA_OPENSSL = "OFF"

[package.metadata.llama-cpp-sys-4.cmake.target."x86_64-unknown-linux-gnu"]
LLAMA_SANITIZE_ADDRESS = "ON"

3. A LLAMA_* environment variable, which always wins, so CI and one-off builds can override anything:

LLAMA_OPENSSL=0 cargo build -p llama-cpp-sys-4

Notes:

• Only LLAMA_* is configurable. The backend options (GGML_METAL, GGML_CUDA, …) are driven by this crate's Cargo features, which also decide what the Rust side links against; enabling a backend behind the feature system's back would configure a library the link step does not expect. A GGML_* key in a config file is reported and skipped.
• The build script's own knobs (LLAMA_PATCH, LLAMA_PREBUILT_*, LLAMA_LIB_PROFILE, LLAMA_STATIC_CRT, LLAMA_NO_SCCACHE, LLAMA_BUILD_SHARED_LIBS, LLAMA_TEST_MODEL, …) are not forwarded.
• LLAMA_OPENSSL=0 is the motivating case: upstream defaults it ON, so any OpenSSL CMake can find gets linked into common — a second TLS stack that applications downloading models themselves never use, but must still ship and patch. With it off, an https:// request through common/http.h fails with an explicit "HTTPS is not supported" error rather than falling back to plain HTTP.
• Config files are tracked with rerun-if-changed, and every LLAMA_* variable that is set gets a rerun-if-env-changed. Exporting a LLAMA_* variable for the first time cannot be detected — Cargo has to be told the name in advance — so run cargo clean -p llama-cpp-sys-4 after that, or configure it in a file instead.

Build dependencies

• clang — required by bindgen to parse the C++ headers
• A C++17 compiler (GCC 9+, Clang 10+, MSVC 2019+)
• cmake and a supported CMake generator (Ninja is preferred)

Regenerating bindings

Bindings are regenerated automatically whenever build.rs or wrapper.h changes. The allowlist covers llama_*, ggml_*, LLAMA_*, and the two common_* functions.

# Force a full rebuild including binding regeneration
touch llama-cpp-sys-4/wrapper.h
cargo build -p llama-cpp-sys-4

Notable upstream API changes handled here

Breaks this crate absorbed as the vendored llama.cpp moved forward. The CHANGELOG.md entry for each release covers that release's window; this is the running summary.

Bindgen configuration

Key decisions in build.rs:

• derive_partialeq(true) with no_partialeq(...) overrides for structs containing function-pointer fields (avoids the unpredictable_function_pointer_comparisons lint).
• opaque_type("std::.*") — C++ STL types are opaque pointers.
• OpenMP auto-detection — reads GGML_OPENMP_ENABLED from the CMake cache rather than relying solely on the openmp feature flag, because some ARM toolchains enable OpenMP unconditionally.