offgrid-sys 0.2.1

Low-level FFI bindings to llama.cpp (Vulkan/CPU), built from source.
docs.rs failed to build offgrid-sys-0.2.1
Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.

offgrid-sys

Raw bindgen bindings to llama.cpp and mtmd, built from the pinned llama.cpp submodule (always in CMake Release). The safe wrapper is the offgrid crate.

On Windows laptops with an AMD iGPU plus another GPU, AMD's switchable-graphics Vulkan layer hangs vkCreateInstance. offgrid sets DISABLE_LAYER_AMD_SWITCHABLE_GRAPHICS_1=1 before initializing; code calling llama_backend_init directly should do the same.

Features

Feature Default Effect
vulkan yes Enables the GGML Vulkan backend (GGML_VULKAN=ON) and links vulkan.
cuda no Enables the GGML CUDA backend (GGML_CUDA=ON). Requires a CUDA toolkit.
bindgen no Regenerate the bindings with bindgen (needs libclang) instead of using the committed src/bindings/{linux,windows}.rs; required on other targets.
dynamic-backends no Shared libs with runtime-loaded backends and all CPU variants (used by the packages).

CPU-only builds: cargo build -p offgrid-sys --no-default-features.

Environment variables

Variable Effect
OFFGRID_NATIVE=1 Sets GGML_NATIVE=ON (tune the CPU backend for the host CPU). Off by default so artifacts are portable.
OFFGRID_TEST_MODEL GGUF model for the tokenize_with_model smoke test (skipped when unset).

Tests

# Device enumeration only
cargo test -p offgrid-sys -- --nocapture

# With model tokenization
OFFGRID_TEST_MODEL=/path/to/model.gguf cargo test -p offgrid-sys -- --nocapture

# CPU-only build (no Vulkan)
cargo build -p offgrid-sys --no-default-features

Bumping the pinned llama.cpp commit

cd crates/offgrid-sys/llama.cpp
git fetch --depth 1 origin <new-commit-sha>
git checkout <new-commit-sha>
cd ../../..
scripts/gen-bindings.sh            # regenerate src/bindings/{linux,windows}.rs
git add crates/offgrid-sys/llama.cpp crates/offgrid-sys/src/bindings
cargo clean -p offgrid-sys && cargo build -p offgrid-sys