skippy-ffi

Raw Rust declarations for the experimental skippy C ABI.

This crate should stay thin: no topology, protocol, benchmark, or product behavior belongs here. Prefer adding safe behavior in skippy-runtime.

Responsibilities

• bind to skippy.h
• expose C-compatible constants, structs, and function declarations
• keep ABI ownership and error shapes explicit
• link the staged llama.cpp and ggml static archives selected by LLAMA_STAGE_BUILD_DIR

Consumers should use skippy-runtime unless they are extending the ABI boundary itself.

Architecture Role

skippy-ffi is the lowest Rust layer in the staged stack. It exposes the patched llama.cpp ABI without owning policy:

flowchart TB
    Mesh["mesh-llm<br/>embedded runtime owner"] --> Runtime
    Server["skippy-server<br/>lifecycle + transport"] --> Runtime["skippy-runtime<br/>safe model/session API"]
    Runtime --> FFI["skippy-ffi<br/>raw ABI declarations"]
    FFI --> Llama["third_party/llama.cpp<br/>skippy.h stage execution"]

Keep this crate close to the C ABI. Higher-level behavior such as topology validation, activation wire conversion, exact-cache policy, package materialization, telemetry, and benchmark policy belongs in the crates above it.

Build Integration

By default the build script statically links the staged llama.cpp checkout from .deps/llama.cpp/build-stage-abi-static. Prepare and build it with just llama-build. Set LLAMA_STAGE_BUILD_DIR to point at another prepared build directory, for example a Linux GPU backend build:

The backend archives are optional. The build script links each one only when it exists in LLAMA_STAGE_BUILD_DIR, so CPU-only and accelerator builds share the same Rust crate.

ABI Contract

The staged ABI is versioned as 0.1.31. The patch header in third_party/llama.cpp/patches/ and the Rust constants in crates/skippy-ffi/src/lib.rs are the source of truth, so keep this README aligned with those files instead of treating it as canonical prose.

Version 0 is still experimental, so callers should treat the ABI as feature-probed rather than permanently stable.

The runtime-event additions are part of that 0.1.26 bump. They add versioned SkippyRuntimeEventV1 and SkippyRuntimeEventReporterV1 structs, keep detail_len fixed-width as u64, and pass the reporter as a separate pointer argument on the _with_events model-open entrypoints instead of extending RuntimeConfig.

Version 0.1.31 adds a request-owned adapter for llama.cpp's upstream ngram-cache proposer. Callers reset or append only target-committed tokens; an optional provisional continuation is read-only input to drafting.

The Rust FFI layer binds skippy_abi_features. This README records ABI intent and compatibility expectations only; higher-level gating belongs in skippy-runtime or later tasks.

All skippy_* functions either return skippy_status or, for ABI discovery/error helpers, return plain values directly. Functions that accept out_error may allocate a skippy_error; callers must pass it to skippy_error_free after reading the status/message.

Opaque handles are caller-owned after successful creation:

Buffer-writing functions use the usual C ABI sizing pattern: the caller passes a pointer, a capacity, and an output byte/count pointer. If the buffer is too small, the function returns BUFFER_TOO_SMALL and reports the required size in the output pointer when the implementation can compute it.

Protocol Shape

The same ABI supports single-stage inference, split runtime stages, state movement, tokenizer/chat helpers, and model slicing:

sequenceDiagram
    participant R as Rust runtime
    participant F as skippy-ffi
    participant L as patched llama.cpp
    R->>F: skippy_model_open(config)
    F->>L: open model or stage slice
    R->>F: skippy_session_create(model)
    R->>F: prefill/decode/verify
    alt middle or final stage
        R->>F: import activation frame or KV/recurrent state
    end
    F-->>R: logits, predicted token, activation frame, or state bytes
    R->>F: skippy_session_free(session)
    R->>F: skippy_model_free(model)

Feature Flags

skippy_abi_features returns a bitmask of the capabilities compiled into the patched llama.cpp library. The Rust FFI layer binds this function, and skippy-runtime uses it together with ABI and symbol checks to decide whether _with_events should be selected. Callers that want feature probing can still read it directly:

Runtime-event compatibility expectations are narrow on purpose:

• skippy-runtime also requires ABI/version checks before selecting the _with_events entrypoints; failing those checks keeps the legacy no-events path in charge.
• Event callbacks stay operation-scoped, and v1 guarantees no callback after the _with_events entrypoint returns.

Function Surface

This section maps every C function in the stage ABI, plus the upstream llama log hook currently bound by this crate.

Upstream llama hook

ABI discovery and errors

Model and session lifecycle

Model open from parts

Execution

Self-speculative proposal

Token and chat helpers

Model introspection and GGUF writing