Expand description
§logit-loom-llamacpp
The safe llama.cpp adapter for Logit Loom.
It maps llama.cpp candidate logits and admitted tokens into the transform and
observer boundaries supplied by logit-loom. It also exposes causal prefill,
compatibility-bound checkpoints, topology-bound activation capture and
transactions, target-authoritative MTP/EAGLE-3 generation, and scoped LoRA
and control vectors. Session::generate_v2 additionally executes
versioned eager or bounded trigger-driven lazy grammar plans without changing
the original generation-plan identity.
§Select a backend
The crate enables no llama.cpp backend feature by default. Select the feature for the deployment explicitly:
[dependencies]
logit-loom = "=0.2.0"
logit-loom-llamacpp = { version = "=0.2.0", features = ["vulkan"] }ModelOptions::default requires accelerator participation and does not retry a
rejected load as CPU-only inference. Use DevicePolicy::Any only when that
fallback is intentional.
§Example
The adapter never downloads a model. Supply a local GGUF explicitly:
use logit_loom::{GenerationPlan, SamplingPlan};
use logit_loom_llamacpp::{
Model, ModelOptions, Runtime, SessionOptions, Tokenization,
};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let model_path = std::path::Path::new("model.gguf");
let mut runtime = Runtime::initialize()?;
runtime.silence_native_logs();
let model = Model::load(&runtime, model_path, ModelOptions::default())?;
let prompt = model.tokenize("Hello", Tokenization { add_bos: true })?;
let mut session = model.session(&runtime, SessionOptions::default())?;
session.prefill(&prompt, true)?;
let output = session.generate(
&GenerationPlan {
sampling: SamplingPlan::default(),
max_tokens: 16,
biases: Vec::new(),
grammar: None,
stops: Vec::new(),
},
None,
None,
)?;
let _exact_bytes = output.bytes;
Ok(())
}Local resource authorities may avoid whole-file digest passes for immutable
sealed artifacts. Model::load_preverified accepts a caller-established
BLAKE3 identity, while Model::load_authorized accepts a provider-owned stable
generation identity and exact length without hashing model payload bytes.
Model::load_lora_authorized provides the same provider-owned boundary for an
already authenticated and sealed LoRA. Mutable and ordinary filesystem paths
continue to use Model::load and Model::load_lora, which verify content
before and after native loading.
Generated pieces remain arbitrary bytes. Call GenerationOutput::text only
when the complete output is expected to be valid UTF-8.
Model::tokenize rejects NUL bytes and inputs larger than
MAX_TOKENIZATION_BYTES before calling the native binding.
§Structured projection
StructuredProjectionController captures one exact prefilled causal boundary,
builds a content-identified table of every tokenizer ID, exact piece bytes, and
EOG classification, and runs eager or lazy native grammar together with
caller-identified exact-byte feedback. An installed authoritative validator
then either selects the complete output or restores the boundary.
attempt performs exactly one attempt. It never retries internally.
Cancellation, validation rejection, feedback errors, validator errors, and
contained callback panics restore the common checkpoint when native state
remains recoverable. The caller may then explicitly call attempt again
within the plan bound. Compiler and validator semantics remain outside Logit
Loom; their exact implementation/configuration identities and content-free
receipts are inside the public mechanical contract.
§Aggregate execution
execute_text_mechanics is the complete TextMechanicsPlanV2 lowering. A
TextMechanicsRequest supplies exact prompt tokens, target/draft context
geometry, installed activation runtimes, borrowed transform and observer
sets, loaded target LoRAs, an optional target control vector, and optional
process-local parent state. Every identity and bound is checked before context
allocation.
The ordinary and MTP/EAGLE-3 paths share one lifecycle: apply ordered target
steering, admit bounded prompt chunks, generate under the selected activation
and callback mechanics, explicitly clear steering in reverse order, then
optionally capture a quiescent checkpoint. A PrefillMonitor may stop either
fresh path at a complete synchronized chunk boundary. That terminal result
contains prefill and cleanup evidence, publishes neither generation nor a
partial checkpoint, and performs no retry or fallback.
Ordinary aggregate checkpoints use
TextMechanicsCheckpointReceiptV2 and retain the opaque native sampler plus
cross-operation stop prefix; speculative snapshots retain their complete
aggregate plan and native sampler. Both forms are process-local and
thread-affine. A continuation must preserve every mechanic apart from its
explicit parent-checkpoint field. Because native state omits the next-token
logits, exact steering is reapplied while reconstructing the restored boundary
and cleared again before successor capture.
§Activation and speculation
ActivationConfiguration validates exact tensor sites, bounded capture plans,
ordered operators, sparse vector banks, and the loaded model topology before
allocating a context. Graph-node selectors are bound to the exact adapter,
binding, llama.cpp revision, and architecture profile. A selected tensor is
copied into owned storage; one complete finite result is written back only
after every declared operator succeeds.
generate_speculative drives either same-model MTP or a separately loaded
EAGLE-3 draft. The supplied SpeculationPlanV1 must name
speculation_implementation_identity(), both exact model/topology identities,
one supported mechanism, and an explicit activation policy. Before allocating
contexts, the adapter also applies llama.cpp HEAD’s vocabulary-compatibility
rules, checks the MTP hidden-row width, and requires an eagle3 draft with
exactly three in-range target extraction layers:
use logit_loom::{
GenerationPlan, SamplingPlan, SpeculationActivationPolicyV1,
SpeculationPlanV1, TextSpeculativeMechanismV1,
};
use logit_loom_llamacpp::{
Model, ModelOptions, Runtime, SpeculativeRequest, Tokenization,
generate_speculative, speculation_implementation_identity,
};
fn main() -> Result<(), Box<dyn std::error::Error>> {
let runtime = Runtime::initialize()?;
let model = Model::load(&runtime, "mtp-model.gguf", ModelOptions::default())?;
let prompt = model.tokenize("A small mechanical experiment:", Tokenization {
add_bos: true,
})?;
let generation = GenerationPlan {
sampling: SamplingPlan::default(),
max_tokens: 16,
biases: Vec::new(),
grammar: None,
stops: Vec::new(),
};
let topology = model.topology().digest()?;
let speculation = SpeculationPlanV1 {
target_model: model.artifact_digest().clone(),
target_topology: topology.clone(),
draft_model: model.artifact_digest().clone(),
draft_topology: topology,
implementation: speculation_implementation_identity(),
mechanism: TextSpeculativeMechanismV1::Mtp,
sequences: 1,
maximum_draft_tokens: 4,
minimum_draft_tokens: 0,
probability_floor_bits: 0.0_f32.to_bits(),
activation: SpeculationActivationPolicyV1::None,
};
let output = generate_speculative(
&runtime,
&model,
&model,
SpeculativeRequest::new(&prompt, &generation, &speculation),
)?;
let _exact_bytes = output.generation.bytes;
Ok(())
}The target sampler verifies the longest proposal prefix. Rejected proposals and end-of-generation selections never reach token observers. Verification captures remain provisional until rollback and native acceptance establish their admitted/rejected positions. The current high-level operation supports one sequence and requires explicit context headroom; larger sequence plans are rejected before allocation rather than silently lowered.
The runnable speculative_mtp example takes one compatible caller-supplied
GGUF and writes exact generated bytes:
cargo run -p logit-loom-llamacpp --example speculative_mtp \
--features vulkan -- /path/to/mtp-model.gguf "Draft from here:"generate_speculative_checkpointed captures exact target, draft, and
MTP/EAGLE-3 implementation state at a quiescent boundary together with an
opaque target-sampler clone, activation configuration, cross-operation stop
state, and causal lineage. resume_speculative_checkpointed validates the
complete parent before allocation, clones the sampler for an independent
branch, restores contexts and implementation state, and captures the next
boundary. Target steering selected through SpeculativeRequest or
SpeculativeContinuationRequest spans the native operation and is cleared
before that boundary is captured.
The checkpoint object is process-local and thread-affine because llama.cpp
provides no portable target-sampler encoding. Its
SpeculativeCheckpointReceiptV1 is serializable mechanical evidence, but the
receipt alone cannot reconstruct the native sampler. No partial portable
checkpoint is advertised.
§Coordinator integration
Runtime::native_backend and Model::native_model provide read-only borrows
for an in-process coordinator that must compose other llama.cpp mechanics,
such as multimodal projection or embeddings, with the same backend and loaded
model. Logit Loom retains backend and model lifecycle ownership. Calls made
through those handles are outside Logit Loom’s plan and receipt surface; a
coordinator must give those calls their own bounded contract and evidence.
Checkpoints bind the model bytes, exact safe-binding source, literal llama.cpp
revision, adapter build identity, and exact session allocation options; native
state is opaque and is not a portable interchange format. A failed automatic
steering cleanup or partial checkpoint restore poisons the session, and
subsequent mutation returns Error::Poisoned.
Applications may persist StateSnapshot::into_parts in their own container
format and reconstruct it with StateSnapshot::from_parts. Reconstruction
validates internal byte and token-lineage identities; restore additionally
requires the original model and compatible backend build. Because the pinned
llama.cpp state format omits next-token logits, restore re-decodes the final
recorded token at its exact position after restoring causal memory. A backend
that cannot remove that one position is rejected, and a failed refresh poisons
the session.
See the compatibility policy and capability status before selecting features or interpreting validation results.
Structs§
- Activation
Capture Output - Captured records and per-plan aggregate receipts drained from a session.
- Activation
Configuration - Validated activation program, vector banks, captures, and graph profile.
- Activation
Program Output - Transaction records and aggregate program receipt drained from a session.
- Authorized
Lora Artifact - Provider-owned identity and exact length for an immutable
LoRAartifact. - Authorized
Model Artifact - Provider-owned identity and exact length for an immutable model artifact.
- Byte
Feedback Candidate - One exact model-tokenizer candidate presented to byte feedback.
- Byte
Feedback Error - Bounded implementation error from exact-byte feedback.
- Control
Vector - In-memory native control-vector data and layer contract.
- Control
Vector Scope - Active reversible control-vector application.
- Generation
Output - Successful bounded generation output.
- Installed
Byte Feedback - Installed byte-feedback implementation bound to its public identity.
- Installed
Structured Validator - Installed validator implementation bound to its public identity.
- Llama
CppTensor Profile - Explicit llama.cpp graph-site compatibility profile.
- Lora
Adapter - Loaded model-compatible
LoRAadapter. - Lora
Application - One borrowed
LoRAand its exact application scale. - Lora
Scope - Active reversible
LoRAapplication. - Model
- Immutable loaded model with content and placement identity.
- Model
Options - Model-loading options exposed by the first adapter release.
- Ordinary
Text Mechanics Snapshot - Opaque ordinary causal state plus its complete aggregate mechanics.
- Prefill
Output - Result of one plain or controlled prefill call.
- Preverified
Model Artifact - Exact content evidence for an immutable model artifact verified by its caller.
- Runtime
- Process-wide initialized llama.cpp backend.
- Session
- Single-owner llama.cpp context with explicit causal lineage.
- Session
Options - Context-allocation options.
- Speculative
Activation Output - Activation evidence produced by one context in a speculative operation.
- Speculative
Activations - Explicit activation runtimes supplied to one speculative operation.
- Speculative
Checkpoint Output - Successful generation plus a reusable quiescent continuation checkpoint.
- Speculative
Checkpoint Request - Aggregate-plan lineage selected for one quiescent speculative checkpoint.
- Speculative
Continuation Request - Borrowed mechanics for continuation from a speculative checkpoint.
- Speculative
Generation Output - Complete successful output of one target-authoritative operation.
- Speculative
Request - Borrowed mechanics and owned runtime configuration for one operation.
- Speculative
Session Options - Target and draft context-allocation options for one speculative operation.
- Speculative
State Snapshot - Opaque process-local continuation state for one quiescent speculative boundary.
- State
Snapshot - Opaque llama.cpp causal state with compatibility-bound identity metadata.
- Steering
Scope - Active aggregate steering application.
- Structured
Attempt Output - Runtime output from one explicit projection attempt.
- Structured
Projection Controller - Single-owner controller for exact rejection recovery and explicit retry.
- Structured
Validation Context - Read-only authoritative validation input.
- Structured
Validation Outcome - Authoritative mechanical validation output before evidence bytes are reduced to digests.
- Structured
Validator Error - Bounded implementation error from an authoritative validator.
- Text
Mechanics Output - Successful whole-operation output and content-free terminal evidence.
- Text
Mechanics Request - Borrowed callbacks/resources and owned runtime configuration for one aggregate text operation.
- Tokenization
- Tokenization flags for exact text admission.
Enums§
- Device
Policy - Runtime placement requirement checked after model loading.
- Error
- Failures from native execution, callback boundaries, or identity checks.
- Text
Mechanics Checkpoint - Opaque causal state captured at a successful aggregate boundary.
- Text
Mechanics Execution Output - Mechanic-specific output retained beside the aggregate receipt.
- Text
Mechanics Resume - Exact causal state selected as the parent of one aggregate operation.
Constants§
- LLAMA_
CPP_ BINDING_ SOURCE_ REVISION - Exact source revision of the reviewed native Rust binding successor.
- LLAMA_
CPP_ BINDING_ VERSION - Exact native Rust binding version used by this adapter release.
- LLAMA_
CPP_ REVISION - Exact llama.cpp revision carried by the ADR 0002 binding successor.
- MAX_
STRUCTURED_ TOKENIZER_ TABLE_ BYTES - Maximum aggregate token-piece bytes retained by a prepared tokenizer table.
- MAX_
STRUCTURED_ VALIDATION_ EVIDENCE_ BYTES - Maximum canonical artifact or validation-report bytes hashed per attempt.
- MAX_
TOKENIZATION_ BYTES - Maximum UTF-8 bytes accepted by one native tokenization call.
Traits§
- Byte
Feedback - Stateful exact-byte candidate feedback for one structured attempt.
- Structured
Validator - Caller-supplied authoritative parser and constraint validator.
Functions§
- execute_
text_ mechanics - Executes one complete topology-bound text-mechanics plan.
- generate_
speculative - Runs one bounded MTP or EAGLE-3 operation.
- generate_
speculative_ checkpointed - Runs one bounded speculative operation and captures complete quiescent continuation state.
- resume_
speculative - Continues one compatible quiescent checkpoint without capturing a successor boundary.
- resume_
speculative_ checkpointed - Continues one compatible quiescent checkpoint and captures the next quiescent boundary.
- speculation_
implementation_ identity - Returns the exact native and safe-wrapper implementation identity.