Expand description
§logit-loom-runtime
An explicit higher-level local runtime for Logit Loom’s llama.cpp adapter.
The crate shortens model loading, exact text admission, bounded generation, transform and observer construction, checkpoint branching, and steering scopes. It does not download models, choose a chat template, silently fall back to CPU, assume generated token pieces are UTF-8, or make claims about output quality.
Use this crate for a direct local llama.cpp workflow. Use logit-loom when
building backend-neutral mechanics, or logit-loom-llamacpp when an
application needs to own the native runtime and model objects separately.
§Select a backend
No native backend feature is enabled by default. Select the deployment feature explicitly:
[dependencies]
logit-loom-runtime = { version = "=0.2.0", features = ["vulkan"] }LoomOptions::default continues to require accelerator participation. Native
logs are preserved unless NativeLogPolicy::Silence is selected.
Loom::load owns one initialized process runtime and one model. Applications
that need several models under a separately managed runtime should use the
lower-level adapter.
Loom::load_qwen3_small adds exact byte-count and SHA-256 verification for the
catalogued Qwen3 0.6B Q8_0 GGUF. It still takes a caller path and
LoomOptions; it does not choose tokenization, chat formatting, placement, or
session allocation. Loom::profile_artifact returns a path-free verification
receipt for profiled loads.
§One bounded completion
The caller supplies a local GGUF, exact text, tokenization flags, and a nonzero token bound:
use logit_loom_runtime::{
GenerationRequest, Loom, LoomOptions, Tokenization,
};
fn run() -> Result<(), Box<dyn std::error::Error>> {
let loom = Loom::load("model.gguf", LoomOptions::default())?;
let output = loom.complete(
"The creature opened its eyes and",
Tokenization { add_bos: true },
GenerationRequest::new(64)?,
)?;
std::io::Write::write_all(&mut std::io::stdout(), output.bytes())?;
Ok(())
}CompletionOutput::bytes is authoritative. Use CompletionOutput::text only
when the complete output is expected to be valid UTF-8.
The one-shot call creates a fresh session, replaces its state with the supplied text, and drops the session after generation. It never applies a hidden system prompt or chat template.
§Controlled generation
Built-in pipeline stages receive versioned identities that bind their exact configuration. Custom callbacks still require a caller-defined stable digest:
use logit_loom_runtime::{
CandidateMode, GenerationRequest, Loom, LoomOptions, PipelineBuilder,
Tokenization,
};
fn run() -> Result<(), Box<dyn std::error::Error>> {
let loom = Loom::load("model.gguf", LoomOptions::default())?;
let mut pipeline = PipelineBuilder::new(CandidateMode::FullVocabulary, 32)?
.rank_bias(1, 4.0)?
.build()?;
let request = GenerationRequest::new(32)?.pipeline(&mut pipeline)?;
let _output = loom.complete(
"The creature opened its eyes and",
Tokenization { add_bos: true },
request,
)?;
Ok(())
}Transforms still run before native grammar, bias, penalties, filtering,
temperature, and terminal sampling. Candidate write-back remains transactional.
GenerationRequest::bias configures native post-transform logit bias;
PipelineBuilder::token_bias is an ordered Rust transform.
§Observe and stop
Generated-token callbacks run synchronously after native causal admission. An observer receives exact token bytes and may request a cooperative stop:
use logit_loom_runtime::{
ControlFlow, Digest, GenerationRequest, Loom, LoomOptions,
ObserversBuilder, Tokenization,
};
fn run() -> Result<(), Box<dyn std::error::Error>> {
let loom = Loom::load("model.gguf", LoomOptions::default())?;
let mut observers = ObserversBuilder::new()
.on_token(
Digest::of_bytes("my-observer", b"stop-after-eight-v1"),
|token| Ok(if token.position >= 8 {
ControlFlow::Stop
} else {
ControlFlow::Continue
}),
)?
.build()?;
let request = GenerationRequest::new(32)?.observers(&mut observers)?;
let _output = loom.complete(
"Exact caller text",
Tokenization { add_bos: true },
request,
)?;
Ok(())
}The position is the native causal position, not a generation-local counter.
Use CancellationToken with ObserversBuilder::cancellation when another
owner should signal the same documented safe boundaries.
§Stateful sessions and steering
LoomSession borrows its Loom and remains single-owner and synchronous.
Text replacement and append are separate methods, and tokenization flags remain
explicit. Checkpoint bytes retain the adapter’s model/build/allocation identity
requirements.
use logit_loom_runtime::{
GenerationRequest, Loom, LoomOptions, Tokenization,
};
fn run() -> Result<(), Box<dyn std::error::Error>> {
let loom = Loom::load("model.gguf", LoomOptions::default())?;
let mut session = loom.session()?;
session.replace_text("A branching point:", Tokenization { add_bos: true })?;
let checkpoint = session.capture_state()?;
let _first = session.generate(GenerationRequest::new(16)?)?;
session.restore_state(&checkpoint)?;
let _second = session.generate(GenerationRequest::new(16)?)?;
Ok(())
}LoomSession::lora and LoomSession::control_vector return typed high-level
scopes. Call clear to observe cleanup and its receipt. Dropped scopes retain
the adapter’s automatic cleanup and poisoning behavior. Checkpoint helpers are
intentionally unavailable on active steering scopes.
use logit_loom_runtime::{
GenerationRequest, Loom, LoomOptions, Tokenization,
};
fn run() -> Result<(), Box<dyn std::error::Error>> {
let loom = Loom::load("model.gguf", LoomOptions::default())?;
let mut adapter = loom.load_lora("adapter.gguf")?;
let mut session = loom.session()?;
let mut steered = session.lora(&mut adapter, 0.75)?;
steered.replace_text("Exact caller text", Tokenization { add_bos: true })?;
let _output = steered.generate(GenerationRequest::new(16)?)?;
let _cleanup = steered.clear()?;
assert!(session.is_healthy());
Ok(())
}§Activation and speculative generation
The runtime re-exports the adapter’s topology-bound activation and
target-authoritative speculation types without hiding their native ownership.
For same-model MTP, pass Loom::raw_runtime and Loom::raw_model to
generate_speculative. EAGLE-3 requires two models loaded under one
explicitly owned native low_level::llamacpp::Runtime, so applications
should construct the lower-level runtime and both Model values directly.
No high-level call chooses tensor sites, mirrors a target activation program onto a draft, changes context allocation, or falls back to ordinary generation. The speculative operation supports one sequence. Its checkpoint façade creates reusable process-local branches from complete target, draft, implementation, sampler, activation, stop-prefix, and causal state; it does not claim a portable sampler encoding. See the adapter guide and compatibility policy for the exact boundary.
§Identities and escape hatches
Loom::model_identity, Loom::backend_identity, and
Loom::backend_compatibility expose the identities used for checkpoint
compatibility. First-party pipeline and cancellation helpers receive
configuration-bound, versioned identities automatically. A custom transform or
observer requires a caller-supplied Digest, because the crate cannot infer
the identity of arbitrary Rust code.
The low_level module re-exports the foundational crates for mechanics that
should remain fully explicit.
See the runtime interface guide, mechanical experiment runbooks, architecture, and compatibility policy for the complete ownership and release boundaries.
Modules§
- low_
level - Direct access to the lower-level crates when the façade is intentionally insufficient.
Structs§
- Activation
Capture Output - Captured records and per-plan aggregate receipts drained from a session.
- Activation
Capture Plan V1 - Bounded activation-capture request.
- Activation
Capture Receipt V1 - Aggregate evidence for one completed activation capture.
- Activation
Capture Record V1 - One topology-bound captured tensor row.
- Activation
Configuration - Validated activation program, vector banks, captures, and graph profile.
- Activation
Invocation Receipt V1 - Evidence for one successful native tensor write-back.
- Activation
Invocation RowV1 - Evidence for one successful copy-transform-write transaction.
- Activation
Operation V1 - One ordered, position-scoped activation operation.
- Activation
Program Output - Transaction records and aggregate program receipt drained from a session.
- Activation
Program Receipt V1 - Aggregate successful accounting for one activation program runtime.
- Activation
Program V1 - Ordered activation operations and observation requests.
- Activation
Statistics V1 - Deterministic scalar statistics represented by exact
f32bits. - Activation
Vector Accumulator V1 - Deterministic accumulator for sparse activation samples.
- Activation
Vector Bank V1 - Topology-bound sparse activation vectors.
- Activation
Vector Provenance V1 - Mechanical provenance for vector construction.
- Activation
Vector RowV1 - One canonical sparse vector row.
- Activation
Vector Sample V1 - One caller-ordered, content-bound sparse activation sample.
- Admission
Output - Result of tokenizing exact caller text and its completed or stopped prefill.
- Audio
Prefill Plan V1 - Exact, model-free contract for one audio projector prefill.
- Audio
Prefill Receipt V1 - Content-free terminal accounting for one projected audio prefill.
- Byte
Feedback Receipt V1 - Complete exact-byte feedback execution for one projection attempt.
- Byte
Feedback Spec V1 - Exact byte-feedback implementation and resource bounds.
- Byte
Feedback Step Receipt V1 - Exact symbolic decision before one native sampling operation.
- Callback
Failure - Complete error or panic isolated at a callback boundary.
- Cancellation
Token - Cloneable one-shot cooperative cancellation signal.
- Causal
Position Range V1 - Inclusive causal-position interval.
- Checkpoint
Receipt - Serializable compatibility metadata for one backend-owned causal checkpoint.
- Completion
Output - Result of one fresh text admission followed by bounded generation.
- Control
Vector - In-memory native control-vector data and layer contract.
- Control
Vector Session - High-level session while one control vector remains active.
- Control
Vector Spec - One in-memory control-vector application.
- Digest
- A domain-separated BLAKE3 digest encoded as 64 lowercase hexadecimal bytes.
- DrySampler
- Don’t Repeat Yourself sampler configuration.
- Generation
Output - Successful bounded generation output.
- Generation
Plan - Complete bounded generation mechanics independent of prompt semantics.
- Generation
Plan V2 - Version-two generation mechanics with explicit eager or lazy grammar.
- Generation
Receipt - Mechanical receipt for one bounded generation call.
- Generation
Request - One bounded generation request and its optional execution controls.
- Grammar
- An eager llama.cpp GBNF grammar.
- Grammar
Plan V2 - Version-two eager or lazy llama.cpp GBNF grammar.
- Llama
CppTensor Profile - Explicit llama.cpp graph-site compatibility profile.
- Logit
Bias - One additive token-logit bias.
- Loom
- One loaded local model and its process-wide llama.cpp runtime.
- Loom
Options - Model loading, default session allocation, and native log options.
- Loom
Session - Higher-level borrowing façade over one single-owner causal session.
- Lora
Adapter - Loaded model-compatible
LoRAadapter. - Lora
Application - One borrowed
LoRAand its exact application scale. - Lora
Session - High-level session while one
LoRAremains active. - Lora
Spec - One model-compatible
LoRAadapter application. - Mirostat
Sampler - Stateful Mirostat terminal sampler configuration.
- Model
Artifact Receipt - Serializable evidence that one exact local artifact was verified.
- Model
Options - Model-loading options exposed by the first adapter release.
- Observed
Token - One token already admitted to backend causal state.
- Observer
Error - Error returned by a generated-token observer.
- Observer
Receipt - Mechanical accounting for one generated-token observer.
- Observer
Set - Ordered observer fan-out behind one backend callback point.
- Observers
Builder - Constructs an ordered, bounded
ObserverSet. - Pipeline
- Ordered, call-scoped transform pipeline.
- Pipeline
Builder - Constructs an ordered
Pipelinewith one shared exposure mode and bound. - Pipeline
Receipt - Aggregate accounting for an ordered transform pipeline.
- Pipeline
Spec - Ordered transform contracts sharing one candidate exposure.
- Prefill
Monitor - Panic-containing wrapper around one controlled-prefill observer.
- Prefill
Observer Error - Error returned by a prefill observer.
- Prefill
Output - Result of one plain or controlled prefill call.
- Prefill
Progress - Progress at one complete text-prefill boundary.
- Prefill
Receipt - Mechanical accounting for controlled prefill.
- Rank
Bias - Adds a finite value to one dynamically ranked candidate.
- Repetition
Sampler - Stateful repetition, frequency, and presence penalties.
- Sampling
Plan - Native sampler configuration.
- Session
Options - Context-allocation options.
- Speculation
Boundary Receipt V1 - Target-authoritative accounting for one completed proposal boundary.
- Speculation
Plan V1 - Complete backend-neutral configuration for one speculative text operation.
- Speculation
Receipt V1 - Aggregate proposal and target-acceptance accounting.
- Speculation
Telemetry Resolution V1 - Final disposition of one provisional activation telemetry record.
- Speculative
Activation Output - Activation evidence produced by one context in a speculative operation.
- Speculative
Activations - Explicit activation runtimes supplied to one speculative operation.
- Speculative
Checkpoint Output - Successful generation plus a reusable quiescent continuation checkpoint.
- Speculative
Checkpoint Receipt V1 - Serializable evidence for one quiescent speculative checkpoint envelope.
- Speculative
Checkpoint Request - Aggregate-plan lineage selected for one quiescent speculative checkpoint.
- Speculative
Continuation Request - Borrowed mechanics for continuation from a speculative checkpoint.
- Speculative
Generation Output - Complete successful output of one target-authoritative operation.
- Speculative
Request - Borrowed mechanics and owned runtime configuration for one operation.
- Speculative
Session Options - Target and draft context-allocation options for one speculative operation.
- Speculative
State Snapshot - Opaque process-local continuation state for one quiescent speculative boundary.
- Stage
- One identified implementation in an ordered pipeline.
- Stage
Receipt - Mechanical accounting for one transform stage.
- State
Snapshot - Opaque llama.cpp causal state with compatibility-bound identity metadata.
- Steering
Receipt - Mechanical receipt for one steering lifecycle transition.
- Steering
Scope - Active aggregate steering application.
- Structured
Attempt Receipt V1 - One generated, incrementally constrained, and validated projection attempt.
- Structured
Compilation Receipt V1 - Evidence that an identified compiler produced one complete projection plan.
- Structured
Mechanic Identity V1 - Content identities for one installed compiler, validator, or feedback implementation and its exact configuration.
- Structured
Projection Plan V1 - Complete compiled mechanics for one structured projection controller.
- Structured
Projection Receipt V1 - Terminal evidence for a projection selected after zero or more explicit cancelled/rejected attempts.
- Structured
Validation Receipt V1 - Content-free evidence from one installed authoritative validator.
- Text
Mechanics Checkpoint Receipt V2 - Exact ordinary-context state captured at an aggregate operation boundary.
- Text
Mechanics Cleanup Receipt V2 - Resource-release and quiescence evidence at aggregate operation completion.
- Text
Mechanics Plan V1 - Complete backend-neutral mechanics for one bounded text operation.
- Text
Mechanics Plan V2 - Aggregate text mechanics with topology-bound activation and speculation.
- Text
Mechanics Receipt V1 - Content-free terminal evidence for
TextMechanicsPlanV1. - Text
Mechanics Receipt V2 - Content-free terminal evidence for
TextMechanicsPlanV2. - Text
Model Topology V1 - Backend-neutral, content-bound model topology.
- Token
Bias - Adds finite per-token values to candidates present in the current view.
- TokenId
- A non-negative model-tokenizer identifier.
- Tokenization
- Tokenization flags for exact text admission.
- Transform
Context - Borrowed state supplied to one Rust logit-transform invocation.
- Transform
Error - Error returned by a user-defined logit transform.
- Transform
Spec - One content-identified transform contract.
Enums§
- Activation
Capture Positions V1 - Causal positions retained by one activation capture.
- Activation
Capture Retention V1 - Data retained for each selected tensor row.
- Activation
Captured Data V1 - Retained data for one captured tensor row.
- Activation
Operator V1 - Built-in transactional tensor operator.
- Activation
Phase V1 - Decode phase at which one activation operation executes.
- Activation
Position Scope V1 - Causal positions selected by one activation operation.
- Activation
Telemetry Disposition V1 - Causal status of telemetry created during target or draft evaluation.
- Activation
Vector Normalization V1 - Per-row vector normalization contract.
- Activation
Vector Operation V1 - Content-free vector construction performed by an accumulator.
- Callback
Phase - Callback phase that produced a contained failure.
- Candidate
Mode - Candidate-logit exposure presented to every stage in a pipeline.
- Control
Flow - Cooperative action returned at a documented safe execution boundary.
- Core
Error - Failures raised while constructing or validating public contracts.
- Device
Policy - Runtime placement requirement checked after model loading.
- Error
- Failures from convenience-layer validation or delegated mechanics.
- Generation
Finish - Why one bounded generation call stopped.
- Grammar
Activation V2 - Activation policy for a version-two native grammar.
- Mirostat
Version - Native Mirostat algorithm version.
- Native
LogPolicy - Native log handling selected before model loading.
- Pipeline
Error - Failure while validating or executing an ordered pipeline.
- Prefill
Finish - Current or terminal disposition of one controlled prefill operation.
- Speculation
Activation Policy V1 - Explicit target and draft activation-program selection.
- Steering
Action - Lifecycle action recorded at the Rust/native boundary.
- Steering
Kind - Session steering resource kind.
- Structured
Attempt Disposition V1 - Effect of one caller-explicit projection attempt.
- Structured
Validation Status V1 - Authoritative post-generation validation result.
- Text
Speculative Mechanism V1 - Speculative mechanisms a loaded text topology can report.
- Text
Tensor Element Type V1 - Tensor element representation expected at a selected graph site.
- Text
Tensor Site Family V1 - Stable family shared by compatible activation vector rows and tensor sites.
- Text
Tensor Site V1 - Exact graph tensor selected for capture or mutation.
Constants§
- LLAMA_
CPP_ BINDING_ SOURCE_ REVISION - Exact source revision of the reviewed native Rust binding successor.
- LLAMA_
CPP_ BINDING_ VERSION - Exact native Rust binding version used by this adapter release.
- LLAMA_
CPP_ REVISION - Exact llama.cpp revision carried by the ADR 0002 binding successor.
- MAX_
ACTIVATION_ CAPTURE_ POSITIONS - Maximum causal positions selected by one capture.
- MAX_
ACTIVATION_ ELEMENTS - Maximum
f32elements copied or retained by one activation operation. - MAX_
ACTIVATION_ INVOCATION_ RECEIPTS - Maximum successful tensor transactions retained in one program receipt.
- MAX_
ACTIVATION_ OBSERVATIONS - Maximum capture-plan observations named by one program.
- MAX_
ACTIVATION_ OPERATIONS - Maximum ordered activation operations in one program.
- MAX_
ACTIVATION_ POSITION_ RANGES - Maximum inclusive causal ranges selected by one contract.
- MAX_
ACTIVATION_ RETAINED_ BYTES - Maximum retained activation bytes declared by one capture.
- MAX_
ACTIVATION_ ROW_ SEQUENCES - Maximum sequence identifiers attached to one logical tensor row.
- MAX_
ACTIVATION_ SITES - Maximum exact tensor sites selected by one capture or operation.
- MAX_
ACTIVATION_ TRANSACTION_ ROWS - Maximum logical rows represented by one tensor transaction.
- MAX_
ACTIVATION_ VECTOR_ INPUTS - Maximum input artifacts represented by vector provenance.
- MAX_
ACTIVATION_ VECTOR_ ROWS - Maximum sparse vector rows in one bank.
- MAX_
AUDIO_ FRAMES - Largest number of mono frames accepted by one audio-prefill plan.
- MAX_
AUDIO_ SAMPLE_ RATE_ HZ - Largest projector-declared sampling rate accepted by this contract.
- MAX_
DRY_ SEQUENCE_ BREAKERS - Maximum exact sequence breakers accepted by one DRY sampler.
- MAX_
DRY_ SEQUENCE_ BREAKER_ BYTES - Maximum UTF-8 bytes accepted in one DRY sequence breaker.
- MAX_
GRAMMAR_ ROOT_ BYTES - Maximum UTF-8 bytes accepted in an eager grammar root name.
- MAX_
GRAMMAR_ SOURCE_ BYTES - Maximum UTF-8 bytes accepted in an eager grammar source.
- MAX_
LAZY_ GRAMMAR_ TRIGGER_ BYTES - Maximum aggregate UTF-8 bytes accepted across lazy-grammar trigger patterns.
- MAX_
LAZY_ GRAMMAR_ TRIGGER_ PATTERNS - Maximum trigger patterns accepted by one lazy grammar.
- MAX_
LAZY_ GRAMMAR_ TRIGGER_ PATTERN_ BYTES - Maximum UTF-8 bytes accepted in one lazy-grammar trigger pattern.
- MAX_
LAZY_ GRAMMAR_ TRIGGER_ TOKENS - Maximum trigger tokens accepted by one lazy grammar.
- MAX_
LOGIT_ BIASES - Maximum additive token biases accepted by one generation plan.
- MAX_
OBSERVERS - Maximum observers sharing one ordered fan-out.
- MAX_
PIPELINE_ STAGES - Maximum stages in one ordered transform pipeline.
- MAX_
SPARSE_ CANDIDATES - Maximum sparse candidates exposed to one transform call.
- MAX_
SPECULATION_ BOUNDARIES - Maximum completed proposal boundaries retained in one aggregate receipt.
- MAX_
SPECULATION_ SEQUENCES - Maximum sequence slots declared by one speculative plan.
- MAX_
SPECULATION_ STATE_ BYTES - Maximum opaque speculative implementation-state bytes.
- MAX_
SPECULATION_ TELEMETRY_ RESOLUTIONS - Maximum activation telemetry resolutions retained at one proposal boundary.
- MAX_
SPECULATIVE_ TOKENS - Maximum speculative tokens requested from one draft boundary.
- MAX_
STOP_ SEQUENCES - Maximum exact byte stop sequences accepted by one generation plan.
- MAX_
STOP_ SEQUENCE_ BYTES - Maximum bytes accepted in one exact stop sequence.
- MAX_
STRUCTURED_ ATTEMPTS - Maximum caller-explicit attempts retained by one projection controller.
- MAX_
STRUCTURED_ CANDIDATES - Maximum model vocabulary accepted by structured byte feedback.
- MAX_
STRUCTURED_ OUTPUT_ BYTES - Maximum generated bytes accepted by one feedback implementation.
- MAX_
STRUCTURED_ SCOPE_ BYTES - Maximum UTF-8 bytes in one caller-defined projection scope.
- MAX_
STRUCTURED_ STEPS - Maximum generated-token decisions accepted by one feedback implementation.
- MAX_
TEXT_ MECHANICS_ CAPTURES - Maximum activation-capture plans bound into one aggregate operation.
- MAX_
TEXT_ MECHANICS_ LORAS - Maximum
LoRAapplications bound into one text-mechanics plan. - MAX_
TEXT_ MODEL_ EXPERTS - Maximum experts represented by one topology contract.
- MAX_
TEXT_ MODEL_ LAYERS - Maximum transformer layers represented by one topology contract.
- MAX_
TEXT_ TENSOR_ NAME_ BYTES - Maximum UTF-8 bytes in a backend-profile tensor name.
- MAX_
TEXT_ TENSOR_ ROW_ ELEMENTS - Maximum elements in one reported model row.
- MAX_
TOKENIZATION_ BYTES - Maximum UTF-8 bytes accepted by one native tokenization call.
Traits§
- Logit
Transform - Safe, synchronous implementation of one per-step logit transformation.
- Observer
- Safe synchronous observer for generated-token boundaries.
- Prefill
Observer - Safe synchronous observer for complete prefill chunks.
Functions§
- activation_
f32_ row_ identity - Returns the canonical identity of one bounded finite activation
f32row. - generate_
speculative - Runs one bounded MTP or EAGLE-3 operation.
- generate_
speculative_ checkpointed - Runs one bounded speculative operation and captures complete quiescent continuation state.
- resume_
speculative_ checkpointed - Continues one compatible quiescent checkpoint and captures the next quiescent boundary.
- speculation_
implementation_ identity - Returns the exact native and safe-wrapper implementation identity.
- structured_
output_ digest - Returns the stable identity used for exact generated projection bytes.
- structured_
tokens_ digest - Returns the stable identity used for a complete ordered token-ID sequence.
Type Aliases§
- Result
- Result type returned by
logit-loom-runtime.