Skip to main content

Crate logit_loom_runtime

Crate logit_loom_runtime 

Source
Expand description

§logit-loom-runtime

An explicit higher-level local runtime for Logit Loom’s llama.cpp adapter.

The crate shortens model loading, exact text admission, bounded generation, transform and observer construction, checkpoint branching, and steering scopes. It does not download models, choose a chat template, silently fall back to CPU, assume generated token pieces are UTF-8, or make claims about output quality.

Use this crate for a direct local llama.cpp workflow. Use logit-loom when building backend-neutral mechanics, or logit-loom-llamacpp when an application needs to own the native runtime and model objects separately.

§Select a backend

No native backend feature is enabled by default. Select the deployment feature explicitly:

[dependencies]
logit-loom-runtime = { version = "=0.2.0", features = ["vulkan"] }

LoomOptions::default continues to require accelerator participation. Native logs are preserved unless NativeLogPolicy::Silence is selected.

Loom::load owns one initialized process runtime and one model. Applications that need several models under a separately managed runtime should use the lower-level adapter.

Loom::load_qwen3_small adds exact byte-count and SHA-256 verification for the catalogued Qwen3 0.6B Q8_0 GGUF. It still takes a caller path and LoomOptions; it does not choose tokenization, chat formatting, placement, or session allocation. Loom::profile_artifact returns a path-free verification receipt for profiled loads.

§One bounded completion

The caller supplies a local GGUF, exact text, tokenization flags, and a nonzero token bound:

use logit_loom_runtime::{
    GenerationRequest, Loom, LoomOptions, Tokenization,
};

fn run() -> Result<(), Box<dyn std::error::Error>> {
    let loom = Loom::load("model.gguf", LoomOptions::default())?;
    let output = loom.complete(
        "The creature opened its eyes and",
        Tokenization { add_bos: true },
        GenerationRequest::new(64)?,
    )?;
    std::io::Write::write_all(&mut std::io::stdout(), output.bytes())?;
    Ok(())
}

CompletionOutput::bytes is authoritative. Use CompletionOutput::text only when the complete output is expected to be valid UTF-8.

The one-shot call creates a fresh session, replaces its state with the supplied text, and drops the session after generation. It never applies a hidden system prompt or chat template.

§Controlled generation

Built-in pipeline stages receive versioned identities that bind their exact configuration. Custom callbacks still require a caller-defined stable digest:

use logit_loom_runtime::{
    CandidateMode, GenerationRequest, Loom, LoomOptions, PipelineBuilder,
    Tokenization,
};

fn run() -> Result<(), Box<dyn std::error::Error>> {
    let loom = Loom::load("model.gguf", LoomOptions::default())?;
    let mut pipeline = PipelineBuilder::new(CandidateMode::FullVocabulary, 32)?
        .rank_bias(1, 4.0)?
        .build()?;
    let request = GenerationRequest::new(32)?.pipeline(&mut pipeline)?;
    let _output = loom.complete(
        "The creature opened its eyes and",
        Tokenization { add_bos: true },
        request,
    )?;
    Ok(())
}

Transforms still run before native grammar, bias, penalties, filtering, temperature, and terminal sampling. Candidate write-back remains transactional. GenerationRequest::bias configures native post-transform logit bias; PipelineBuilder::token_bias is an ordered Rust transform.

§Observe and stop

Generated-token callbacks run synchronously after native causal admission. An observer receives exact token bytes and may request a cooperative stop:

use logit_loom_runtime::{
    ControlFlow, Digest, GenerationRequest, Loom, LoomOptions,
    ObserversBuilder, Tokenization,
};

fn run() -> Result<(), Box<dyn std::error::Error>> {
    let loom = Loom::load("model.gguf", LoomOptions::default())?;
    let mut observers = ObserversBuilder::new()
        .on_token(
            Digest::of_bytes("my-observer", b"stop-after-eight-v1"),
            |token| Ok(if token.position >= 8 {
                ControlFlow::Stop
            } else {
                ControlFlow::Continue
            }),
        )?
        .build()?;
    let request = GenerationRequest::new(32)?.observers(&mut observers)?;
    let _output = loom.complete(
        "Exact caller text",
        Tokenization { add_bos: true },
        request,
    )?;
    Ok(())
}

The position is the native causal position, not a generation-local counter. Use CancellationToken with ObserversBuilder::cancellation when another owner should signal the same documented safe boundaries.

§Stateful sessions and steering

LoomSession borrows its Loom and remains single-owner and synchronous. Text replacement and append are separate methods, and tokenization flags remain explicit. Checkpoint bytes retain the adapter’s model/build/allocation identity requirements.

use logit_loom_runtime::{
    GenerationRequest, Loom, LoomOptions, Tokenization,
};

fn run() -> Result<(), Box<dyn std::error::Error>> {
    let loom = Loom::load("model.gguf", LoomOptions::default())?;
    let mut session = loom.session()?;
    session.replace_text("A branching point:", Tokenization { add_bos: true })?;
    let checkpoint = session.capture_state()?;

    let _first = session.generate(GenerationRequest::new(16)?)?;
    session.restore_state(&checkpoint)?;
    let _second = session.generate(GenerationRequest::new(16)?)?;
    Ok(())
}

LoomSession::lora and LoomSession::control_vector return typed high-level scopes. Call clear to observe cleanup and its receipt. Dropped scopes retain the adapter’s automatic cleanup and poisoning behavior. Checkpoint helpers are intentionally unavailable on active steering scopes.

use logit_loom_runtime::{
    GenerationRequest, Loom, LoomOptions, Tokenization,
};

fn run() -> Result<(), Box<dyn std::error::Error>> {
    let loom = Loom::load("model.gguf", LoomOptions::default())?;
    let mut adapter = loom.load_lora("adapter.gguf")?;
    let mut session = loom.session()?;
    let mut steered = session.lora(&mut adapter, 0.75)?;
    steered.replace_text("Exact caller text", Tokenization { add_bos: true })?;
    let _output = steered.generate(GenerationRequest::new(16)?)?;
    let _cleanup = steered.clear()?;
    assert!(session.is_healthy());
    Ok(())
}

§Activation and speculative generation

The runtime re-exports the adapter’s topology-bound activation and target-authoritative speculation types without hiding their native ownership. For same-model MTP, pass Loom::raw_runtime and Loom::raw_model to generate_speculative. EAGLE-3 requires two models loaded under one explicitly owned native low_level::llamacpp::Runtime, so applications should construct the lower-level runtime and both Model values directly.

No high-level call chooses tensor sites, mirrors a target activation program onto a draft, changes context allocation, or falls back to ordinary generation. The speculative operation supports one sequence. Its checkpoint façade creates reusable process-local branches from complete target, draft, implementation, sampler, activation, stop-prefix, and causal state; it does not claim a portable sampler encoding. See the adapter guide and compatibility policy for the exact boundary.

§Identities and escape hatches

Loom::model_identity, Loom::backend_identity, and Loom::backend_compatibility expose the identities used for checkpoint compatibility. First-party pipeline and cancellation helpers receive configuration-bound, versioned identities automatically. A custom transform or observer requires a caller-supplied Digest, because the crate cannot infer the identity of arbitrary Rust code.

The low_level module re-exports the foundational crates for mechanics that should remain fully explicit.

See the runtime interface guide, mechanical experiment runbooks, architecture, and compatibility policy for the complete ownership and release boundaries.

Modules§

low_level
Direct access to the lower-level crates when the façade is intentionally insufficient.

Structs§

ActivationCaptureOutput
Captured records and per-plan aggregate receipts drained from a session.
ActivationCapturePlanV1
Bounded activation-capture request.
ActivationCaptureReceiptV1
Aggregate evidence for one completed activation capture.
ActivationCaptureRecordV1
One topology-bound captured tensor row.
ActivationConfiguration
Validated activation program, vector banks, captures, and graph profile.
ActivationInvocationReceiptV1
Evidence for one successful native tensor write-back.
ActivationInvocationRowV1
Evidence for one successful copy-transform-write transaction.
ActivationOperationV1
One ordered, position-scoped activation operation.
ActivationProgramOutput
Transaction records and aggregate program receipt drained from a session.
ActivationProgramReceiptV1
Aggregate successful accounting for one activation program runtime.
ActivationProgramV1
Ordered activation operations and observation requests.
ActivationStatisticsV1
Deterministic scalar statistics represented by exact f32 bits.
ActivationVectorAccumulatorV1
Deterministic accumulator for sparse activation samples.
ActivationVectorBankV1
Topology-bound sparse activation vectors.
ActivationVectorProvenanceV1
Mechanical provenance for vector construction.
ActivationVectorRowV1
One canonical sparse vector row.
ActivationVectorSampleV1
One caller-ordered, content-bound sparse activation sample.
AdmissionOutput
Result of tokenizing exact caller text and its completed or stopped prefill.
AudioPrefillPlanV1
Exact, model-free contract for one audio projector prefill.
AudioPrefillReceiptV1
Content-free terminal accounting for one projected audio prefill.
ByteFeedbackReceiptV1
Complete exact-byte feedback execution for one projection attempt.
ByteFeedbackSpecV1
Exact byte-feedback implementation and resource bounds.
ByteFeedbackStepReceiptV1
Exact symbolic decision before one native sampling operation.
CallbackFailure
Complete error or panic isolated at a callback boundary.
CancellationToken
Cloneable one-shot cooperative cancellation signal.
CausalPositionRangeV1
Inclusive causal-position interval.
CheckpointReceipt
Serializable compatibility metadata for one backend-owned causal checkpoint.
CompletionOutput
Result of one fresh text admission followed by bounded generation.
ControlVector
In-memory native control-vector data and layer contract.
ControlVectorSession
High-level session while one control vector remains active.
ControlVectorSpec
One in-memory control-vector application.
Digest
A domain-separated BLAKE3 digest encoded as 64 lowercase hexadecimal bytes.
DrySampler
Don’t Repeat Yourself sampler configuration.
GenerationOutput
Successful bounded generation output.
GenerationPlan
Complete bounded generation mechanics independent of prompt semantics.
GenerationPlanV2
Version-two generation mechanics with explicit eager or lazy grammar.
GenerationReceipt
Mechanical receipt for one bounded generation call.
GenerationRequest
One bounded generation request and its optional execution controls.
Grammar
An eager llama.cpp GBNF grammar.
GrammarPlanV2
Version-two eager or lazy llama.cpp GBNF grammar.
LlamaCppTensorProfile
Explicit llama.cpp graph-site compatibility profile.
LogitBias
One additive token-logit bias.
Loom
One loaded local model and its process-wide llama.cpp runtime.
LoomOptions
Model loading, default session allocation, and native log options.
LoomSession
Higher-level borrowing façade over one single-owner causal session.
LoraAdapter
Loaded model-compatible LoRA adapter.
LoraApplication
One borrowed LoRA and its exact application scale.
LoraSession
High-level session while one LoRA remains active.
LoraSpec
One model-compatible LoRA adapter application.
MirostatSampler
Stateful Mirostat terminal sampler configuration.
ModelArtifactReceipt
Serializable evidence that one exact local artifact was verified.
ModelOptions
Model-loading options exposed by the first adapter release.
ObservedToken
One token already admitted to backend causal state.
ObserverError
Error returned by a generated-token observer.
ObserverReceipt
Mechanical accounting for one generated-token observer.
ObserverSet
Ordered observer fan-out behind one backend callback point.
ObserversBuilder
Constructs an ordered, bounded ObserverSet.
Pipeline
Ordered, call-scoped transform pipeline.
PipelineBuilder
Constructs an ordered Pipeline with one shared exposure mode and bound.
PipelineReceipt
Aggregate accounting for an ordered transform pipeline.
PipelineSpec
Ordered transform contracts sharing one candidate exposure.
PrefillMonitor
Panic-containing wrapper around one controlled-prefill observer.
PrefillObserverError
Error returned by a prefill observer.
PrefillOutput
Result of one plain or controlled prefill call.
PrefillProgress
Progress at one complete text-prefill boundary.
PrefillReceipt
Mechanical accounting for controlled prefill.
RankBias
Adds a finite value to one dynamically ranked candidate.
RepetitionSampler
Stateful repetition, frequency, and presence penalties.
SamplingPlan
Native sampler configuration.
SessionOptions
Context-allocation options.
SpeculationBoundaryReceiptV1
Target-authoritative accounting for one completed proposal boundary.
SpeculationPlanV1
Complete backend-neutral configuration for one speculative text operation.
SpeculationReceiptV1
Aggregate proposal and target-acceptance accounting.
SpeculationTelemetryResolutionV1
Final disposition of one provisional activation telemetry record.
SpeculativeActivationOutput
Activation evidence produced by one context in a speculative operation.
SpeculativeActivations
Explicit activation runtimes supplied to one speculative operation.
SpeculativeCheckpointOutput
Successful generation plus a reusable quiescent continuation checkpoint.
SpeculativeCheckpointReceiptV1
Serializable evidence for one quiescent speculative checkpoint envelope.
SpeculativeCheckpointRequest
Aggregate-plan lineage selected for one quiescent speculative checkpoint.
SpeculativeContinuationRequest
Borrowed mechanics for continuation from a speculative checkpoint.
SpeculativeGenerationOutput
Complete successful output of one target-authoritative operation.
SpeculativeRequest
Borrowed mechanics and owned runtime configuration for one operation.
SpeculativeSessionOptions
Target and draft context-allocation options for one speculative operation.
SpeculativeStateSnapshot
Opaque process-local continuation state for one quiescent speculative boundary.
Stage
One identified implementation in an ordered pipeline.
StageReceipt
Mechanical accounting for one transform stage.
StateSnapshot
Opaque llama.cpp causal state with compatibility-bound identity metadata.
SteeringReceipt
Mechanical receipt for one steering lifecycle transition.
SteeringScope
Active aggregate steering application.
StructuredAttemptReceiptV1
One generated, incrementally constrained, and validated projection attempt.
StructuredCompilationReceiptV1
Evidence that an identified compiler produced one complete projection plan.
StructuredMechanicIdentityV1
Content identities for one installed compiler, validator, or feedback implementation and its exact configuration.
StructuredProjectionPlanV1
Complete compiled mechanics for one structured projection controller.
StructuredProjectionReceiptV1
Terminal evidence for a projection selected after zero or more explicit cancelled/rejected attempts.
StructuredValidationReceiptV1
Content-free evidence from one installed authoritative validator.
TextMechanicsCheckpointReceiptV2
Exact ordinary-context state captured at an aggregate operation boundary.
TextMechanicsCleanupReceiptV2
Resource-release and quiescence evidence at aggregate operation completion.
TextMechanicsPlanV1
Complete backend-neutral mechanics for one bounded text operation.
TextMechanicsPlanV2
Aggregate text mechanics with topology-bound activation and speculation.
TextMechanicsReceiptV1
Content-free terminal evidence for TextMechanicsPlanV1.
TextMechanicsReceiptV2
Content-free terminal evidence for TextMechanicsPlanV2.
TextModelTopologyV1
Backend-neutral, content-bound model topology.
TokenBias
Adds finite per-token values to candidates present in the current view.
TokenId
A non-negative model-tokenizer identifier.
Tokenization
Tokenization flags for exact text admission.
TransformContext
Borrowed state supplied to one Rust logit-transform invocation.
TransformError
Error returned by a user-defined logit transform.
TransformSpec
One content-identified transform contract.

Enums§

ActivationCapturePositionsV1
Causal positions retained by one activation capture.
ActivationCaptureRetentionV1
Data retained for each selected tensor row.
ActivationCapturedDataV1
Retained data for one captured tensor row.
ActivationOperatorV1
Built-in transactional tensor operator.
ActivationPhaseV1
Decode phase at which one activation operation executes.
ActivationPositionScopeV1
Causal positions selected by one activation operation.
ActivationTelemetryDispositionV1
Causal status of telemetry created during target or draft evaluation.
ActivationVectorNormalizationV1
Per-row vector normalization contract.
ActivationVectorOperationV1
Content-free vector construction performed by an accumulator.
CallbackPhase
Callback phase that produced a contained failure.
CandidateMode
Candidate-logit exposure presented to every stage in a pipeline.
ControlFlow
Cooperative action returned at a documented safe execution boundary.
CoreError
Failures raised while constructing or validating public contracts.
DevicePolicy
Runtime placement requirement checked after model loading.
Error
Failures from convenience-layer validation or delegated mechanics.
GenerationFinish
Why one bounded generation call stopped.
GrammarActivationV2
Activation policy for a version-two native grammar.
MirostatVersion
Native Mirostat algorithm version.
NativeLogPolicy
Native log handling selected before model loading.
PipelineError
Failure while validating or executing an ordered pipeline.
PrefillFinish
Current or terminal disposition of one controlled prefill operation.
SpeculationActivationPolicyV1
Explicit target and draft activation-program selection.
SteeringAction
Lifecycle action recorded at the Rust/native boundary.
SteeringKind
Session steering resource kind.
StructuredAttemptDispositionV1
Effect of one caller-explicit projection attempt.
StructuredValidationStatusV1
Authoritative post-generation validation result.
TextSpeculativeMechanismV1
Speculative mechanisms a loaded text topology can report.
TextTensorElementTypeV1
Tensor element representation expected at a selected graph site.
TextTensorSiteFamilyV1
Stable family shared by compatible activation vector rows and tensor sites.
TextTensorSiteV1
Exact graph tensor selected for capture or mutation.

Constants§

LLAMA_CPP_BINDING_SOURCE_REVISION
Exact source revision of the reviewed native Rust binding successor.
LLAMA_CPP_BINDING_VERSION
Exact native Rust binding version used by this adapter release.
LLAMA_CPP_REVISION
Exact llama.cpp revision carried by the ADR 0002 binding successor.
MAX_ACTIVATION_CAPTURE_POSITIONS
Maximum causal positions selected by one capture.
MAX_ACTIVATION_ELEMENTS
Maximum f32 elements copied or retained by one activation operation.
MAX_ACTIVATION_INVOCATION_RECEIPTS
Maximum successful tensor transactions retained in one program receipt.
MAX_ACTIVATION_OBSERVATIONS
Maximum capture-plan observations named by one program.
MAX_ACTIVATION_OPERATIONS
Maximum ordered activation operations in one program.
MAX_ACTIVATION_POSITION_RANGES
Maximum inclusive causal ranges selected by one contract.
MAX_ACTIVATION_RETAINED_BYTES
Maximum retained activation bytes declared by one capture.
MAX_ACTIVATION_ROW_SEQUENCES
Maximum sequence identifiers attached to one logical tensor row.
MAX_ACTIVATION_SITES
Maximum exact tensor sites selected by one capture or operation.
MAX_ACTIVATION_TRANSACTION_ROWS
Maximum logical rows represented by one tensor transaction.
MAX_ACTIVATION_VECTOR_INPUTS
Maximum input artifacts represented by vector provenance.
MAX_ACTIVATION_VECTOR_ROWS
Maximum sparse vector rows in one bank.
MAX_AUDIO_FRAMES
Largest number of mono frames accepted by one audio-prefill plan.
MAX_AUDIO_SAMPLE_RATE_HZ
Largest projector-declared sampling rate accepted by this contract.
MAX_DRY_SEQUENCE_BREAKERS
Maximum exact sequence breakers accepted by one DRY sampler.
MAX_DRY_SEQUENCE_BREAKER_BYTES
Maximum UTF-8 bytes accepted in one DRY sequence breaker.
MAX_GRAMMAR_ROOT_BYTES
Maximum UTF-8 bytes accepted in an eager grammar root name.
MAX_GRAMMAR_SOURCE_BYTES
Maximum UTF-8 bytes accepted in an eager grammar source.
MAX_LAZY_GRAMMAR_TRIGGER_BYTES
Maximum aggregate UTF-8 bytes accepted across lazy-grammar trigger patterns.
MAX_LAZY_GRAMMAR_TRIGGER_PATTERNS
Maximum trigger patterns accepted by one lazy grammar.
MAX_LAZY_GRAMMAR_TRIGGER_PATTERN_BYTES
Maximum UTF-8 bytes accepted in one lazy-grammar trigger pattern.
MAX_LAZY_GRAMMAR_TRIGGER_TOKENS
Maximum trigger tokens accepted by one lazy grammar.
MAX_LOGIT_BIASES
Maximum additive token biases accepted by one generation plan.
MAX_OBSERVERS
Maximum observers sharing one ordered fan-out.
MAX_PIPELINE_STAGES
Maximum stages in one ordered transform pipeline.
MAX_SPARSE_CANDIDATES
Maximum sparse candidates exposed to one transform call.
MAX_SPECULATION_BOUNDARIES
Maximum completed proposal boundaries retained in one aggregate receipt.
MAX_SPECULATION_SEQUENCES
Maximum sequence slots declared by one speculative plan.
MAX_SPECULATION_STATE_BYTES
Maximum opaque speculative implementation-state bytes.
MAX_SPECULATION_TELEMETRY_RESOLUTIONS
Maximum activation telemetry resolutions retained at one proposal boundary.
MAX_SPECULATIVE_TOKENS
Maximum speculative tokens requested from one draft boundary.
MAX_STOP_SEQUENCES
Maximum exact byte stop sequences accepted by one generation plan.
MAX_STOP_SEQUENCE_BYTES
Maximum bytes accepted in one exact stop sequence.
MAX_STRUCTURED_ATTEMPTS
Maximum caller-explicit attempts retained by one projection controller.
MAX_STRUCTURED_CANDIDATES
Maximum model vocabulary accepted by structured byte feedback.
MAX_STRUCTURED_OUTPUT_BYTES
Maximum generated bytes accepted by one feedback implementation.
MAX_STRUCTURED_SCOPE_BYTES
Maximum UTF-8 bytes in one caller-defined projection scope.
MAX_STRUCTURED_STEPS
Maximum generated-token decisions accepted by one feedback implementation.
MAX_TEXT_MECHANICS_CAPTURES
Maximum activation-capture plans bound into one aggregate operation.
MAX_TEXT_MECHANICS_LORAS
Maximum LoRA applications bound into one text-mechanics plan.
MAX_TEXT_MODEL_EXPERTS
Maximum experts represented by one topology contract.
MAX_TEXT_MODEL_LAYERS
Maximum transformer layers represented by one topology contract.
MAX_TEXT_TENSOR_NAME_BYTES
Maximum UTF-8 bytes in a backend-profile tensor name.
MAX_TEXT_TENSOR_ROW_ELEMENTS
Maximum elements in one reported model row.
MAX_TOKENIZATION_BYTES
Maximum UTF-8 bytes accepted by one native tokenization call.

Traits§

LogitTransform
Safe, synchronous implementation of one per-step logit transformation.
Observer
Safe synchronous observer for generated-token boundaries.
PrefillObserver
Safe synchronous observer for complete prefill chunks.

Functions§

activation_f32_row_identity
Returns the canonical identity of one bounded finite activation f32 row.
generate_speculative
Runs one bounded MTP or EAGLE-3 operation.
generate_speculative_checkpointed
Runs one bounded speculative operation and captures complete quiescent continuation state.
resume_speculative_checkpointed
Continues one compatible quiescent checkpoint and captures the next quiescent boundary.
speculation_implementation_identity
Returns the exact native and safe-wrapper implementation identity.
structured_output_digest
Returns the stable identity used for exact generated projection bytes.
structured_tokens_digest
Returns the stable identity used for a complete ordered token-ID sequence.

Type Aliases§

Result
Result type returned by logit-loom-runtime.