Expand description
el-runtime — the Core: inference session lifecycle, the port traits that
the collaborator contexts plug into, and the decode-loop orchestrator
(ADR-001). Air-gap is structural (ADR-004): this crate has no network
dependency, and the only outbound seam is the opt-in ports::HybridRelay.
The decode step composes collaborators in a fixed, invariant order
(docs/ddd/domain-events.md): grammar mask → safety adjust → sample →
commit, so safety steering only ever operates over already-legal tokens.
Structs§
- Allow
AllMasker - Allow-all grammar — every token legal (used when no schema is registered).
- Identity
Compressor - Identity compressor — passes the prompt through unchanged.
- Inference
Session - One live generation. Constructing it requires a
LoadPermit, so a model that has not passed the provenance gate (ADR-006) cannot reach the runtime — the Conformist relationship is enforced in the type system. - Logit
Adjustment - A vector subtracted from target logits to steer away from unsafe output. Sparse and integer (milli-logits) for deterministic, allocation-light steps.
- Null
Engine - A trivial engine that emits EOS immediately after prefill. Lets you exercise the full session lifecycle without the Candle adapter (ADR-002 is the real engine). Not for production inference.
- Ports
- The collaborator ports bound to a session.
relayisNoneby default — air-gapped.
Traits§
- Grammar
Masker - Grammar Constraint port (llguidance — context 4). Returns a per-token allow
mask of length
vocab;true= legal this step. - Hybrid
Relay - Opt-in LAN relay (ADR-004 HybridMode). Implementations MUST stay on the local network — there is no cloud variant.
- Inference
Engine - The inference engine adapter (
RuntimeAcl). Implemented for real by Candle in the excluded adapterel-engine-candle(ADR-002). - Prompt
Compressor - Prompt Compression port (LLMLingua-2 — context 2).
- Safety
Steerer - Per-step safety intervention. The runtime applies this after the grammar mask and before sampling.