Skip to main content

Crate el_runtime

Crate el_runtime 

Source
Expand description

el-runtime — the Core: inference session lifecycle, the port traits that the collaborator contexts plug into, and the decode-loop orchestrator (ADR-001). Air-gap is structural (ADR-004): this crate has no network dependency, and the only outbound seam is the opt-in ports::HybridRelay.

The decode step composes collaborators in a fixed, invariant order (docs/ddd/domain-events.md): grammar mask → safety adjust → sample → commit, so safety steering only ever operates over already-legal tokens.

Structs§

AllowAllMasker
Allow-all grammar — every token legal (used when no schema is registered).
IdentityCompressor
Identity compressor — passes the prompt through unchanged.
InferenceSession
One live generation. Constructing it requires a LoadPermit, so a model that has not passed the provenance gate (ADR-006) cannot reach the runtime — the Conformist relationship is enforced in the type system.
LogitAdjustment
A vector subtracted from target logits to steer away from unsafe output. Sparse and integer (milli-logits) for deterministic, allocation-light steps.
NullEngine
A trivial engine that emits EOS immediately after prefill. Lets you exercise the full session lifecycle without the Candle adapter (ADR-002 is the real engine). Not for production inference.
Ports
The collaborator ports bound to a session. relay is None by default — air-gapped.

Traits§

GrammarMasker
Grammar Constraint port (llguidance — context 4). Returns a per-token allow mask of length vocab; true = legal this step.
HybridRelay
Opt-in LAN relay (ADR-004 HybridMode). Implementations MUST stay on the local network — there is no cloud variant.
InferenceEngine
The inference engine adapter (RuntimeAcl). Implemented for real by Candle in the excluded adapter el-engine-candle (ADR-002).
PromptCompressor
Prompt Compression port (LLMLingua-2 — context 2).
SafetySteerer
Per-step safety intervention. The runtime applies this after the grammar mask and before sampling.