Skip to main content

Crate inference

Crate inference 

Source
Expand description

Host side of rlm.infer and rlm.map (inference.ts): one private, bounded model conversation per frame over explicit context views, with {calls, tokens, depth} budget accounting, contract validation and bounded re-asks, and Incomplete observations when budget or repairs run out or timeout_ms passes. Also serves rlm.load (content-addressed handles under the session) and rlm.frames (frame traces); every other request goes to the inner handler.

A session’s frames share one FrameSession: the budget ledger (a nested frame draws on its parent frame’s pool), the traces (kept as session values, so rlm.frames reads them in any later cell), and the frames still running. With a task host (FrameTasks) every frame is an rlm-frame@1 task: admitted under its caller, journaled, cancelled with the calling cell, charged to the usage ledger request by request; a frame with budget.depth > 1 runs its conversation on a lane of its own with the rlm cell as its only tool.

Modules§

payload
What the frame path adds to a provider payload (inference.ts beforePayload): the contract as the Claude Code CLI’s json_schema (withContractSchema), and the output cap at the request’s tranche (capOutputTokens), whatever the API’s payload shape.

Structs§

FrameExecution
One rlm-frame@1 task run, as the task host hands it over (NativeFrameRun).
FrameSession
The frames of one session; clones of the Arc share them.
Frames
Serves rlm.map, rlm.models and the frame-model defaults of rlm.infer; everything else goes to inference.
Inference
Serves rlm.infer, rlm.load and rlm.frames for one lane; everything else goes to inner.
MemoryTraceStore
Traces in memory (createMemoryFrameStore): a session without durable storage.
ModelSettings
The saved model settings (/settings -> Models), read for every request so a change applies to the next frame.

Enums§

FrameGate
What the tree-limit gate decided for a frame’s request.

Constants§

INFERENCE_PROMPT
The bounded-inference API description for the model’s runtime guide.

Traits§

FrameTasks
The task host frames are admitted through (NativeHostApi): rlm-frame@1 tasks in the session’s task journal, the usage ledger they charge, and the lanes that frames with depth run their rlm cells on.
TraceStore
Where frame traces live (FrameTraceStore): one document per key, replaced whole.

Functions§

contract_errors
typebox Errors(contract, value) as parseReply words them: "<instancePath or value> <message>".
frame_env
The variables frames read (the frame and review models, the map token cap), with their values in var.

Type Aliases§

FrameMeter
A frame conversation’s context use, once it has one.
ModelResolver
Resolves a provider/model a frame names (model=, rlm.frameModel) to the model and the key its requests carry, as the session resolves a lane’s model; an error refuses the frame.