Trait LlmCodec

Source

pub trait LlmCodec: Send + Sync {
    // Required methods
    fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest>;
    fn encode(
        &self,
        annotated: &AnnotatedLlmRequest,
        original: &LlmRequest,
    ) -> Result<LlmRequest>;
}

Expand description

A bidirectional translator between opaque LlmRequest content and structured AnnotatedLlmRequest.

Codecs are implemented by integration patches (LangChain, LangChain-NVIDIA, LangGraph, etc.) since each SDK has its own request format. They are registered by name in the global codec registry.

§Design

Synchronous: decode/encode are pure data transforms (JSON restructuring), not I/O operations. This matches existing guardrails and request intercepts.
Send + Sync: Required because NemoFlowContextState is behind Arc<RwLock<>> and accessed from async contexts.
Trait object: Codecs are registered at runtime (e.g., by Python patches), so the Rust core cannot know concrete types at compile time. Store as Arc<dyn LlmCodec>.

Required Methods§

Source

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest>

Parse opaque request content into structured form.

Source

fn encode( &self, annotated: &AnnotatedLlmRequest, original: &LlmRequest, ) -> Result<LlmRequest>

Merge structured changes back into the opaque request.

The original parameter is the pre-intercept LlmRequest, used to preserve fields that the Codec does not structurally model. Implementations MUST use merge-not-replace semantics: overlay structured changes onto the original content, do not construct a fresh content object.