Trait LlmCodec

Source

pub trait LlmCodec: Send + Sync {
    // Required methods
    fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest>;
    fn encode(
        &self,
        annotated: &AnnotatedLlmRequest,
        original: &LlmRequest,
    ) -> Result<LlmRequest>;

    // Provided method
    fn codec_identity(&self) -> LlmCodecIdentity { ... }
}

Expand description

A bidirectional translator between opaque LlmRequest content and structured AnnotatedLlmRequest.

Codecs are implemented by framework integrations and provider adapters because each SDK has its own request format. A codec is supplied per call by the caller; the built-in provider codecs can also be selected from a raw payload via crate::codec::resolve.

§Design

Synchronous: decode/encode are pure data transforms (JSON restructuring), not I/O operations. This matches existing guardrails and request intercepts.
Send + Sync: Required because NemoRelayContextState is behind Arc<RwLock<>> and accessed from async contexts.
Trait object: Codecs are registered at runtime by callers or bindings, so the Rust core cannot know concrete types at compile time. Store as Arc<dyn LlmCodec>.

Required Methods§

Source

fn decode(&self, request: &LlmRequest) -> Result<AnnotatedLlmRequest>

Parse opaque request content into structured form.

Source

fn encode( &self, annotated: &AnnotatedLlmRequest, original: &LlmRequest, ) -> Result<LlmRequest>

Merge structured changes back into the opaque request.

The original parameter is the pre-intercept LlmRequest, used to preserve fields that the Codec does not structurally model. Implementations MUST use merge-not-replace semantics: overlay structured changes onto the original content, do not construct a fresh content object.

Provided Methods§

Source

fn codec_identity(&self) -> LlmCodecIdentity

Return this codec’s identity for LLM sanitizer context.

Custom codecs should keep the default LlmCodecIdentity::Opaque unless they have a stable runtime registration ID. Callers must not infer a provider surface from an opaque implementation or request shape.