Skip to main content

LlmResponseCodec

nemo_relay::codec::traits

Trait LlmResponseCodec 

Source 
pub trait LlmResponseCodec: Send + Sync {
    // Required method
    fn decode_response(&self, response: &Json) -> Result<AnnotatedLlmResponse>;

    // Provided method
    fn codec_identity(&self) -> LlmCodecIdentity { ... }
}
Expand description

Decode-only codec for LLM API responses.

Unlike LlmCodec (which is bidirectional for requests), response codecs are introspection-only: they parse a raw response into structured form but never need to encode back. This matches the pipeline design where responses are observed, not modified.

§Design

  • Synchronous: decode_response is a pure data transform (JSON parsing), not an I/O operation.
  • Send + Sync: Required for storage in Arc behind RwLock.
  • Trait object: Codecs are registered at runtime, stored as Arc<dyn LlmResponseCodec>.
  • Fallible: Returns Result; managed call sites may omit annotations on decode failure, while manual lifecycle bindings may surface the error.

§Two-Phase Decode

Implementations should use a two-phase decode pattern:

  1. Deserialize raw JSON into API-specific intermediate structs
  2. Map intermediate structs into the normalized AnnotatedLlmResponse

Required Methods§

Source

fn decode_response(&self, response: &Json) -> Result<AnnotatedLlmResponse>

Parse a raw JSON response into normalized structured form.

Implementations should return Err only for genuinely unparseable input.

Provided Methods§

Source

fn codec_identity(&self) -> LlmCodecIdentity

Return this codec’s identity for LLM sanitizer context.

Custom codecs should keep the default LlmCodecIdentity::Opaque unless they have a stable runtime registration ID.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§