Skip to main content

LlmResponseCodec

nemo_flow::codec::traits

Trait LlmResponseCodec 

Source 
pub trait LlmResponseCodec: Send + Sync {
    // Required method
    fn decode_response(&self, response: &Json) -> Result<AnnotatedLlmResponse>;
}
Expand description

Decode-only codec for LLM API responses.

Unlike LlmCodec (which is bidirectional for requests), response codecs are introspection-only: they parse a raw response into structured form but never need to encode back. This matches the pipeline design where responses are observed, not modified.

§Design

  • Synchronous: decode_response is a pure data transform (JSON parsing), not an I/O operation.
  • Send + Sync: Required for storage in Arc behind RwLock.
  • Trait object: Codecs are registered at runtime, stored as Arc<dyn LlmResponseCodec>.
  • Fallible: Returns Result; managed call sites may omit annotations on decode failure, while manual lifecycle bindings may surface the error.

§Two-Phase Decode

Implementations should use a two-phase decode pattern:

  1. Deserialize raw JSON into API-specific intermediate structs
  2. Map intermediate structs into the normalized AnnotatedLlmResponse

Required Methods§

Source

fn decode_response(&self, response: &Json) -> Result<AnnotatedLlmResponse>

Parse a raw JSON response into normalized structured form.

Implementations should return Err only for genuinely unparseable input.

Implementors§