Skip to main content

StreamingStructuredOutputExt

Trait StreamingStructuredOutputExt 

Source
pub trait StreamingStructuredOutputExt: BaseChatModel {
    // Provided method
    fn stream_structured_output<'life0, 'life1, 'async_trait, T>(
        &'life0 self,
        schema: Value,
        prompt: &'life1 str,
    ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<T, StructuredOutputError>> + Send>>, StructuredOutputError>> + Send + 'async_trait>>
       where T: DeserializeOwned + Serialize + Clone + PartialEq + Unpin + Send + Sync + 'static + 'async_trait,
             Self: Sync + 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait { ... }
}
Expand description

Trait that extends BaseChatModel with streaming structured output capabilities.

Provides a streaming variant of with_structured_output that yields partial T values as the model generates tokens. The target type T should derive #[serde(default)] or use Option fields so that partial JSON can be deserialized with missing fields filled by defaults.

Provided Methods§

Source

fn stream_structured_output<'life0, 'life1, 'async_trait, T>( &'life0 self, schema: Value, prompt: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Pin<Box<dyn Stream<Item = Result<T, StructuredOutputError>> + Send>>, StructuredOutputError>> + Send + 'async_trait>>
where T: DeserializeOwned + Serialize + Clone + PartialEq + Unpin + Send + Sync + 'static + 'async_trait, Self: Sync + 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Stream structured output from a chat model.

Returns a stream of T values. Each item represents the best partial result that could be parsed from the tokens received so far. The final item in the stream is the complete result.

§Arguments
  • schema - A JSON Schema describing the expected output shape.
  • prompt - The user prompt to send to the LLM.
§Returns

A stream of Result<T, StructuredOutputError> items.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§

Source§

impl<M: BaseChatModel> StreamingStructuredOutputExt for M

Blanket implementation: every BaseChatModel automatically gets StreamingStructuredOutputExt.