Skip to main content

stream_structured_output

Function stream_structured_output 

Source
pub async fn stream_structured_output<T, M>(
    llm: &M,
    schema: Value,
    prompt: &str,
) -> Result<Pin<Box<dyn Stream<Item = Result<T, StructuredOutputError>> + Send>>, StructuredOutputError>
where T: DeserializeOwned + Serialize + Clone + PartialEq + Unpin + Send + Sync + 'static, M: BaseChatModel + ?Sized,
Expand description

Stream structured output from a chat model.

Returns a stream of partial T values as the model generates tokens. The implementation:

  1. Calls stream_chat with a prompt that asks for JSON matching the schema
  2. Accumulates tokens through the PartialJsonParser
  3. At each successful parse, yields a T value (partial fields filled by serde defaults, rest default)
  4. On stream end, yields the final complete T

§Arguments

  • llm - Any type implementing BaseChatModel.
  • schema - A JSON Schema describing the expected output.
  • prompt - The user prompt to send to the LLM.

§Returns

A Result containing a stream of Result<T, StructuredOutputError> items, or a StructuredOutputError if the stream could not be set up.

§Type requirements

The target type T should use #[serde(default)] or Option fields so that partial JSON can be deserialized with missing fields filled by defaults. If T does not support default deserialization, partial results will fail and only the final complete result will be yielded.

§Example

use serde::{Deserialize, Serialize};
use langchainrust::core::structured_output::stream_structured_output;
use futures_util::StreamExt;

#[derive(Debug, Deserialize, Serialize, Clone)]
#[serde(default)]
struct Person {
    name: String,
    age: u32,
}

impl Default for Person {
    fn default() -> Self {
        Self { name: String::new(), age: 0 }
    }
}

let mut stream = stream_structured_output::<Person, _>(
    &llm, schema, "Tell me about Alice"
).await?;
while let Some(result) = stream.next().await {
    let person = result?;
    println!("Partial: name={}, age={}", person.name, person.age);
}