Skip to main content

generate_streaming

Function generate_streaming 

Source
pub async fn generate_streaming(
    client: &dyn LlmClient,
    req: &StructuredRequest,
    on_partial: PartialObjectCallback,
) -> Result<StructuredResult>
Expand description

Generate a structured JSON object with streaming partial updates.

Calls on_partial with progressively more complete partial objects as tokens arrive. Returns the final validated object.

In streaming mode, max_repair_attempts defaults to 0 because a repair would reset the partial object stream (confusing for consumers).