Module generation 

Portable generation, messages, function continuation, and streaming values.

Construct a GenerateRequest for an already selected crate::ModelBackend. Returned assistant Message values retain the exact associated continuation; append them unchanged, then append correlated FunctionResult values in a tool message. providers-kit declares functions but never executes them.

GenerateResponse is authoritative at completion. GenerateEventStream provides bounded incremental observation from supplied Models; partial text or hosted projections cannot reconstruct opaque replay state. Serialization rejects messages carrying continuation instead of silently dropping it.

Consume a stream and retain the authoritative response

This requires OpenAI credentials and access to the exact selected text model. A complete stream contains one Completed; earlier text is only incremental observation. Keep the returned assistant message, not a reconstruction from deltas. A terminal reason such as Length or ContentFilter is not an ordinary successful answer. This example accepts only Stop and does not compare wording to its prompt.

use futures_util::StreamExt;
use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId};
use providers_kit::config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig};
use providers_kit::generation::{ContentPart, FinishReason, GenerateEvent, GenerateRequest, Message};
let provider = Provider::from_config(ProviderConfig::new(
    ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)),
    TransportConfig::default(),
))?;
let work = async {
    let model = provider.model(ModelId::new("gpt-6-luna")?)?;
    let request = GenerateRequest::new(vec![Message::user_text("Explain a rainbow briefly.")?])?;
    let mut stream = model.generate_stream(&request).await?;
    let mut text = String::new();
    let mut completed = None;
    while let Some(event) = stream.next().await {
        match event? {
            GenerateEvent::Started { model: returned } => assert_eq!(returned, *model.model_id()),
            GenerateEvent::ContentDelta { delta } => text.push_str(&delta),
            GenerateEvent::Completed(response) => {
                assert!(completed.is_none(), "duplicate completion");
                completed = Some(response);
            }
            GenerateEvent::ThinkingDelta { .. } | GenerateEvent::HostedTool(_) | GenerateEvent::Usage(_) => {}
        }
    }
    let response = completed.ok_or("stream ended before Completed")?;
    assert_eq!(response.model, *model.model_id());
    assert_eq!(response.finish_reason, Some(FinishReason::Stop));
    let final_text: String = response.message.content().iter().filter_map(|part| match part {
        ContentPart::Text { text } => Some(text.as_str()), _ => None,
    }).collect();
    assert!(!final_text.trim().is_empty());
    assert_eq!(text, final_text); // Fidelity within Kit output, not native parity.
    Ok::<_, Box<dyn std::error::Error>>(response.message)
}.await;
let cleanup = provider.shutdown().await;
if let Err(error) = &cleanup {
    eprintln!("Cleanup failed: {error}");
}
let _unchanged_assistant = work?;
cleanup?;

Correlated function continuation

providers-kit declares functions; the application executes them. This bounded two-generation example requires the selected OpenAI model, credentials, and function support. Validate the call before dispatching application behavior. The result copies its exact id/name, and the assistant is moved unchanged into history. It includes opaque reasoning/replay state that visible text cannot reconstruct.

use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId};
use providers_kit::config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig};
use providers_kit::generation::{ContentPart, FinishReason, FunctionResult, GenerateRequest,
    GenerationConfig, JsonSchema, Message, ProviderFunctionTool, ProviderToolSpec, ToolChoice};
let provider = Provider::from_config(ProviderConfig::new(
    ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)),
    TransportConfig::default(),
))?;
let work = async {
    let model = provider.model(ModelId::new("gpt-6-luna")?)?;
    let lookup = ProviderFunctionTool::new("lookup", Some("Read the local item".into()),
        JsonSchema::new(serde_json::json!({"type":"object","properties":{},"additionalProperties":false}))?)?;
    let initial = Message::user_text("Look up the local item and summarize it.")?;
    let request = GenerateRequest::new(vec![initial.clone()])?
        .with_tools(vec![ProviderToolSpec::Function(lookup)])?
        .with_generation(GenerationConfig::default().with_tool_choice(ToolChoice::Required))?;
    let first = model.generate(&request).await?;
    assert_eq!(first.model, *model.model_id());
    assert_eq!(first.finish_reason, Some(FinishReason::ToolCall));
    let [call] = first.message.function_calls() else { return Err("expected one lookup".into()); };
    assert_eq!(call.name(), "lookup");
    assert_eq!(call.arguments(), &serde_json::json!({}));
    let local_item = serde_json::json!({"name":"paper","quantity":3});
    let result = FunctionResult::new(call.id().map(str::to_owned), call.name(),
        vec![ContentPart::Json { value: local_item }], false)?;
    assert_eq!(result.call_id(), call.id());
    let followup = GenerateRequest::new(vec![initial, first.message, Message::tool(vec![result])?])?;
    let second = model.generate(&followup).await?;
    assert_eq!(second.model, *model.model_id());
    assert_eq!(second.finish_reason, Some(FinishReason::Stop));
    assert!(second.message.function_calls().is_empty());
    assert!(second.message.content().iter().any(|part| matches!(part,
        ContentPart::Text { text } if !text.trim().is_empty())));
    Ok::<(), Box<dyn std::error::Error>>(())
}.await;
let cleanup = provider.shutdown().await;
if let Err(error) = &cleanup {
    eprintln!("Cleanup failed: {error}");
}
work?;
cleanup?;

Structured output and reasoning

Request shape is local and testable. Model compatibility is checked when the request executes: an omitted reasoning override remains omitted; unsupported sampling/reasoning/schema combinations fail rather than silently changing settings. A schema constrains shape; ordinary prompt wording is not an equality guarantee.

use providers_kit::generation::{GenerateRequest, GenerationConfig, JsonSchema, Message,
    ReasoningConfig, ReasoningEffort, ResponseFormat};
let schema = JsonSchema::new(serde_json::json!({
    "type":"object", "properties":{"summary":{"type":"string"}},
    "required":["summary"], "additionalProperties":false
}))?;
let request = GenerateRequest::<()>::new(vec![Message::user_text("Summarize the supplied note.")?])?
    .with_response_format(ResponseFormat::JsonSchema(schema))
    .with_reasoning(ReasoningConfig { effort: Some(ReasoningEffort::Low), ..Default::default() })
    .with_generation(GenerationConfig::default()
        .with_max_output_tokens(std::num::NonZeroU32::new(4096).unwrap()))?;
assert!(matches!(request.response_format(), Some(ResponseFormat::JsonSchema(_))));
assert_eq!(request.reasoning().unwrap().effort, Some(ReasoningEffort::Low));
assert!(request.generation().temperature().is_none());

The same request pattern can be executed on an exact supporting model. This requires credentials and current access to gpt-6-luna; no sampling override is inferred from the reasoning setting. Inspect both terminal outcome and the returned JSON rather than assuming the prompt’s prose guarantees a value.

use providers_kit::{Provider, ProviderBackend, ProviderConfig, ModelBackend, ModelId};
use providers_kit::config::{ApiKey, OpenAiConfig, ProviderKind, TransportConfig};
use providers_kit::generation::{ContentPart, FinishReason, GenerateRequest, JsonSchema,
    Message, ReasoningConfig, ReasoningEffort, ResponseFormat};
let provider = Provider::from_config(ProviderConfig::new(
    ProviderKind::OpenAi(OpenAiConfig::new(ApiKey::new("replace-with-your-key")?)),
    TransportConfig::default(),
))?;
let work = async {
    let model = provider.model(ModelId::new("gpt-6-luna")?)?;
    let schema = JsonSchema::new(serde_json::json!({"type":"object",
        "properties":{"summary":{"type":"string"}}, "required":["summary"], "additionalProperties":false}))?;
    let request = GenerateRequest::new(vec![Message::user_text("Summarize: rain supports plant growth.")?])?
        .with_response_format(ResponseFormat::JsonSchema(schema))
        .with_reasoning(ReasoningConfig { effort: Some(ReasoningEffort::Low), ..Default::default() });
    let response = model.generate(&request).await?;
    assert_eq!(response.model, *model.model_id());
    assert_eq!(response.finish_reason, Some(FinishReason::Stop));
    let text: String = response.message.content().iter().filter_map(|part| match part {
        ContentPart::Text { text } => Some(text.as_str()), _ => None,
    }).collect();
    let value: serde_json::Value = serde_json::from_str(&text)?;
    assert!(value["summary"].is_string());
    assert_eq!(value.as_object().ok_or("expected an object")?.len(), 1);
    Ok::<(), Box<dyn std::error::Error>>(())
}.await;
let cleanup = provider.shutdown().await;
if let Err(error) = &cleanup {
    eprintln!("Cleanup failed: {error}");
}
work?;
cleanup?;

Structs

FunctionCall

Validated client function call requested by a model.

FunctionResult

Validated caller-supplied result for a client function call.

GenerateEventStream

Value-erased stream of normalized generation events.

GenerateRequest

Validated model-bound generation request.

GenerateResponse

Final normalized generation result.

GenerationConfig

Optional sampling and tool-choice overrides.

GenerationContinuation

Opaque provider-authored state required to continue one model turn.

JsonSchema

Validated strict object JSON schema.

Message

Portable message with private function and continuation state.

ProviderFunctionTool

Validated client-executed function declaration shown to a model.

ReasoningConfig

Optional reasoning controls attached to a generation request.

Usage

Token accounting reported by a provider response.

Enums

AllowedToolMode

Policy for how strongly the allowed-tool list should be enforced.

Artifact

Artifact returned by generation or hosted execution.

Citation

Citation attached to generated output.

ContentPart

Ordered portable message content.

FileReference

Reference that can be submitted as provider-managed or remote model input.

FinishReason

Portable completion reason.

GenerateEvent

Incremental generation event.

ProviderToolSpec

Tool declaration supplied to a generation request.

ReasoningEffort

Portable reasoning effort hint for providers that support explicit thinking controls.

ReasoningSummaryMode

Controls how much reasoning summary a provider should return.

ResponseFormat

Requested output format for a generation request.

Role

Portable message role.

ToolChoice

Portable tool-selection policy for a request.