siumai-extras 0.11.0-beta.6

Optional utilities for siumai: schema validation, telemetry, and server adapters
Documentation

Siumai Extras

Optional utilities for the siumai LLM library.

Features

This crate provides optional functionality that extends siumai without adding heavy dependencies to the core library:

  • schema - JSON Schema validation for structured outputs
  • telemetry - Advanced tracing and logging with tracing-subscriber
  • server - Server adapters for Axum and other web frameworks
  • mcp - MCP (Model Context Protocol) integration for dynamic tool discovery
  • all - Enable all features

Installation

[dependencies]

siumai = "0.11.0-beta.6"

siumai-extras = { version = "0.11.0-beta.6", features = ["schema", "telemetry", "mcp"] }

Usage

Orchestrator and high-level object helpers do not require any extra features. Schema validation and tracing are opt-in via the schema and telemetry features.

High-level structured objects

Provider-agnostic helpers for generating typed JSON objects:

use serde::Deserialize;
use siumai::prelude::unified::*;
use siumai_extras::highlevel::object::{generate_object, GenerateObjectOptions};

#[derive(Deserialize, Debug)]
struct Post { title: String }

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let reg = registry::global();
    let model = reg.language_model("openai:gpt-4o-mini")?;

    let (post, _resp) = generate_object::<Post>(
        &model,
        vec![user!("Return JSON: {\"title\":\"hi\"}")],
        None,
        GenerateObjectOptions::default(),
    )
    .await?;

    println!("{}", post.title);
    Ok(())
}

If you enable the schema feature, GenerateObjectOptions::schema is validated via siumai_extras::schema before deserializing into T.

Orchestrator & agents

Multi-step tool calling, agents, and stop conditions:

use serde_json::json;
use siumai::prelude::unified::*;
use siumai_extras::orchestrator::{
    ToolLoopAgent, ToolResolver, ToolChoice, step_count_is,
};

struct SimpleResolver;

#[async_trait::async_trait]
impl ToolResolver for SimpleResolver {
    async fn call_tool(
        &self,
        name: &str,
        args: serde_json::Value,
    ) -> Result<serde_json::Value, siumai::error::LlmError> {
        match name {
            "get_weather" => {
                let city = args.get("city").and_then(|v| v.as_str()).unwrap_or("Unknown");
                Ok(json!({ "city": city, "temperature": 72, "condition": "sunny" }))
            }
            _ => Err(siumai::error::LlmError::InternalError(format!(
                "Unknown tool: {}",
                name
            ))),
        }
    }
}

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let reg = registry::global();
    let client = reg.language_model("openai:gpt-4o-mini")?;

    let weather_tool = Tool::function(
        "get_weather",
        "Get weather for a city",
        json!({
            "type": "object",
            "properties": { "city": { "type": "string" } },
            "required": ["city"]
        }),
    );

    let agent = ToolLoopAgent::new(client, vec![weather_tool], vec![step_count_is(10)])
        .with_system("You are a helpful assistant.")
        .with_tool_choice(ToolChoice::Required);

    let messages = vec![ChatMessage::user("What's the weather in Tokyo?").build()];
    let resolver = SimpleResolver;

    let result = agent.generate(messages, &resolver).await?;
    println!("Answer: {}", result.text().unwrap_or_default());
    Ok(())
}

You can attach telemetry to the agent or orchestrator using siumai::experimental::observability::telemetry::TelemetryConfig:

use siumai::experimental::observability::telemetry::TelemetryConfig;
use siumai_extras::orchestrator::OrchestratorBuilder;

let telemetry = TelemetryConfig::builder()
    .record_inputs(false)
    .record_outputs(false)
    .record_usage(true)
    .build();

let builder = OrchestratorBuilder::new().telemetry(telemetry);

Schema Validation

use siumai_extras::schema::SchemaValidator;

// Validate JSON against a schema
let validator = SchemaValidator::new(schema)?;
validator.validate(&json_value)?;

Telemetry

use siumai_extras::telemetry::init_subscriber;

// Initialize tracing subscriber
init_subscriber(config)?;

Server Adapters

use siumai_extras::server::axum::to_sse_response;

// Convert ChatStream to Axum SSE response
let sse = to_sse_response(stream, options);

If you are building an OpenAI-compatible gateway and need to output OpenAI Responses SSE, siumai-extras also provides a helper that:

  • bridges provider-specific ChatStreamEvent::Custom parts into openai:* stream parts, and
  • serializes the stream into OpenAI Responses SSE frames.
use axum::response::Response;
use axum::body::Body;
use siumai_extras::server::axum::to_openai_responses_sse_response;
use siumai::prelude::unified::ChatStream;

fn handler(stream: ChatStream) -> Response<Body> {
    to_openai_responses_sse_response(stream)
}

See the runnable example: siumai-extras/examples/openai-responses-gateway.rs (streaming + non-streaming). For custom conversion hooks, see: siumai-extras/examples/gateway-custom-transform.rs. For request-normalization bridge demos, see:

  • siumai-extras/examples/anthropic-to-openai-responses-gateway.rs
  • siumai-extras/examples/openai-responses-to-anthropic-gateway.rs For custom lossy-policy handling, see:
  • siumai-extras/examples/gateway-loss-policy.rs

If you need to expose multiple downstream protocol surfaces from the same upstream stream, use the transcoder helper:

use axum::{body::Body, response::Response};
use siumai::experimental::bridge::BridgeTarget;
use siumai::prelude::unified::ChatStream;
use siumai_extras::server::axum::{
    TargetSseFormat, TranscodeSseOptions, to_transcoded_sse_response,
};

fn handler(stream: ChatStream) -> Response<Body> {
    to_transcoded_sse_response(
        stream,
        TargetSseFormat::OpenAiResponses,
        TranscodeSseOptions::strict().with_bridge_source(BridgeTarget::AnthropicMessages),
    )
}

When a streaming gateway route is cross-protocol and you want strict inspected rejection or a custom BridgeLossPolicy, declare the upstream protocol with TranscodeSseOptions::with_bridge_source(...). That enables the same source-aware loss-policy decision path used by the lower-level core stream bridge helpers while keeping the Axum helper surface.

If your gateway route also needs to read downstream request bodies or buffered upstream bodies under GatewayBridgePolicy, use the Axum runtime helpers instead of open-coding to_bytes(...):

use axum::body::Body;
use siumai_extras::server::{GatewayBridgePolicy};
use siumai_extras::server::axum::read_request_json_with_policy;

let policy = GatewayBridgePolicy::default().with_request_body_limit_bytes(128 * 1024);
let request_json: serde_json::Value =
    read_request_json_with_policy(Body::from(r#"{"input":"hello"}"#), &policy).await?;

If you need to customize the conversion logic, the recommended path is GatewayBridgePolicy + BridgeOptions + typed bridge hooks as demonstrated in siumai-extras/examples/gateway-custom-transform.rs.

If the request-side requirement is hosted-tool compatibility across protocols, prefer ProviderToolRewriteCustomization attached through GatewayBridgePolicy::with_customization(...) or NormalizeRequestOptions::with_bridge_customization(...) instead of patching raw downstream JSON. The Anthropic -> OpenAI gateway example demonstrates that path for anthropic.web_fetch_20250910 -> openai.web_search.

The two request-normalization bridge demos intentionally show a different path:

  • source protocol request JSON -> explicit request normalizer -> ChatRequest
  • execute on a fixed upstream model handle
  • transcode the resulting unified response/stream back into the chosen target protocol

That is useful when you want the bridge surface to stay explicit and testable instead of hiding protocol translation inside route-local JSON glue.

Migration guidance for gateway routes now lives at:

  • docs/workstreams/protocol-bridge-gateway/migration.md

Recommended route shapes now live at:

  • docs/workstreams/protocol-bridge-gateway/route-recipes.md

That recipes note covers the currently recommended and test-backed gateway compositions for:

  • provider-native ingress -> normalized runtime -> downstream JSON/SSE
  • buffered upstream proxy/runtime routes
  • cross-protocol SSE with inspected strict rejection
  • hosted-tool compatibility via typed request customization

The raw event-transform helper is still available as an escape hatch:

use axum::{body::Body, response::Response};
use siumai::prelude::unified::{ChatStream, ChatStreamEvent};
use siumai_extras::server::axum::{
    TargetSseFormat, TranscodeSseOptions, to_transcoded_sse_response_with_transform,
};

fn handler(stream: ChatStream) -> Response<Body> {
    to_transcoded_sse_response_with_transform(
        stream,
        TargetSseFormat::OpenAiResponses,
        TranscodeSseOptions::strict(),
        |ev: ChatStreamEvent| vec![ev],
    )
}

For non-streaming gateways, you can also transcode a ChatResponse into a provider-native JSON response body:

use axum::{body::Body, response::Response};
use siumai::prelude::*;
use siumai_extras::server::axum::{
    TargetJsonFormat, TranscodeJsonOptions, to_transcoded_json_response,
};

fn handler(resp: ChatResponse) -> Response<Body> {
    to_transcoded_json_response(resp, TargetJsonFormat::OpenAiResponses, TranscodeJsonOptions::default())
}

If you want to customize conversion for non-streaming responses, prefer the response-level transform hook (no JSON parse/round-trip):

use axum::{body::Body, response::Response};
use siumai::prelude::*;
use siumai_extras::server::axum::{
    TargetJsonFormat, TranscodeJsonOptions, to_transcoded_json_response_with_response_transform,
};

fn handler(resp: ChatResponse) -> Response<Body> {
    to_transcoded_json_response_with_response_transform(
        resp,
        TargetJsonFormat::OpenAiResponses,
        TranscodeJsonOptions::default(),
        |r| {
            r.content = MessageContent::Text("[REDACTED]".to_string());
        },
    )
}

MCP Integration

use siumai::prelude::unified::*;
use siumai_extras::mcp::mcp_tools_from_stdio;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let reg = registry::global();
    let model = reg.language_model("openai:gpt-4o-mini")?;
    let messages = vec![user!("Use the available tools to help me")];

    // Connect to MCP server and get tools
    let (tools, resolver) = mcp_tools_from_stdio("node mcp-server.js").await?;

    // Use with Siumai orchestrator (from siumai-extras)
    let (response, _) = siumai_extras::orchestrator::generate(
        &model,
        messages,
        Some(tools),
        Some(&resolver),
        vec![siumai_extras::orchestrator::step_count_is(10)],
        Default::default(),
    )
    .await?;

    println!("{}", response.content_text().unwrap_or_default());
    Ok(())
}

Documentation

License

Licensed under either of:

at your option.