systemprompt-ai 0.21.1

Provider-agnostic LLM integration for systemprompt.io AI governance — Anthropic, OpenAI, Gemini, and local models unified behind one governed pipeline with cost tracking and audit.
Documentation

Production infrastructure for AI agents

Website · Documentation · Guides · Core · Template · Discord

systemprompt-ai

Crates.io Docs.rs License: BSL-1.1 codecov

Every model call through one audited path. Anthropic, OpenAI, and Gemini answer to one governed pipeline, so the prompt, the tool call, the tokens, and the cost land in your database instead of a vendor's.

Layer: Domain — business-logic modules that implement systemprompt.io features. Part of the systemprompt-core workspace.

Overview

Integrations · Any AI Agent

Multi-provider AI integration with tool execution, structured output, and request tracking.

This module provides a unified interface for AI model interactions across multiple providers (Anthropic, OpenAI, Gemini) with support for:

  • Text generation and streaming
  • Tool/function calling with MCP integration
  • Structured JSON output with schema validation
  • Image generation
  • Google Search grounding
  • Request tracking and cost estimation

Architecture

This is an internal service library with no HTTP endpoints. AI capabilities are exposed through consuming modules (primarily the Agent module).

The AI module uses dependency injection for tool operations via the ToolProvider trait from systemprompt-traits. This allows:

  • MCP module provides McpToolProvider for MCP-based tools
  • AI module provides NoopToolProvider for non-tool use cases
  • Other implementations possible without modifying AI module

Usage

[dependencies]
systemprompt-ai = "0.21"

use systemprompt_ai::{AiService, AiRequest, AiMessage, NoopToolProvider};
use systemprompt_database::DbPool;
use std::sync::Arc;

let tool_provider = Arc::new(NoopToolProvider::new());
let ai_service = AiService::new(&db_pool, &registry, &ai_config, tool_provider, None)?;

let request = AiRequest::builder(
    vec![AiMessage::user("Hello!")],
    "gemini",
    "gemini-2.5-flash",
    8192,
    context,
)
.build();

let response = ai_service.generate(&request).await?;

Module Layout

Module Purpose
models/ Unified AiRequest/AiResponse types plus provider-specific request and response DTOs for Anthropic, OpenAI, and Gemini.
repository/ Compile-time-verified persistence for requests, messages, tool calls, request/response payloads, quota buckets, gateway policies, and safety findings.
services/core/ AiService (top-level orchestration) and ImageService, with request storage and structured logging.
services/providers/ Per-provider implementations of the AiProvider trait for Anthropic, OpenAI, and Gemini, plus the image providers.
services/gateway/ Governance policy ingestion, safety scanning, route selection, and system-prompt overrides. Re-exported at the crate root.
services/schema/ Tool-schema transformation, including discriminated-union splitting for providers that reject anyOf.
services/tooled/ Tool-execution orchestration: runs calls through the injected ToolProvider and synthesizes responses.
services/structured_output/ JSON extraction and schema validation for structured output.
services/storage/ ImageStorage local blob storage for generated images.
services/config/ ConfigValidator for AI configuration.

Database

Tables: ai_requests, ai_request_messages, ai_request_tool_calls, ai_request_payloads, ai_quota_buckets, ai_gateway_policies, ai_safety_findings.

License

BSL-1.1 (Business Source License). Source-available for evaluation, testing, and non-production use. Production use requires a commercial license. Each version converts to Apache 2.0 four years after publication. See LICENSE.

systemprompt.io · Documentation · Guides · Live Demo · Template · crates.io · docs.rs · Discord

Domain layer · Own how your organization uses AI.