Expand description
The AI gateway: a protocol-translating proxy in front of upstream LLM providers.
Inbound requests in one wire protocol (Anthropic Messages, OpenAI
Responses) are parsed into a canonical form, dispatched to an upstream
provider via the protocol adapters, and rendered back in the caller’s
protocol. GatewayService orchestrates the flow; supporting modules cover
policy resolution, quota enforcement, safety scanning, usage
captures, pricing, the upstream and safety-scanner registry, and
the audit trail.
The safety-scanner contract —
SafetyScanner,
Finding, the built-in scanners, and
register_safety_scanner! —
lives in systemprompt-ai; registry::SafetyScannerRegistry resolves the
scanner names a policy selects against the built-ins plus any extension
registrations.
Re-exports§
pub use audit::GatewayAudit;pub use audit::GatewayRequestContext;pub use captures::CapturedToolUse;pub use captures::CapturedUsage;pub use protocol::InboundAdapter;pub use protocol::OutboundAdapter;pub use protocol::OutboundAdapterRegistration;pub use protocol::OutboundCtx;pub use protocol::OutboundOutcome;pub use registry::GatewayUpstreamRegistry;pub use registry::SafetyScannerRegistry;pub use service::DispatchInputs;pub use service::GatewayService;pub use service::REQUEST_ID_HEADER;
Modules§
- audit
- Persistence of gateway request lifecycle to the AI-request audit trail.
- captures
- parse
- policy
- Resolution and caching of the effective gateway policy.
- pricing
- Pricing resolution for gateway requests.
- protocol
- Protocol translation between caller and upstream LLM wire formats.
- quota
- registry
- service
- Gateway dispatch entry point: route resolution, policy and quota checks, upstream send, and response finalization.
- stream_
tap - Streaming response tap: re-renders upstream canonical events to the inbound wire format while accumulating a full response snapshot for the audit sink.