Crate synapse 

synapse — LLM router and gateway.

Modules

config

Layered runtime config (env + file paths). Env takes precedence.

embeddings

Embeddings engine: OpenAI-shaped types, the provider trait, and batch splitting.

error

Gateway error type → OpenAI-shaped JSON + HTTP status.

gateway

In-process LLM gateway: routing, fallback, native Vertex, ledger, metrics. Transport-independent core; the axum HTTP layer (server) delegates here.

guard

Configurable input guardrails backed by llm-guard. Per-route policy selection with fallback to a default policy.

ledger

Pluggable cost ledger. The hot path enqueues onto a bounded channel drained by a background writer; on a full channel we drop + count, never block.

observability

OpenLLMetry gen_ai.* span attributes + Prometheus metric emission. Both lanes emit the same shape so native-Vertex calls are not blind.

pricing

Static pricing table + cost calculation.

providers

Provider catalog: genai clients + circuit breakers, keyed by provider id.

resilience

Retries + circuit breakers for synapse outbound HTTP clients.

routing

Routing: request model, lane classification, route table, executor.

server

axum surface: a thin HTTP layer that delegates to the in-process Gateway.

vertex_native

Native Vertex REST lane: preserves cachedContents, gs:// media URIs, and strict responseSchema that the OpenAI-compatible standard lane cannot express.