Module server 

OpenAI-compatible chat completions server.

Provides an Axum-based HTTP server with the following endpoints:

Method	Path	Description
POST	/v1/chat/completions	Chat completion (streaming and non-streaming)
GET	/v1/models	List available models
GET	/health	Liveness probe
GET	/metrics	Prometheus text exposition

Use create_router or create_router_with_metrics to build the Axum router, then serve it with axum::serve.

Structs

AppState

Server state.

ChatChoice

A choice in the completion response.

ChatCompletionRequest

Chat completion request.

ChatCompletionResponse

Chat completion response.

ChatMessage

Chat message (OpenAI-compatible).

QueueDepthTracker

Request queue depth tracker.

ServerConfig

Server configuration with request management.

Usage

Token usage info.

Constants

REQUEST_ID_HEADER

Header name used for end-to-end request correlation. Request handlers echo whatever the client supplied in the response, or generate a fresh UUIDv4-style id when the header is absent.

Functions

create_router

Create the Axum router.

create_router_with_metrics

Create the Axum router with a shared metrics instance.

create_router_with_pool

Create the Axum router from a pre-built EnginePool.

create_server

Create the full server setup: router + graceful shutdown future.

request_id_header_map

Build response headers for a RequestId. Returns a HeaderMap with the X-Request-ID set to the canonical 36-char UUID form.

resolve_request_id

Resolve a RequestId from an incoming request header, falling back to a freshly generated id when none is supplied or when the supplied value is malformed (in either case we still want a usable id to thread through tracing spans and the response).

serve_with_shutdown

Start server with graceful shutdown support.

shutdown_signal

Create a shutdown signal that responds to SIGTERM and SIGINT (Ctrl+C).