Struct AxumServer 

pub struct AxumServer { /* private fields */ }

Axum-based server implementation.

The server is built around AppState, which holds an optional engine per modality. Handlers fault to 503 when the modality they need isn’t loaded, instead of running stub error logic.

Implementations

impl AxumServer

pub fn from_state(state: AppState) -> Self

Create a server with a fully populated AppState.

pub fn from_llm(engine: Arc<dyn LlmInferenceEngine + Send + Sync>) -> Self

Convenience constructor for an LLM-only server (chat / completions).

pub fn from_embed(engine: Arc<dyn EmbedEngine + Send + Sync>) -> Self

Convenience constructor for an embedding-only server (/v1/embeddings).

pub fn from_transcribe(engine: Arc<dyn TranscribeEngine + Send + Sync>) -> Self

Convenience constructor for a transcription-only server (/v1/audio/transcriptions).

pub fn from_tts(engine: Arc<dyn TtsEngine + Send + Sync>) -> Self

Convenience constructor for a TTS-only server (/v1/audio/speech).

pub fn with_auto_config(self, auto_config: ResolvedFerrumConfig) -> Self

Attach the startup auto-configuration decision trace exposed by /health. Constructors keep this optional so tests and non-LLM deployments can use the server without a model-specific resolver.

pub fn with_prompt_template( self, prompt_template: Option<ModelChatTemplate>, ) -> Self

Attach the loaded model’s prompt template, if available. This keeps OpenAI request aliases from driving prompt-family selection.

pub fn with_default_enable_thinking(self, enable_thinking: Option<bool>) -> Self

Set the server-wide reasoning default for requests that omit chat_template_kwargs.enable_thinking. Per-request values still win.

pub fn with_served_model_registry(self, registry: ServedModelRegistry) -> Self

Install the public OpenAI model namespace used for request routing and /v1/models. The registry keeps public aliases separate from the engine’s internal model id.

pub fn with_lora_adapters( self, base_model_id: impl Into<String>, adapters: Vec<LoraAdapterModel>, ) -> Result<Self>

Attach startup-loaded LoRA adapter model ids.

Trait Implementations

impl HttpServer for AxumServer

fn start<'life0, 'life1, 'async_trait>( &'life0 self, config: &'life1 ServerConfig, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Start the server

fn stop<'life0, 'async_trait>( &'life0 self, timeout: Duration, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Stop the server gracefully

fn is_running(&self) -> bool

Check if server is running

fn address(&self) -> Option<SocketAddr>

Get server address

fn register_handler( &mut self, _path: &str, _method: HttpMethod, _handler: Box<dyn RequestHandler>, )

Register a request handler

fn register_middleware(&mut self, _middleware: Box<dyn Middleware>)

Register middleware

fn get_metrics(&self) -> ServerMetrics

Get server metrics

fn health_check<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = HealthStatus> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Health check