Struct HTTPServer 

pub struct HTTPServer {
    pub host: String,
    pub port: u16,
    pub serialize_requests: bool,
    pub max_retries: i32,
    pub rescue_enabled: bool,
    pub model_name: String,
    /* private fields */
}

HTTP server configuration for the OpenAI-compatible proxy.

Fields

host: String

Host to bind.

port: u16

Port to bind.

serialize_requests: bool

Whether to serialize requests (single-worker queue).

max_retries: i32

Maximum retries for tool-call validation.

rescue_enabled: bool

Whether rescue parsing is enabled.

model_name: String

The model name reported in responses.

Implementations

impl HTTPServer

pub fn new( host: &str, port: u16, serialize_requests: bool, max_retries: i32, rescue_enabled: bool, model_name: &str, ) -> Self

Creates a new HTTPServer instance with the specified binding and validation options.

pub fn serve_blocking<C>( self: Arc<Self>, client: Arc<C>, ctx: Arc<Mutex<ContextManager>>, ) -> Result<()>

where C: LLMClient + 'static,

Serve the HTTP API using axum on a single-threaded local executor.

Binds to self.host:self.port. Because LLMClient async methods produce non-Send futures (native AFIT), this server runs on a tokio::task::LocalSet which removes the cross-thread Send requirement, matching Python’s single-threaded asyncio model.

Requests are serialized through self.request_mutex when serialize_requests=true, matching Python’s asyncio.Semaphore(1).

pub fn serve_blocking_with_shutdown<C>( self: Arc<Self>, client: Arc<C>, ctx: Arc<Mutex<ContextManager>>, shutdown_rx: Receiver<()>, ) -> Result<()>

where C: LLMClient + 'static,

Serve with graceful shutdown triggered by shutdown_rx resolving.

Same single-threaded LocalSet model as serve_blocking.

impl HTTPServer

pub fn cors_headers() -> Vec<(&'static str, &'static str)>

Build CORS headers for OPTIONS responses.