Skip to main content

create_server

Function create_server 

Source
pub async fn create_server(
    engine: InferenceEngine<'static>,
    tokenizer: Option<TokenizerBridge>,
    addr: SocketAddr,
) -> Result<(), Box<dyn Error + Send + Sync>>
Expand description

Create the full server setup: router + graceful shutdown future.

Returns a future that runs the server until a shutdown signal is received.