Struct ServerConfig 

pub struct ServerConfig {
    pub host: String,
    pub port: u16,
    pub max_concurrent_requests: usize,
    pub max_body_bytes: usize,
    pub request_timeout: Option<Duration>,
    pub batching: BatchConfig,
    pub grpc_port: Option<u16>,
    pub admin_token: Option<String>,
    pub api_keys: Vec<String>,
    pub rate_limit: Option<ClientRateLimit>,
    pub tls: Option<TlsSettings>,
}

Fields

host: String

port: u16

max_concurrent_requests: usize

Maximum number of inferences running at once; extra requests get 503

max_body_bytes: usize

Maximum request body size in bytes

request_timeout: Option<Duration>

Per-request inference timeout; exceeded requests get 504

batching: BatchConfig

Dynamic batching of concurrent requests

grpc_port: Option<u16>

Port for the Open Inference Protocol over gRPC (feature grpc); None serves HTTP only

admin_token: Option<String>

Enables POST /admin/models/{model}/reload for clients presenting this bearer token; None leaves the admin API off

api_keys: Vec<String>

When not empty, every route but the health probes needs one of these keys, as Authorization: Bearer <key> or X-API-Key: <key>

rate_limit: Option<ClientRateLimit>

Per-client rate limit; clients are their API key, or their IP address without keys

tls: Option<TlsSettings>

Serve HTTPS (and gRPC over TLS) instead of plain text; needs the tls feature

Trait Implementations

impl Clone for ServerConfig

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ServerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ServerConfig

fn default() -> Self

Returns the “default value” for a type.