Module api 

Re-exports

pub use crate::batching::BatchConfig;

pub use crate::error::ErrorResponse;

Modules

grpc

The Open Inference Protocol over gRPC (feature grpc, --grpc-port): the same service as KServe’s and Triton’s inference.GRPCInferenceService, so their gRPC clients work unchanged.

Structs

ClientAddr

The peer address of a connection, plain or TLS, for the rate limit

HealthResponse

MetricsResponse

GET /metrics: server-level counters for monitoring (originally contributed in #32)

ModelInfoResponse

ModelMetrics

Per-model part of GET /metrics

ModelSummary

ModelsResponse

GET /models

PredictRequest

PredictResponse

ReadinessResponse

GET /readyz

ReloadResponse

POST /admin/models/{model}/reload

Server

Serve your own models from Rust, with everything the furnace binary does: dynamic batching, the Open Inference Protocol over HTTP (and gRPC), metrics, readiness, reloads, API keys, rate limits and TLS, all configured through ServerConfig.

ServerConfig

TlsSettings

--tls-cert, --tls-key and --tls-client-ca

Enums

PredictInputData

PredictOutputData

Constants

BATCH_SIZE_HEADER

Request: number of items in a binary body (omit for a single item). Response: number of items in a binary body.

INFERENCE_TIME_HEADER

Binary response: what inference_time_ms holds in the JSON response

OCTET_STREAM

Media type of the raw little-endian f32 input/output format

OUTPUT_SHAPE_HEADER

Binary response: per-item output shape, comma separated (e.g. 1000)

REQUEST_ID_HEADER

Request and response: an ID for the request, taken from the client (up to 128 visible ASCII characters) or generated, attached to every log line of the request