Expand description
Model serving capabilities with REST API and gRPC support
Modules§
Structs§
- ApiConfig
- Batch
Stats - Health
Checker - Inference
Request - Inference
Response - Load
Balancer - Load balancer for multiple model servers
- Model
Info - Model
Server - Rate
Limit - Server
Config - Server
Metrics