Expand description
Multi-model serving: manage base models + LoRA adapters with smart routing.
Supports:
- Multiple base model configurations
- Hot-swappable LoRA adapter registry
- Request routing by model ID
- Model alias resolution
- Adapter composition (stacking multiple LoRAs)
Structs§
- Adapter
Ref - A reference to a single LoRA adapter with a blending weight.
- Adapter
Stack - Adapter composition: apply multiple LoRA adapters in sequence.
- Model
Endpoint - Metadata for a served model variant.
- ModelId
- A model endpoint identifier.
- Model
List Entry - Entry for an OpenAI-compatible
/v1/modelsresponse. - Model
Registry - The multi-model registry.
- Model
Router - Smart request router: selects the best model endpoint for a request.
Enums§
- Endpoint
Status - Status of a model endpoint.
- Routing
Error - Errors that can occur when routing a request to a model endpoint.