Expand description
Core distributed functionality implementation.
This module provides the essential distributed model loading capabilities with a focus on practical deployment scenarios.
Structs§
- Cluster
Status - Cluster status information
- Inference
Request - Inference request for distributed models
- Inference
Response - Response from distributed inference
- Simple
Distributed Manager - Simple distributed model manager for practical deployment
- Simple
Distributed Model - Simplified distributed model
- Simple
Load Balancer - Simple load balancer implementation
- Simple
Model Shard - Simplified model shard
- Simple
Node Info - Simplified node information
Enums§
- Request
Priority - Priority levels for inference requests