Expand description
Routing policies for federation
Policies determine HOW nodes are selected for inference requests. Multiple policies can be composed (scored, weighted, chained).
Structsยง
- Composite
Policy - Combines multiple policies with weighted scoring
- Cost
Policy - Cost-based routing policy
- Health
Policy - Health-based routing policy
- Latency
Policy - Latency-based routing policy
- Locality
Policy - Locality-based routing policy
- Privacy
Policy - Privacy-based routing policy
- Routing
Policy - Routing policy configuration
- Selection
Criteria - Criteria for selecting nodes