Skip to main content

Module policy

Module policy 

Source
Expand description

Routing policies for federation

Policies determine HOW nodes are selected for inference requests. Multiple policies can be composed (scored, weighted, chained).

Structsยง

CompositePolicy
Combines multiple policies with weighted scoring
CostPolicy
Cost-based routing policy
HealthPolicy
Health-based routing policy
LatencyPolicy
Latency-based routing policy
LocalityPolicy
Locality-based routing policy
PrivacyPolicy
Privacy-based routing policy
RoutingPolicy
Routing policy configuration
SelectionCriteria
Criteria for selecting nodes