Enum Routing 

pub enum Routing {
    Aggregated(LoadBalance),
    BootstrapRoom,
}

How requests are assigned to engines (one engine per attention-DP rank).

Variants

Aggregated(LoadBalance)

Aggregated serving. A request’s data_parallel_rank pins it to that rank; other requests are placed by the policy.

BootstrapRoom

Prefill/decode disaggregation: rank bootstrap_room % num_engines, the rule both roles apply so a request’s KV lands on the matching rank. Every request must carry a bootstrap_room.

Trait Implementations

impl Clone for Routing

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Routing

impl Debug for Routing

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for Routing

impl PartialEq for Routing

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for Routing