Enum Optimize 

pub enum Optimize {
    Latency,
    Throughput,
    Memory,
}

What the planner is trying to make small.

Distinct from PlacementPolicy, which says how to split. Two clusters can both be RAM-balanced and want opposite plans: a chat endpoint wants the fewest hops it can get away with, a batch job wants every stage the same length so the pipeline never stalls.

Variants

Latency

Time to first token. Every extra stage is a network hop on the critical path, so this prefers to use as FEW nodes as will hold the model, packing each one full before reaching for the next.

Throughput

Tokens per second. With the pipeline full, throughput is set by the slowest stage, so spread work until stage times match — extra hops cost nothing once they overlap.

Memory

Headroom. Spread as thinly as possible across every node, leaving the most free RAM per machine. For sharing a cluster with other work.

Trait Implementations

impl Clone for Optimize

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Optimize

impl Debug for Optimize

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Optimize

fn default() -> Self

Returns the “default value” for a type.

impl<'de> Deserialize<'de> for Optimize

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Eq for Optimize

impl PartialEq for Optimize

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Serialize for Optimize

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.

impl StructuralPartialEq for Optimize