Struct QStarPolicy 

pub struct QStarPolicy { /* private fields */ }

The split policy for one served model.

Implementations

impl QStarPolicy

pub fn fixed_cap(max_fetch: usize) -> Self

A fixed cap of max_fetch experts fetched per layer per step.

This is what an unbenchmarked machine gets, with max_fetch == 1: enough to keep the cache warming without betting the step on a link whose speed nobody measured. max_fetch == 0 never fetches (pure CPU); a very large cap is effectively pure offload.

pub fn from_fraction(fraction: f64) -> Self

A measured fetch fraction. Replaces the cap entirely.

pub fn fraction_q16(&self) -> u64

pub fn fraction(&self) -> Option<f64>

The configured fraction as a float, for reporting.

pub fn split(&self, missing: usize) -> QStarSplit

Split missing cache misses between the link and the CPU.

Trait Implementations

impl Clone for QStarPolicy

fn clone(&self) -> QStarPolicy

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for QStarPolicy

impl Debug for QStarPolicy

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for QStarPolicy

impl PartialEq for QStarPolicy

fn eq(&self, other: &QStarPolicy) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for QStarPolicy