Struct SchedulerConfig 

pub struct SchedulerConfig {
    pub max_num_batched_tokens: u32,
    pub max_num_seqs: u32,
    pub policy: String,
    pub enable_chunked_prefill: bool,
    pub long_prefill_token_threshold: u32,
    pub max_num_partial_prefills: u32,
    pub block_size: u32,
}

Fields

max_num_batched_tokens: u32

Maximum number of tokens processed in a single iteration

max_num_seqs: u32

Maximum number of sequences that can run concurrently

policy: String

Scheduling policy: “fcfs” or “priority”

enable_chunked_prefill: bool

Enable chunked prefilling

long_prefill_token_threshold: u32

Maximum tokens to prefill in a single iteration (vLLM’s long_prefill_token_threshold) Defaults to 4% of max_model_len if not specified

max_num_partial_prefills: u32

Maximum number of sequences that can be partially prefilled concurrently (vLLM default: 1) This limits how many NEW waiting requests can start prefilling per iteration

block_size: u32

Block size for KV cache (in tokens)

Implementations

impl SchedulerConfig

pub fn set_default_prefill_threshold(&mut self, max_model_len: u32)

Set default prefill threshold based on max model length (vLLM uses 4%) Only sets threshold if max_num_partial_prefills > 1 (matching vLLM behavior)

Trait Implementations

impl Clone for SchedulerConfig

fn clone(&self) -> SchedulerConfig

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for SchedulerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for SchedulerConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.