Struct BatchRows 

pub struct BatchRows {
    pub request_ids: Vec<String>,
    pub request_pool_indices: Vec<i32>,
    pub input_lengths: Vec<i32>,
    pub prefill_lengths: Vec<i32>,
    pub extend_prefix_lens: Vec<i32>,
    pub state_prefix_cached: Vec<bool>,
    pub decode_input_ids: Vec<i32>,
    pub spec_candidate_ids: Vec<Vec<i32>>,
    pub num_extends: usize,
    pub tables: BlockTables,
}

A batch’s rows (the scheduler’s TransferBatchView); extend rows first.

Fields

request_ids: Vec<String>

Request ids.

request_pool_indices: Vec<i32>

Each row’s request slot.

input_lengths: Vec<i32>

Tokens each row feeds this forward.

prefill_lengths: Vec<i32>

Each row’s prompt length.

extend_prefix_lens: Vec<i32>

Extend rows: tokens already in the cache ahead of the chunk.

state_prefix_cached: Vec<bool>

Extend rows, a Decode admission: every state group’s prompt-end checkpoint was found locally.

decode_input_ids: Vec<i32>

Decode rows: on a Prefill node’s remote decode, each request’s first output token.

spec_candidate_ids: Vec<Vec<i32>>

Decode rows: the drafter’s candidates, empty for none.

num_extends: usize

Extend rows.

tables: BlockTables

Every cache group’s table.

Trait Implementations

impl Clone for BatchRows

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for BatchRows

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for BatchRows

fn default() -> Self

Returns the “default value” for a type.

impl Eq for BatchRows

impl PartialEq for BatchRows

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for BatchRows