Struct RuntimeDescriptor 

pub struct RuntimeDescriptor {

    pub runtime_id: String,
    pub runtime_version: String,
    pub max_batch: u32,
    pub max_seqs: u32,
    pub max_seq_len: u64,
    pub vocab_size: u32,
    pub page_size_tokens: u32,
    pub kv_bytes_per_token: u64,
    pub cells_total: u64,
    pub truncate_partial: bool,
    pub copy_shares_cells: bool,
    pub takeover_preferred: bool,
    pub seed_min_tokens: u64,
    pub cache_resident_cells: u64,
    pub cache_exported_cells: u64,
    pub recurrent: bool,
    pub export_encodings: Vec<u8>,
    pub engine_sampling: bool,
    pub engine_draws: bool,
    pub verify_rows: bool,
    pub verify_batches: bool,
    pub prefill_step_tokens: u32,
    pub weights_identity: [u8; 32],
    pub architecture: String,
    pub backend: String,
    pub sampling_defaults: SamplingDefaults,
}

Fields

runtime_id: String

runtime_version: String

max_batch: u32

max_seqs: u32

max_seq_len: u64

vocab_size: u32

page_size_tokens: u32

kv_bytes_per_token: u64

cells_total: u64

truncate_partial: bool

copy_shares_cells: bool

takeover_preferred: bool

seed_min_tokens: u64

cache_resident_cells: u64

Cells the prefix cache may keep in the pool: the state its entries hold that no lane or bare sequence shares. 0 = no bound: an entry at rest costs the other sequences nothing. A runtime whose steps cost by what its pool holds names what it affords (llama.cpp’s unified cache runs every step’s attention over the cells up to the highest one in use, whoever holds them); past it the least recently used entries leave the pool, exported or evicted.

cache_exported_cells: u64

Tokens of state the prefix cache may hold out of the pool as lossless exports. 0 = none: an entry that leaves the pool is evicted.

recurrent: bool

export_encodings: Vec<u8>

engine_sampling: bool

engine_draws: bool

verify_rows: bool

step_rows answers a row for every position of a feed: what verifying a draft in one pass needs.

verify_batches: bool

step_rows runs every lane’s draft in one pass; a runtime that runs them one lane at a time drafts only while one lane decodes.

prefill_step_tokens: u32

The most prompt tokens one prefill step runs (the runtime’s own micro-batch); a tick asked to yield stops between such steps. 0: a tick’s prefill is one step.

weights_identity: [u8; 32]

architecture: String

backend: String

sampling_defaults: SamplingDefaults

Trait Implementations

impl Clone for RuntimeDescriptor

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for RuntimeDescriptor

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.