Struct TokenizedGenerateReqInput 

pub struct TokenizedGenerateReqInput {

    pub rid: Option<StrOrList>,
    pub http_worker_ipc: Option<String>,
    pub input_text: Option<String>,
    pub input_ids: Option<Vec<i32>>,
    pub sampling_params: SamplingParams,
    pub return_logprob: bool,
    pub logprob_start_len: i64,
    pub top_logprobs_num: i64,
    pub token_ids_logprob: Option<Vec<i32>>,
    pub stream: bool,
    pub input_embeds: Option<Vec<Value>>,
    pub session_params: Option<SessionParams>,
    pub custom_logit_processor: Option<String>,
    pub return_hidden_states: bool,
    pub created_time: f64,
    pub bootstrap_host: Option<String>,
    pub bootstrap_port: Option<i64>,
    pub bootstrap_room: Option<u64>,
    pub input_multi_ids: Option<Vec<Vec<i32>>>,
    pub input_extra_infos: Option<Vec<Dict<Value, Value>>>,
    pub input_ids_unpadded: Option<Vec<i32>>,
    pub multimodal_inputs: Option<MultimodalInputs>,
    pub validation_error: Option<String>,
    pub data_parallel_rank: Option<i64>,
}

A tokenized generate request, the payload of an ADD message.

The first two fields come from the Python BaseReq base class.

Fields

rid: Option<StrOrList>

Request id. Generate requests always carry a single string.

http_worker_ipc: Option<String>

In-process HTTP worker address; unused across processes.

input_text: Option<String>

Prompt text, when the frontend has it.

input_ids: Option<Vec<i32>>

Prompt token ids; None on the input-embeds path.

sampling_params: SamplingParams

Sampling parameters.

return_logprob: bool

Return the sampled tokens’ logprobs.

logprob_start_len: i64

First prompt position to return logprobs for; -1 is the last prompt token only.

top_logprobs_num: i64

Top-k alternatives per position.

token_ids_logprob: Option<Vec<i32>>

Token ids to score per position.

stream: bool

Stream outputs as they are generated.

input_embeds: Option<Vec<Value>>

Prompt embeddings (nested float lists).

session_params: Option<SessionParams>

Continual-prompting session.

custom_logit_processor: Option<String>

Serialized custom logit processor.

return_hidden_states: bool

Return hidden states.

created_time: f64

Creation time, in seconds.

bootstrap_host: Option<String>

PD: prefill bootstrap host.

bootstrap_port: Option<i64>

PD: prefill bootstrap port.

bootstrap_room: Option<u64>

PD: transfer room id; frontends mint it as a random 64-bit value.

input_multi_ids: Option<Vec<Vec<i32>>>

Additional token-id streams for multi-stream models.

input_extra_infos: Option<Vec<Dict<Value, Value>>>

Model-specific per-request inputs.

input_ids_unpadded: Option<Vec<i32>>

Prompt ids before multimodal placeholder substitution.

multimodal_inputs: Option<MultimodalInputs>

Multimodal payload.

validation_error: Option<String>

Set by a validating transport: the engine finishes the request with an abort carrying this reason.

data_parallel_rank: Option<i64>

Pin to an attention-DP rank.

Implementations

impl TokenizedGenerateReqInput

pub fn new(rid: impl Into<String>, sampling_params: SamplingParams) -> Self

A request with every optional field at its schema default.

Trait Implementations

impl Clone for TokenizedGenerateReqInput

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TokenizedGenerateReqInput

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl PartialEq for TokenizedGenerateReqInput

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for TokenizedGenerateReqInput

impl Wire for TokenizedGenerateReqInput

fn encode(&self, encoder: &mut Encoder)

Append this value to a message.

fn decode(decoder: &mut Decoder<'_>) -> Result<Self>

Read one value of this type.