Struct RequestOutput 

pub struct RequestOutput<'a> {

    pub rid: &'a str,
    pub output_ids: &'a [i32],
    pub finish_reason: Option<&'a str>,
    pub prompt_tokens: i64,
    pub completion_tokens: i64,
    pub cached_tokens: i64,
    pub output_token_logprobs: &'a [f64],
    pub finish_details: Option<&'a FinishDetails>,
    pub spec_verify_ct: Option<i64>,
    pub accept_draft_tokens: Option<f64>,
    pub input_token_logprobs: Option<(&'a [Option<f64>], &'a [i32])>,
    pub input_top_logprobs: Option<PromptTopLogprobRows<'a>>,
    pub output_top_logprobs: Option<TopLogprobRows<'a>>,
}

One request’s slice of a BatchTokenIDOutSlim.

Fields

rid: &'a str

Request id.

output_ids: &'a [i32]

Token ids generated this step.

finish_reason: Option<&'a str>

The finish type, or None while the request runs.

prompt_tokens: i64

Prompt length.

completion_tokens: i64

Tokens generated so far.

cached_tokens: i64

Prompt tokens served from the prefix cache.

output_token_logprobs: &'a [f64]

Logprobs of this step’s tokens; empty when not requested.

finish_details: Option<&'a FinishDetails>

The finish reason in full, once the request ended (from a producer that sends it).

spec_verify_ct: Option<i64>

Verify rounds so far, with speculative decoding.

accept_draft_tokens: Option<f64>

Accepted tokens per verify round, once finished with speculative decoding.

input_token_logprobs: Option<(&'a [Option<f64>], &'a [i32])>

Prompt logprobs and their tokens, in the frame that ships them.

input_top_logprobs: Option<PromptTopLogprobRows<'a>>

The prompt positions’ top-k alternatives, in the frame that ships them.

output_top_logprobs: Option<TopLogprobRows<'a>>

This step’s tokens’ top-k alternatives, when the request asked for them.

Trait Implementations

impl<'a> Clone for RequestOutput<'a>

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Copy for RequestOutput<'a>

impl<'a> Debug for RequestOutput<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'a> PartialEq for RequestOutput<'a>

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl<'a> StructuralPartialEq for RequestOutput<'a>