Struct OutputChunk 

pub struct OutputChunk {
    pub output_ids: Vec<i32>,
    pub finish_reason: Option<String>,
    pub prompt_tokens: i64,
    pub completion_tokens: i64,
    pub cached_tokens: i64,
    pub output_token_logprobs: Vec<f64>,
    pub finish_details: Option<FinishDetails>,
    pub spec_verify_ct: Option<i64>,
    pub accept_draft_tokens: Option<f64>,
    pub input_token_logprobs: Option<(Vec<Option<f64>>, Vec<i32>)>,
    pub input_top_logprobs: Option<PromptTopLogprobs>,
    pub output_top_logprobs: Option<TopLogprobs>,
}

One step of a request’s output.

Fields

output_ids: Vec<i32>

Token ids generated this step.

finish_reason: Option<String>

The finish type ("stop", "length", "abort") on the last chunk.

prompt_tokens: i64

Prompt length.

completion_tokens: i64

Tokens generated so far.

cached_tokens: i64

Prompt tokens served from the prefix cache.

output_token_logprobs: Vec<f64>

Logprobs of this step’s tokens; empty when not requested.

finish_details: Option<FinishDetails>

The finish reason in full on the last chunk (the matched token, an abort’s message and codes), from an engine that sends it.

spec_verify_ct: Option<i64>

Verify rounds so far, with speculative decoding.

accept_draft_tokens: Option<f64>

Accepted tokens per verify round once finished, with speculative decoding.

input_token_logprobs: Option<(Vec<Option<f64>>, Vec<i32>)>

The prompt logprobs (None for the first position) and their tokens, in the one chunk that carries them.

input_top_logprobs: Option<PromptTopLogprobs>

The prompt positions’ top-k alternatives, beside the prompt logprobs.

output_top_logprobs: Option<TopLogprobs>

This step’s tokens’ top-k alternatives, when requested.

Trait Implementations

impl Clone for OutputChunk

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for OutputChunk

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl From<&RequestOutput<'_>> for OutputChunk

fn from(row: &RequestOutput<'_>) -> Self

Converts to this type from the input type.

impl PartialEq for OutputChunk

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for OutputChunk