Struct PredictOutput 

pub struct PredictOutput<'a> {
    pub buffer: BufferView<'a>,
    pub field_ranges: SmallVec<[(Sym, Range<usize>); 4]>,
    pub prompt_tokens: u32,
    pub completion_tokens: u32,
}

Zero-copy predict output.

Fields

buffer: BufferView<'a>

Response buffer.

field_ranges: SmallVec<[(Sym, Range<usize>); 4]>

Field ranges into buffer (symbol to byte range mapping).

prompt_tokens: u32

Number of tokens in the prompt.

completion_tokens: u32

Number of tokens in the completion.

Implementations

impl<'a> PredictOutput<'a>

pub fn get(&self, sym: Sym) -> Option<StrView<'a>>

Get a field value by symbol.

pub fn get_by_name(&self, name: &str) -> Option<StrView<'a>>

Get a field value by name.