Struct DetokenizeOptions 

pub struct DetokenizeOptions {
    pub partial: bool,
    pub render_special: bool,
}

Options for Detokenizer::render.

Fields

partial: bool

If true, this is not the final chunk — buffer any trailing partial UTF-8 sequence rather than emitting replacement characters. Set to false on the last chunk so the buffer flushes.

render_special: bool

If true, render special tokens (e.g. <|eos|>) as text. Default: false.

Trait Implementations

impl Clone for DetokenizeOptions

fn clone(&self) -> DetokenizeOptions

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DetokenizeOptions

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for DetokenizeOptions

fn default() -> DetokenizeOptions

Returns the “default value” for a type.

impl Copy for DetokenizeOptions