Struct TokenizeRequest

pub struct TokenizeRequest<'input> {
    pub text: &'input str,
    pub model: Option<GenerateModel>,
}

Fields

text: &'input str

The string to be tokenized

model: Option<GenerateModel>

optional - The model to use for tokenization. Custom models can also be supplied with their full ID.

Trait Implementations

impl<'input> Debug for TokenizeRequest<'input>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'input> Serialize for TokenizeRequest<'input>

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.