Struct TokenUtf8Buffer

pub struct TokenUtf8Buffer(/* private fields */);

Used to buffer incoming tokens until they produce a valid string of UTF-8 text.

Tokens are not valid UTF-8 by themselves. However, the LLM will produce valid UTF-8 from multiple tokens. This helps alleviate that issue.

Implementations

impl TokenUtf8Buffer

pub const fn new() -> Self

Create a new buffer.

pub fn push(&mut self, token: &[u8]) -> Option<String>

Add a token to the buffer. If the buffer contains a valid string of UTF-8 text, it is returned and the buffer is cleared for next use.

pub fn adapt_callback<'a, E: Error + 'static>( callback: impl FnMut(&str) -> Result<(), E> + 'a, ) -> impl FnMut(&[u8]) -> Result<(), E> + 'a

Adapt a &str callback so that it can be used in a &[u8] context.

Trait Implementations

impl Clone for TokenUtf8Buffer

fn clone(&self) -> TokenUtf8Buffer

Returns a duplicate of the value.

const fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Default for TokenUtf8Buffer

fn default() -> TokenUtf8Buffer

Returns the “default value” for a type.

impl PartialEq for TokenUtf8Buffer

fn eq(&self, other: &TokenUtf8Buffer) -> bool

Tests for self and other values to be equal, and is used by ==.

const fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl Eq for TokenUtf8Buffer

impl StructuralPartialEq for TokenUtf8Buffer