bpe_openai::normalizer

Trait Normalizable

pub trait Normalizable<'a> {
    // Required method
    fn normalize(self, nfc: bool) -> NormalizedString<'a>;
}

Expand description

Helper trait which converts string types into NormalizedString. Calling normalize on a NormalizedString is a no-op.

Required Methods§

fn normalize(self, nfc: bool) -> NormalizedString<'a>

Implementations on Foreign Types§

impl<'a> Normalizable<'a> for &'a str

fn normalize(self, nfc: bool) -> NormalizedString<'a>

impl<'a, T> Normalizable<'a> for &'a T
where T: AsRef<str>,

fn normalize(self, nfc: bool) -> NormalizedString<'a>

Implementors§

impl<'a> Normalizable<'a> for NormalizedString<'a>