Struct SentenceEmbeddingsTokenizerConfig

pub struct SentenceEmbeddingsTokenizerConfig {
    pub add_prefix_space: Option<bool>,
    pub strip_accents: Option<bool>,
    pub do_lower_case: Option<bool>,
}

Configuration for transformer’s tokenizer

Fields

add_prefix_space: Option<bool>

strip_accents: Option<bool>

do_lower_case: Option<bool>

Trait Implementations

impl Clone for SentenceEmbeddingsTokenizerConfig

fn clone(&self) -> SentenceEmbeddingsTokenizerConfig

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Config for SentenceEmbeddingsTokenizerConfig

fn from_file<P: AsRef<Path>>(path: P) -> Self

Loads a Config object from a JSON file. The format is expected to be aligned with the Transformers library configuration files for each model. The parsing will fail if non-optional keys expected by the model are missing.

impl Debug for SentenceEmbeddingsTokenizerConfig

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for SentenceEmbeddingsTokenizerConfig

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for SentenceEmbeddingsTokenizerConfig

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.