Skip to main content

Trainable

minbpe::base

Trait Trainable 

Source 
pub trait Trainable: Tokenizer {
    // Required method
    fn train(&mut self, text: &str, vocab_size: Token, verbose: bool);
}
Expand description

A Tokenizer that can be trained.

Required Methods§

Source

fn train(&mut self, text: &str, vocab_size: Token, verbose: bool)

Train a vocabulary of size vocab_size in distinct Tokens from text.

Dyn Compatibility§

This trait is dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementors§