pub struct Tokenization {
pub add_bos: bool,
}Expand description
Tokenization flags for exact text admission.
Fields§
§add_bos: boolAdd the model’s beginning-of-sequence token.
Trait Implementations§
Source§impl Clone for Tokenization
impl Clone for Tokenization
Source§fn clone(&self) -> Tokenization
fn clone(&self) -> Tokenization
Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source. Read moreimpl Copy for Tokenization
Source§impl Debug for Tokenization
impl Debug for Tokenization
Source§impl Default for Tokenization
impl Default for Tokenization
Source§fn default() -> Tokenization
fn default() -> Tokenization
Returns the “default value” for a type. Read more
impl Eq for Tokenization
Source§impl PartialEq for Tokenization
impl PartialEq for Tokenization
impl StructuralPartialEq for Tokenization
Auto Trait Implementations§
impl Freeze for Tokenization
impl RefUnwindSafe for Tokenization
impl Send for Tokenization
impl Sync for Tokenization
impl Unpin for Tokenization
impl UnsafeUnpin for Tokenization
impl UnwindSafe for Tokenization
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more