Struct tantivy::tokenizer::SimpleTokenizer [−][src]
pub struct SimpleTokenizer;
Expand description
Tokenize the text by splitting on whitespaces and punctuation.
Trait Implementations
Creates a token stream for a given str
.
Auto Trait Implementations
impl RefUnwindSafe for SimpleTokenizer
impl Send for SimpleTokenizer
impl Sync for SimpleTokenizer
impl Unpin for SimpleTokenizer
impl UnwindSafe for SimpleTokenizer
Blanket Implementations
Mutably borrows from an owned value. Read more
Convert Box<dyn Trait>
(where Trait: Downcast
) to Box<dyn Any>
. Box<dyn Any>
can
then be further downcast
into Box<ConcreteType>
where ConcreteType
implements Trait
. Read more
Convert Rc<Trait>
(where Trait: Downcast
) to Rc<Any>
. Rc<Any>
can then be
further downcast
into Rc<ConcreteType>
where ConcreteType
implements Trait
. Read more
Convert &Trait
(where Trait: Downcast
) to &Any
. This is needed since Rust cannot
generate &Any
’s vtable from &Trait
’s. Read more
Convert &mut Trait
(where Trait: Downcast
) to &Any
. This is needed since Rust cannot
generate &mut Any
’s vtable from &mut Trait
’s. Read more