Struct cang_jie::tokenizer::CangJieTokenizer
source · pub struct CangJieTokenizer {
pub worker: Arc<Jieba>,
pub option: TokenizerOption,
}
Fields§
§worker: Arc<Jieba>
Separation algorithm provider
option: TokenizerOption
Separation config
Trait Implementations§
source§impl Clone for CangJieTokenizer
impl Clone for CangJieTokenizer
source§fn clone(&self) -> CangJieTokenizer
fn clone(&self) -> CangJieTokenizer
Returns a copy of the value. Read more
1.0.0 · source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
Performs copy-assignment from
source
. Read moresource§impl Debug for CangJieTokenizer
impl Debug for CangJieTokenizer
source§impl Default for CangJieTokenizer
impl Default for CangJieTokenizer
source§impl Tokenizer for CangJieTokenizer
impl Tokenizer for CangJieTokenizer
source§fn token_stream<'a>(&mut self, text: &'a str) -> CangjieTokenStream<'a>
fn token_stream<'a>(&mut self, text: &'a str) -> CangjieTokenStream<'a>
Cut text into tokens
§type TokenStream<'a> = CangjieTokenStream<'a>
type TokenStream<'a> = CangjieTokenStream<'a>
The token stream returned by this Tokenizer.
Auto Trait Implementations§
impl RefUnwindSafe for CangJieTokenizer
impl Send for CangJieTokenizer
impl Sync for CangJieTokenizer
impl Unpin for CangJieTokenizer
impl UnwindSafe for CangJieTokenizer
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Mutably borrows from an owned value. Read more
§impl<T> Downcast for Twhere
T: Any,
impl<T> Downcast for Twhere
T: Any,
§fn into_any(self: Box<T>) -> Box<dyn Any>
fn into_any(self: Box<T>) -> Box<dyn Any>
Convert
Box<dyn Trait>
(where Trait: Downcast
) to Box<dyn Any>
. Box<dyn Any>
can
then be further downcast
into Box<ConcreteType>
where ConcreteType
implements Trait
.§fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
fn into_any_rc(self: Rc<T>) -> Rc<dyn Any>
Convert
Rc<Trait>
(where Trait: Downcast
) to Rc<Any>
. Rc<Any>
can then be
further downcast
into Rc<ConcreteType>
where ConcreteType
implements Trait
.§fn as_any(&self) -> &(dyn Any + 'static)
fn as_any(&self) -> &(dyn Any + 'static)
Convert
&Trait
(where Trait: Downcast
) to &Any
. This is needed since Rust cannot
generate &Any
’s vtable from &Trait
’s.§fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
fn as_any_mut(&mut self) -> &mut (dyn Any + 'static)
Convert
&mut Trait
(where Trait: Downcast
) to &Any
. This is needed since Rust cannot
generate &mut Any
’s vtable from &mut Trait
’s.