Enum SegmentLanguage 

pub enum SegmentLanguage {
    Chinese,
    Japanese,
    Korean,
}

The language a Tokenizer::Segment segments with.

Separate from [crate::expr::Language], which selects a Snowball stemming algorithm: the two vocabularies do not overlap. Snowball strips suffixes and has no CJK algorithms, while these languages are segmented by dictionary-driven morphological analysis and have no stemmer. Keeping them apart makes SNOWBALL(KOREAN) and SEGMENT(ENGLISH) unrepresentable rather than merely rejected.

Variants

Chinese

Japanese

Korean

Implementations

impl SegmentLanguage

pub fn walk_revisioned_variant_name( wire_revision: u16, discriminant: u32, ) -> Option<&'static str>

Resolve a wire discriminant at wire_revision to the variant identifier at that revision. Returns None if no such variant exists at that revision.

Generated by the revisioned derive.

pub fn walk_revisioned_variant_table( wire_revision: u16, ) -> &'static [(&'static str, u32)]

Variant name + discriminant pairs at wire_revision. Returns an empty slice if the revision is unknown. Generated by the revisioned derive.

impl SegmentLanguage

pub const REVISION: u16 = 1u16

The revision number of this type, as declared via #[revisioned(revision = N)].

impl SegmentLanguage

pub fn as_str(self) -> &'static str

Trait Implementations

impl Clone for SegmentLanguage

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for SegmentLanguage

impl Debug for SegmentLanguage

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl DeserializeRevisioned for SegmentLanguage

fn deserialize_revisioned<R: Read>(reader: &mut R) -> Result<Self, Error>

Deserializes a new instance of the struct from the specified reader.

impl Display for SegmentLanguage

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Eq for SegmentLanguage

impl From<SegmentLanguage> for SegmentLanguage

fn from(v: SegmentLanguage) -> Self

Converts to this type from the input type.

impl From<SegmentLanguage> for SegmentLanguage

fn from(v: SegmentLanguage) -> Self

Converts to this type from the input type.

impl Hash for SegmentLanguage

fn hash<__H: Hasher>(&self, state: &mut __H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for SegmentLanguage

fn eq(&self, other: &Self) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl Revisioned for SegmentLanguage

fn revision() -> u16

Returns the current revision of this type.

fn type_id() -> TypeId

where Self: 'static,

Returns the type id of this type.

impl SerializeRevisioned for SegmentLanguage

fn serialize_revisioned<W: Write>(&self, writer: &mut W) -> Result<(), Error>

Serializes the struct using the specified writer.

impl SkipCheckRevisioned for SegmentLanguage

fn skip_check_revisioned<R: Read>(reader: &mut R) -> Result<(), Error>

impl SkipRevisioned for SegmentLanguage

fn skip_revisioned<R: Read>(reader: &mut R) -> Result<(), Error>

fn skip_revisioned_slice(reader: &mut SliceReader<'_>) -> Result<(), Error>

Consume through bytes backed by SliceReader without allocating a buffer for bulk segments.

impl StructuralPartialEq for SegmentLanguage

impl WalkRevisioned for SegmentLanguage

type Walker<'r, R: BorrowedReader + 'r> = SegmentLanguageWalker<'r, R>

Walker produced by walk_revisioned.

fn walk_revisioned<'r, R: BorrowedReader>( reader: &'r mut R, ) -> Result<Self::Walker<'r, R>, Error>

Read the type’s revision-level prefix and return a walker positioned at the start of the body.