Docs.rs
  • nipah_tokenizer-0.1.0
    • nipah_tokenizer 0.1.0
    • Permalink
    • Docs.rs crate page
    • Links
    • Repository
    • crates.io
    • Source
    • Owners
    • JoaoVictorVP
    • Dependencies
      • bencher ^0.1.5 normal
      • lazy_static ^1.4.0 normal
    • Versions
    • 28.72% of the crate is documented
  • Platform
    • i686-pc-windows-msvc
    • i686-unknown-linux-gnu
    • x86_64-apple-darwin
    • x86_64-pc-windows-msvc
    • x86_64-unknown-linux-gnu
  • Feature flags
  • docs.rs
    • About docs.rs
    • Privacy policy
  • Rust
    • Rust website
    • The Book
    • Standard Library API Reference
    • Rust by Example
    • The Cargo Guide
    • Clippy Documentation

nipah_tokenizer0.1.0

Crate Items

  • Structs
  • Enums
  • Functions
  • Type Aliases

List of all items

Structs

  • options::DEFAULT_END_OF_LINES
  • options::DEFAULT_SCOPES
  • options::DEFAULT_SEPARATORS
  • options::EndOfLine
  • options::Scope
  • options::Separator
  • options::SplitAggregator
  • options::TokenizerOptions
  • token::Token
  • token::TokenData
  • token::TokenPosition

Enums

  • options::IncludeMode
  • token::TokenType

Functions

  • options::default
  • options::default_split_aggregators
  • options::default_try_id
  • tokenizer::split_string
  • tokenizer::tokenize

Type Aliases

  • options::SplitAggregatorFn