roketok 0.3.1

This crate provides a way to simply set up a tokenizer and use it. Not recommended for simple tokenizers as this crate adds a bunch of stuff to support many if not all kinds of tokenizers.
Documentation
  • Feature flags
  • This release does not have any feature flags.

roketok

There is currently very little information to present on this page because Docs.rs has only limited support for extracting structured feature metadata from Cargo crates. This issue is tracked in Rust RFC #3416. Check this library's main docs, readme, and Cargo.toml in case its authors have documentation for features available there instead.

This release does not have any feature flags.