alyze 0.1.2

High-performance text analysis for full-text search

Coverage
37.5%
3 out of 8 items documented1 out of 5 items with examples
Size
Source code size: 109.01 kB This is the summed size of all the files inside the crates.io package for this release.
Documentation size: 2.71 MB This is the summed size of all files generated by rustdoc for all configured targets
Ø build duration
this release: 1m 4s Average build duration of successful builds.
all releases: 46s Average build duration of successful builds in releases after 2024-10-23.
Links
Repository
crates.io
Dependencies
Versions
Owners

alyze

A high-performance tokenization and analysis implementation for full-text search. Provides a UAX #29 compliant tokenizer, implemented with a hand-rolled deterministic finite automaton (DFA). On my laptop (M3 Pro), can tokenize 64MiB of English Wikipedia in ~172ms, or roughly ~372 MiB/s. Non-ASCII text will be slower.

This crate is currently in alpha, but we have ambitions to expand the scope of this crate to encompass a full suite of analysis tools, including stemming, stopword removal, case folding, etc. During alpha development, backwards compatibility is not guaranteed, but we'll do our best to minimize breaking changes.