crossandra 0.0.2

A straightforward tokenization library for seamless text processing.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

# crossandra-rs
`crossandra-rs` is a work-in-progress ⚠️, straightforward tokenization library for seamless text
processing. A simplified Rust implementation of the Python [Crossandra] library.

## Usage
Add this to your `Cargo.toml`:
```toml
[dependencies]
crossandra = "0.0.1"
```

Import and use like this:
```rust
use crossandra::{Tokenizer, common};

fn main() {
    let word_finder = Tokenizer::default()
        .with_patterns(vec![common::WORD.clone()])
        .expect("built-in pattern should be safe");

    let text = "Hello, world!";

    for token in word_finder.tokenize(text).flatten() {
        println!("{:?}", token);
    }
    // Token { name: "word", value: "Hello", position: 0}
    // Token { name: "word", value: "world", position: 7}
}
```
---
### Documentation
The documentation is available at [docs.rs/crossandra][docs].

### Acknowledgements
Huge thanks to [@Maneren][Maneren] for his invaluable guidance in developing
this library 🫶

### License
`crossandra-rs` is licensed under the [MIT License].  
© [trag1c], 2024

[Crossandra]: https://github.com/trag1c/crossandra
[docs]: https://docs.rs/crossandra/
[Maneren]: https://github.com/Maneren
[MIT License]: https://opensource.org/license/mit/
[trag1c]: https://github.com/trag1c