slabs 0.1.0

Text chunking for RAG: fixed, sentence, recursive, and semantic strategies
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

# slabs

Text chunking for RAG pipelines.

Dual-licensed under MIT or Apache-2.0.

## Strategies

| Strategy | Use Case | Complexity |
|----------|----------|------------|
| Fixed | Homogeneous content, baselines | $O(n)$ |
| Sentence | Prose, articles | $O(n)$ |
| Recursive | General-purpose | $O(n \log n)$ |
| Semantic | Topic coherence (`semantic` feature) | $O(nd)$ |

## Example

```rust
use slabs::{Chunker, RecursiveChunker};

let chunker = RecursiveChunker::prose(500);
let text = "Your long document here...";
let slabs = chunker.chunk(text);

for slab in slabs {
    println!("[{}..{}]: {}", slab.start, slab.end, slab.text);
}
```