anda_db_tfs 0.5.14

A full-text search library using the BM25 ranking algorithm in Rust.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

# anda_db_tfs

`anda_db_tfs` is the BM25 full-text retrieval engine used by AndaDB. It is an
embedded text-search component designed for long-term textual memory, lexical
ranking, and tokenizer customization inside Rust applications.

## What This Crate Provides

- BM25 ranking for textual retrieval
- tokenizer pipelines for embedded search workloads
- optional jieba-based tokenization for CJK text
- concurrent reads and writes
- incremental persistence through dirty-bucket flushing
- reusable search primitives for the `anda_db` collection layer

## When to Use It

Use `anda_db_tfs` when you need:

- full-text search without operating an external search service
- lexical ranking over one or more document fields
- tokenizer control for multilingual AI-memory workloads
- incremental on-disk persistence for an embedded search index

## Getting Started

Add the crate to your project:

```toml
[dependencies]
anda_db_tfs = { version = "0.5", features = ["full"] }
```

This crate is normally used through `anda_db`, but it can also be used as a
standalone embedded BM25 engine.

## Technical Reference

Deep technical documentation for this crate lives in:

- [docs/anda_db_tfs.md](../../docs/anda_db_tfs.md)
- [docs/anda_db.md](../../docs/anda_db.md)

## Related Crates

- `anda_db` for collection-level hybrid retrieval
- `anda_db_hnsw` for vector search that can be fused with BM25 results

## License

MIT. See [LICENSE](../../LICENSE).