ffbpe 0.1.8

Unicode-aware, streaming BPE training and tiktoken-compatible encoding
Documentation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
/target
Cargo.lock
out/
dist/
*.svg
.cache
uv.lock

_lib.*.so
_lib.*.pyd
__pycache__/
merges.txt
vocab.json
merges.*.txt
vocab.*.json