ferrotensor-loader 0.2.0

Safetensors model loading and tokenizer support for Ferrotensor
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# ferrotensor-loader

Model file loading for [Ferrotensor](../README.md): safetensors (single file and
sharded through `model.safetensors.index.json`), Llama-style `config.json`
parsing, an affine-quantised weight upload path, and a `Tokenizer` with
chat-template encoding and a `Sampler`.

```rust,ignore
use ferrotensor_loader::Tokenizer;

let tok = Tokenizer::from_file("./models/tinyllama/tokenizer.json")?;
let ids = tok.encode("Hello, world");
println!("{}", tok.decode(&ids));
```

The snippet needs a model directory on disk, so it is not run in CI.