ferrotensor-cli

The ferrotensor-cli command line for Ferrotensor: LoRA fine-tuning (train), evaluation (eval) and sampling (generate, alias infer) for small Llama-style models such as SmolLM2 and TinyLlama, on any GPU wgpu supports (Metal, Vulkan, DX12).

Install it from crates.io (the binary is called ferrotensor), or run it from a checkout with cargo run -p ferrotensor-cli --release --:

cargo install ferrotensor-cli
ferrotensor --help

cargo run -p ferrotensor-cli --release -- --help            # commands
cargo run -p ferrotensor-cli --release -- train --help      # every training flag

# LoRA SFT with response-only loss and the ChatML template
cargo run -p ferrotensor-cli --release -- train \
  --model ./data/models/SmolLM2-135M \
  --data ./data/alpaca/train.jsonl --eval-data ./data/alpaca/val.jsonl \
  --out ./data/runs/sft --max-steps 250 --grad-accum 8 --lr 5e-4

# Sample from the fine-tuned adapter (rank/alpha/targets are read from the file)
cargo run -p ferrotensor-cli --release -- generate \
  --model ./data/models/SmolLM2-135M \
  --lora ./data/runs/sft/lora_adapters.safetensors \
  --prompt "Give three tips for staying healthy." --temperature 0.7 --top-p 0.9

# Held-out loss and perplexity
cargo run -p ferrotensor-cli --release -- eval \
  --model ./data/models/SmolLM2-135M --data ./data/alpaca/val.jsonl \
  --lora ./data/runs/sft/lora_adapters.safetensors

A full walkthrough with measured numbers is in docs/examples/lora-sft.md.

• SFT data: .jsonl (one object per line) or a .json array of {"prompt": ..., "response": ...} objects, optional "system".
• Pretraining data (--task pretrain): .txt lines, a .json array of strings or token ids, or packed little-endian u32 tokens in .bin.
• Outputs in --out: lora_adapters.safetensors (+ .state.json for --resume), metrics.jsonl (one line per logged step / eval) and training_state.json (read by ferrotensor-cli monitor).
• ferrotensor-cli demo (or --demo) plays a simulated training UI; it does not train anything.