ferrotensor-graph

Transformer building blocks and the model builder of the LLM stack in Ferrotensor: LoRAAdapter and linear layers with optional LoRA, attention with rotary embeddings, SwiGLU MLP, RMSNorm, KV caches, flash attention wrappers (FlashAttention, GroupedQueryAttention, MultiQueryAttention, SlidingWindowAttention) and Lfm2Model, a Llama/LFM2-style model assembled from a config and safetensors weights. Experimental.

You normally reach this crate through the CLI, which loads a model directory and attaches LoRA adapters:

cargo run -p ferrotensor-cli --release -- infer --model ./models/tinyllama --prompt "Hello"

Model loading lives in ferrotensor-loader; training in ferrotensor-trainer.