Crate ferrotensor_trainer 

Training loop for Ferrotensor language models.

Trainer runs forward, a response-masked cross-entropy loss (masked_cross_entropy), backward and an optimizer step on an Lfm2Model (Llama-style or LFM2), with gradient accumulation, global gradient-norm clipping, an optional learning-rate schedule, a loss scaler and file-based multi-process gradient averaging. data builds SFT and pretraining datasets, checkpoint saves and restores LoRA adapters and optimizer state.

Re-exports

pub use data::build_sft_example;

pub use data::Batch;

pub use data::DataLoader;

pub use data::Dataset;

pub use data::TextDataset;

pub use data::TrainingExample;

pub use demo::print_banner;

pub use demo::run_demo;

pub use demo::DemoConfig;

pub use scaler::GradScaler;

pub use scaler::GradScalerStats;

pub use distributed::FileGradSyncConfig;

Modules

checkpoint

Saving and restoring LoRA adapters and training state.

data

Data Pipeline for Training

demo

Simulated fine-tuning demo for the CLI (ferrotensor-cli --demo).

distributed

Multi-rank gradient averaging for Tier-2 training.

scaler

Dynamic Loss Scaling for Mixed Precision Training

Structs

MaskedLoss

A differentiable masked cross-entropy loss, see masked_cross_entropy.

StepStats

Result of one Trainer::train_step_masked call.

TrainError

Error from a training step.

Trainer

Runs optimization steps on a model.

Functions

masked_cross_entropy

Mean cross-entropy of logits ([.., seq, vocab]) against target_ids over the positions where loss_mask is 1, as a differentiable scalar.

masked_cross_entropy_from_logits

Average cross-entropy over masked target positions using host logits.