Skip to main content

Module model_builder

Module model_builder 

Source
Expand description

Model builder: constructs an Lfm2Model from a WeightsContext.

Loads Hugging Face safetensors checkpoints of two families into the same decoder structure, with optional LoRA adapters:

  • Llama-style (model_type llama/mistral, e.g. SmolLM2, TinyLlama): every block is RoPE grouped-query attention plus a SwiGLU MLP; weights named self_attn.{q,k,v,o}_proj, mlp.{gate,up,down}_proj, input_layernorm, post_attention_layernorm and model.norm.
  • LFM2 (layer_types present): blocks are short convolutions or attention, named operator_norm, ffn_norm, feed_forward.w{1,2,3}, with the final norm stored as model.embedding_norm.

Base weights are frozen (Tensor::frozen) unless Lfm2ModelBuilder::with_trainable_base is set, so LoRA training skips their gradients.

Structs§

BuildError
Error returned when a checkpoint cannot be turned into a model.
Lfm2ModelBuilder
Builds an Lfm2Model from loaded weights.
LoRaConfig
Configuration for LoRA adapters

Enums§

Architecture
Weight-naming family of a checkpoint.

Functions§

load_lfm2_model
Load the model in model_path (config.json plus safetensors).