Skip to main content

Module skillbake

Module skillbake 

Source
Expand description

Native DTG-MA skill bake (Patent 2) — no Python, no torch.

The certified recipe of converter/make_skill_l1fcd.py, in Rust on the FcdModel f32 replica:

  • Phase A — a trainable L1 mask over FFN neurons (one logit per neuron, applied to the input of down_proj as σ(m)): pure LM loss on the task corpus + a progressive L1 penalty. Every 30 steps the binarized mask (σ>τ) is scored on held-out chunks; the best checkpoint — the denoising bottom — is restored at the end. Pruning noise neurons IMPROVES the model before it starts to hurt.
  • Phase B — FCD: the FFN of the last N layers trains against the same LM loss with the hard mask active (cosine LR), held-out gated, best checkpoint restored.

Attention (softmax and GDN alike) is FROZEN and carries no gradient — exactly like the reference recipe (torch.no_grad() around the attention branch): the backward walks the residual stream through the FFN chain only, which is what makes a pure-Rust backward small.

Structs§

BakeArtifacts
The trained artifacts: everything the defrag writer needs, f32.
BakeHyper
Hyper-parameters — defaults are the certified recipe.
BakeReport
What the bake measured and produced.

Functions§

skill_bake
The whole recipe. log receives progress lines.