Expand description
FCD polish trainer — native Rust quality-polish for O(1)-converted
models (docs/RUST_FCD.md). Removes the last Python dependency from
the cortiq convert --o1 pipeline.
Certified recipe (torch reference nystrom_fcd_full2_06b.py,
Qwen3-0.6B 28/28): train ONLY the LN gains + FFN of the converted
layers, loss = (1−0.7)·CE + 0.7·KL(teacher‖student), AdamW 5e-5
(torch defaults), grad clip 1.0, batch 2×512 fresh random windows,
quick deterministic val every 25 steps, restore the best checkpoint.
Structure:
- the whole model is dequantized to f32 once; teacher and student SHARE the frozen set, trainables get separate master copies (the KL anchor never drifts);
- layer-level activation checkpointing: the forward keeps only each layer’s input hidden; the backward re-runs one layer at a time;
- converted layers use the certified matrix form of the Nyström joint kernel in f64 (fcd_ops), M constant in backward;
- GDN (GatedDeltaNet) layers of Qwen3.5-class hybrids run frozen in BOTH teacher and student, with a true BPTT through-backward (fcd_ops::gdn_*) so trainable layers BELOW them still learn; vmf_phase linear layers are refused (no backward yet).
Structs§
- FcdHyper
- Training hyper-parameters (defaults = the certified recipe).
- FcdModel
- The f32 training replica of a .cmf model (≤ 1B targets).
- FcdReport
- What the polish measured — written into
provenance.fcdand reported by the CLI. - Gate
Report - What the generation gate saw and decided.
- GenGate
Cfg - Generation-gate configuration (Patent 16 draft, claim 13: checkpoint selection gated on generation-behavior metrics measured through the SERVED kernel, not on the training objective alone).
- Train
State - Master copies + grads + AdamW moments of the trainable tensors.
Functions§
- gate_
pass - Gate predicate (Patent 16 draft, claim 13): a checkpoint PASSES iff
no prompt’s loop score exceeds
thresholdAND none exceeds its zero-shot baseline by more thanslack— boundary values pass. - loop_
score - Loopiness of a generated id sequence: 1 − unique 4-grams / total. 0 = no repeated 4-gram; near 1 = a tight loop.
- run_
polish - Run the full certified polish: train, early-stop/restore-best, and
write
<out>(source tensors byte-copied, polished LN/FFN as f32). - select_
checkpoint - Checkpoint selection: lowest val ppl AMONG GATE-PASSING checkpoints (ties → earliest). None = nothing passed → the caller must restore the zero-shot state (identity polish): the stage must never make generation worse than conversion alone. (Patent 16 draft, claim 13.)