Expand description
Pretraining Pipeline - SONA-optimized model pretraining configuration
Generates optimal pretraining configurations based on SONA benchmark results:
- 2211 ops/sec throughput
- <0.5ms latency per layer
- +55% quality improvement
- 134 tests passing
Structsยง
- Dataset
Config - Dataset configuration
- Hardware
Config - Hardware configuration
- Lora
Pretrain Config - LoRA pretraining configuration
- Pretrain
Config - Pretraining configuration based on SONA benchmarks
- Pretrain
Package - Pretraining package result
- Pretrain
Pipeline - Pretraining pipeline orchestrator
- Sona
Optimizations - SONA-specific optimizations
- Training
Config - Training hyperparameters