gras
Neural Architecture Search via a Continuous Step-Race
gras is a lightweight, high-performance Genetic Programming library for Neural Architecture Search in Rust. It evolves neural network topologies under a step-race: a unified, continuous evolutionary loop where all networks train and evaluate synchronously on a shared deterministic stream, culling and birthing individuals dynamically.
Why gras?
Hand-designing network architectures is slow, error-prone, and biased.
gras automates discovery: each hidden node independently evolves its own width, activation function, merge (combine) operation, and normalization structure—all dynamically wired and evaluated in real-time.
Representation: Hand-designed vs. Evolved
Hand-designed — uniform, straight pipeline:
graph LR
I([Input 100]) --> H1[64 · relu]
H1 --> H2[64 · relu]
H2 --> O([Output 1])
gras evolved — diverse widths, heterogeneous activations, complex multi-path routing:
graph LR
I([Input 100]) ==> H1[64 · relu]
H1 ==> H2[[96 · gelu]]
H1 ==> H3(32 · silu)
I ==> H4[48 · mish]
H2 ==> H4
H3 ==> H4
H2 ==> H5[64 · tanh]
H4 ==> H5
H5 ==> O([Output 1])
How the Step-Race Loop Works
Evolution is continuous, always active, and gated by a historical checkpoint ledger:
graph TD
A([Seed Initial Population]) --> B[Group Step:<br/>Every live net trains on same batch]
B --> C[Every Step:<br/>Trigger Crossover & Mutation Rolls]
C --> D{Crossover child<br/>clears checkpoint gates?}
D -->|Yes| E[Cull worst net &<br/>Insert crossover child]
D -->|No| F[Discard child<br/>pop unchanged]
C --> G[Mutation Roll:<br/>Select victim via inverse fitness]
G --> H[Cull victim &<br/>Insert random immigrant]
E --> I{clock > 0 &&<br/>clock % checkpoint_every == 0?}
F --> I
H --> I
I -->|Yes| J[Write pop-mean fitness<br/>to checkpoints.json ledger]
I -->|No| B
J --> B
Features
- Continuous Step-Race Engine — Dynamic culling and insertion on every step instead of rigid generation boundaries.
- Checkpoint-Gated Crossover — Crossover children must prove themselves during a history-replay catch-up by clearing a gate of historical population means.
- Data-Agnostic Trainer Interface — The evolutionary loop handles selection, culling, and metadata. You own the training recipe (Adam, SGD, schedules, custom losses).
- 100% Deterministic — Identical
run_seedand config guarantees identical weight initializations, batches, and evolutionary histories. - Bulletproof Resume —
RaceEngine::resumereloads active networks, ignores culled tombstones, validates population counts, and replays each net with bit-identical metric parity checked upon revival. - 16 Activations — Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos.
- Combine (Merge) Operators — Add, Mean, Max, Min (+ Multiply, Subtract, Divide).
- Standardize Operators — Identity, LayerNorm.
- Rich Diagnostic Visualization — Topology markdown tables, edge lists, ASCII diagrams, and native Mermaid flowchart generation.
Installation
Add this to your Cargo.toml:
[]
= "1.0"
# Compile with CUDA support
= { = "1.0", = ["cuda"] }
Requirements
- Rust (edition 2024+)
- C/C++ compiler — gcc/clang (needed by the underlying
flodlautograd compilation) - libtorch — PyTorch C++ runtime precompiled for your platform
Before compiling or running, initialize the environment:
# CPU setup
# CUDA setup
Quick Start Showcase (examples/mnist_race.rs)
We ship a complete, first-class documented showcase example under examples/mnist_race.rs illustrating the full capability of the library and config builder.
# Run the MNIST race showcase on CPU
&&
# Run on GPU (if CUDA features are enabled)
&&
# Custom CLI arguments
&&
CLI Options Showcase
The example supports quick CLI configuration parameters:
| Flag | Default | Description |
|---|---|---|
--seed N |
random (recorded) | Run seed for 100% reproducible execution |
--pop N |
20 | Population size (active live networks in the race) |
--steps N |
10 | Maximum step budget for the race |
--checkpoint-every N |
10 | Cadence (in steps) of checkpoint gate logs |
--log-level L |
"summ" | Verbosity level (summ, minimal, full, none) |
Library Quick Start
use ;
use TabularTrainer;
use ;
Detailed Option Surface
1. Budgets and Halts
First budget hit cleanly stops the race:
| Option | Default | Builder method | Description |
|---|---|---|---|
max_steps |
None | .set_max_steps(n) |
Total global step budget limit |
target_score |
None | .set_target_score(v) |
Stop once best network hits this fitness |
fitness_std_threshold |
None | .set_fitness_std_threshold(v) |
Convergence stop: fire when population smoothed-fitness std < v (all nets equally good or equally stuck) |
fitness_std_min_steps |
0 | .set_fitness_std_min_steps(n) |
Warmup for the std stop — earliest step it may fire (protects the early "everyone equally bad" phase) |
One stop criterion at a time. At most ONE of max_steps / target_score / fitness_std_threshold may be set — build() errors otherwise. Two active budgets would silently mask each other in the stop log; pick the one that means what you intend.
2. Genetic Evolution Parameters
| Option | Default | Builder method | Description |
|---|---|---|---|
pop_size |
5 | .set_pop_size(n) |
Number of active networks in population |
checkpoint_every |
10 | .set_checkpoint_every(n) |
Step cadence of ledger checkpoints |
crossover_rolls |
1 | .set_crossover_rolls(n) |
Crossover attempts rolled per step |
crossover_retries |
0 | .set_crossover_retries(n) |
Extra full retries per crossover roll after a gate rejection (fresh parents + generate + gate each retry; every attempt recorded in history.csv) |
mutate_rolls |
1 | .set_mutate_rolls(n) |
Immigrant rolls processed per step |
crossover_prob |
0.5 | .set_crossover_prob(v) |
Crossover execution probability |
mutate_prob |
0.2 | .set_mutate_prob(v) |
Mutation execution probability |
Data-Agnostic "Bring Your Own Trainer"
The RaceEngine acts as an orchestrator. It knows nothing about backpropagation or loss computations. By implementing the Trainer trait (src/trainer/mod.rs), you can configure custom training schemas entirely from user space:
Contract rules enforced at runtime:
- Replay Determinism — Call
seed_step_randomness(ctx.net_seed, step, 0)intrain_stepto synchronize random dropout masks. - In-place updates — The engine owns the networks in memory; you borrow, apply gradients, and return.
We ship TabularTrainer out of the box as a reference recipe, and examples/custom_trainer.rs showcases momentum, linear warmup schedules, and delayed evaluation strategies.
Outputs and Tooling
Run Directory Layout
Each run creates a dedicated run directory containing:
results/<timestamp>/
├── engine.json # Run identity + the COMPLETE config snapshot (every knob)
├── checkpoints.json # The evolution gate ledger history
├── history.csv # Unified event log, typed by the `type` column.
│ # Individual identity = (hash, net_seed): the same
│ # topology hash can reappear across eras (re-born or
│ # regenerated children); net_seed disambiguates.
│ # `metric` rows (one per step × live net): step, hash,
│ # net_seed, origin, entered_at_step, train_loss,
│ # eval_loss, fitness, <informative metrics...>
│ # `attempt` rows (one per evolution event): branch,
│ # attempt, outcome (inserted / rejected_gate),
│ # gate_index, child_fitness, bar, victim,
│ # victim_net_seed, pop_size
└── nets/
├── <hash>.json # Live net: topology, net_seed, step, last_metrics, meta
└── <culled_hash>.json # Tombstone: is_alive=false, culled_at_step, cull_reason,
# final_smoothed_fitness
Precision. Every numeric field in these artifacts is the raw f32 (serde /
Display round-trip, i.e. full precision). Only the console log rounds —
per-step metrics to 2 decimals, as mean ± std. There is no options.csv: run
settings live in engine.json, and the unified history lives in history.csv,
which covers every net that ever lived (culled ones included, via metric
rows) and every evolution attempt (inserted or gate-rejected, via attempt
rows) — not just the surviving frontier.
Solo-Net Recovery Tooling (examples/train_by_hash.rs)
Old engines required complex analysis to train an interesting candidate further. In gras, simply point our training tool to any run directory and network hash. It will automatically load the blueprint, rebuild the network, replay the batch stream deterministically to restore its weights, and continue training solo:
License
gras is licensed under the MIT License.