gras
Neural Architecture Search via a Continuous Step-Race
gras is a lightweight, high-performance Genetic Programming library for Neural Architecture Search in Rust. It evolves neural network topologies under a step-race: a unified, continuous evolutionary loop where all networks train and evaluate synchronously on a shared deterministic stream, culling and birthing individuals dynamically.
Why gras?
Hand-designing network architectures is slow, error-prone, and biased.
gras automates discovery: each hidden node independently evolves its own width, activation function, merge (combine) operation, and normalization structure—all dynamically wired and evaluated in real-time.
Representation: Hand-designed vs. Evolved
Hand-designed — uniform, straight pipeline:
graph LR
I([Input 100]) --> H1[64 · relu]
H1 --> H2[64 · relu]
H2 --> O([Output 1])
gras evolved — diverse widths, heterogeneous activations, complex multi-path routing:
graph LR
I([Input 100]) ==> H1[64 · relu]
H1 ==> H2[[96 · gelu]]
H1 ==> H3(32 · silu)
I ==> H4[48 · mish]
H2 ==> H4
H3 ==> H4
H2 ==> H5[64 · tanh]
H4 ==> H5
H5 ==> O([Output 1])
How the Step-Race Loop Works
Evolution is continuous, always active, and gated by a historical checkpoint ledger:
graph TD
A([Seed Initial Population]) --> B[Group Step:<br/>Every live net trains on same batch]
B --> C[Every Step:<br/>Trigger Crossover & Mutation Rolls]
C --> D{Crossover child<br/>clears checkpoint gates?}
D -->|Yes| E[Cull worst net &<br/>Insert crossover child]
D -->|No| F[Discard child<br/>pop unchanged]
C --> G[Mutation Roll:<br/>Select victim via inverse fitness]
G --> H[Cull victim &<br/>Insert random immigrant]
E --> I{clock > 0 &&<br/>clock % checkpoint_every == 0?}
F --> I
H --> I
I -->|Yes| J[Write pop-mean fitness<br/>to checkpoints.json ledger]
I -->|No| B
J --> B
Features
- Continuous Step-Race Engine — Dynamic culling and insertion on every step instead of rigid generation boundaries.
- Checkpoint-Gated Crossover — Crossover children must prove themselves during a history-replay catch-up by clearing a gate of historical population means.
- Data-Agnostic Trainer Interface — The evolutionary loop handles selection, culling, and metadata. You own the training recipe (Adam, SGD, schedules, custom losses).
- 100% Deterministic — Identical
run_seedand config guarantees identical weight initializations, batches, and evolutionary histories. - Bulletproof Resume —
RaceEngine::resumereloads active networks, ignores culled tombstones, validates population counts, and replays each net with bit-identical metric parity checked upon revival. - 16 Activations — Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos.
- Combine (Merge) Operators — Add, Mean, Max, Min (+ Multiply, Subtract, Divide).
- Standardize Operators — Identity, LayerNorm.
- Rich Diagnostic Visualization — Topology markdown tables, edge lists, ASCII diagrams, and native Mermaid flowchart generation.
Installation
Add this to your Cargo.toml:
[]
= "1.0"
# Compile with CUDA support
= { = "1.0", = ["cuda"] }
Requirements
- Rust (edition 2024+)
- C/C++ compiler — gcc/clang (needed by the underlying
flodlautograd compilation) - libtorch — PyTorch C++ runtime precompiled for your platform
Before compiling or running, initialize the environment:
# CPU setup
# CUDA setup
Quick Start Showcase (examples/mnist.rs)
We ship a complete, first-class documented showcase example under examples/mnist.rs illustrating the full capability of the library and config builder.
# Run the MNIST race showcase on CPU
&&
# Run on GPU (if CUDA features are enabled)
&&
CLI Options Showcase
The example supports quick CLI configuration parameters:
| Flag | Default | Description |
|---|---|---|
--seed N |
random (recorded) | Run seed for 100% reproducible execution |
--pop N |
20 | Population size (active live networks in the race) |
--steps N |
10 | Maximum step budget for the race |
--checkpoint-every N |
10 | Cadence (in steps) of checkpoint gate logs |
--log-level L |
"summ" | Verbosity level (summ, minimal, full, none) |
CLI Options Showcase
The example supports quick CLI configuration parameters:
| Flag | Default | Description |
|---|---|---|
--seed N |
random (recorded) | Run seed for 100% reproducible execution |
--pop N |
20 | Population size (active live networks in the race) |
--steps N |
10 | Maximum step budget for the race |
--checkpoint-every N |
10 | Cadence (in steps) of checkpoint gate logs |
--log-level L |
"summ" | Verbosity level (summ, minimal, full, none) |
Library Quick Start
use ;
use TabularTrainer;
use ;
Detailed Option Surface
1. Budgets and Halts
First budget hit cleanly stops the race:
| Option | Default | Builder method | Description |
|---|---|---|---|
max_steps |
None | .set_max_steps(n) |
Total global step budget limit |
max_target_fitness |
None | .set_max_target_fitness(v) |
Stop once best network hits this fitness |
Stop criteria race each other. Any combination of max_steps / max_target_fitness may be set — at every step all set criteria are evaluated in priority order (steps → target) and the first to fire ends the run; the stop log names which one fired. With only one set, that criterion is simply the only racer. (custom_stop joins the race too, always evaluated last.)
Post-race pruner (optional). With .set_pop_pruner(true) + .set_pop_pruner_method(PopPrunerMethod::Hard, steps), a stop reason becomes a transition instead of an exit: the engine culls every net except the top-elite_count (default 1: the champion; each cull is recorded as a pruner attempt row + a pruned tombstone) and keeps training the survivors for steps more steps — same trainer, optimizer state, LR, and shared stream; evolution and stop criteria are off (they are evolution-phase concerns). Every solo step lands in history.csv and the survivors' net JSONs like a race step.
2. Genetic Evolution Parameters
| Option | Default | Builder method | Description |
|---|---|---|---|
pop_size |
5 | .set_pop_size(n) |
Number of active networks in population |
checkpoint_every |
10 | .set_checkpoint_every(n) |
Step cadence of ledger checkpoints |
crossover_rolls |
1 | .set_crossover_rolls(n) |
Crossover attempts rolled per step |
crossover_retries |
0 | .set_crossover_retries(n) |
Extra full retries per crossover roll after a gate rejection (fresh parents + generate + gate each retry; every attempt recorded in history.csv) |
mutate_rolls |
1 | .set_mutate_rolls(n) |
Immigrant rolls processed per step |
crossover_prob |
0.5 | .set_crossover_prob(v) |
Crossover execution probability |
mutate_prob |
0.2 | .set_mutate_prob(v) |
Mutation execution probability |
Data-Agnostic "Bring Your Own Trainer"
The RaceEngine acts as an orchestrator. It knows nothing about backpropagation or loss computations. By implementing the Trainer trait (src/trainer/mod.rs), you can configure custom training schemas entirely from user space:
Contract rules enforced at runtime:
- Replay Determinism — Call
seed_step_randomness(ctx.net_seed, step, 0)intrain_stepto synchronize random dropout masks. - In-place updates — The engine owns the networks in memory; you borrow, apply gradients, and return.
We ship TabularTrainer out of the box as a reference recipe, and examples/custom_trainer.rs showcases momentum, linear warmup schedules, and delayed evaluation strategies.
Outputs and Tooling
Run Directory Layout
Each run creates a dedicated run directory containing:
results/<timestamp>/
├── engine.json # Run identity + the COMPLETE config snapshot (every knob)
├── checkpoints.json # The evolution gate ledger history
├── history.csv # Unified event log, typed by the `type` column.
│ # Individual identity = (hash, net_seed): the same
│ # topology hash can reappear across eras (re-born or
│ # regenerated children); net_seed disambiguates.
│ # `metric` rows (one per step × live net): step, hash,
│ # net_seed, origin, entered_at_step, train_loss,
│ # eval_loss, fitness, <informative metrics...>
│ # `attempt` rows (one per evolution event): branch,
│ # attempt, outcome (inserted / rejected_gate),
│ # gate_index, child_fitness, bar, victim,
│ # victim_net_seed, pop_size
└── nets/
├── <hash>.json # Live net: topology, net_seed, step, last_metrics, meta
└── <culled_hash>.json # Tombstone: is_alive=false, culled_at_step, cull_reason,
# final_smoothed_fitness
Precision. Every numeric field in these artifacts is the raw f32 (serde /
Display round-trip, i.e. full precision). Only the console log rounds —
per-step metrics to 2 decimals, as mean ± std. There is no options.csv: run
settings live in engine.json, and the unified history lives in history.csv,
which covers every net that ever lived (culled ones included, via metric
rows) and every evolution attempt (inserted or gate-rejected, via attempt
rows) — not just the surviving frontier.
Stop summary & champion topology
At stop (any log level), the run writes two extra things:
elite-<hash>.md— the champion's (elite rank #1) full topology as Markdown: nodes table, edge list, ASCII wiring diagram, and a Mermaid flowchart. Written unconditionally — it's an artifact, not a log line. In the Mermaid diagram, box borders scale with the node'shidden_dimand wire thickness scales with hop distance (long-range skips render thick).- A condensed stop report: frontier snapshot count, the resume command, and the final elite list (top-k by smoothed fitness with origin, birth step, and parameter count).
Solo-Net Recovery Tooling (examples/train_by_hash.rs)
Old engines required complex analysis to train an interesting candidate further. In gras, simply point our training tool to any run directory and network hash. It will automatically load the blueprint, rebuild the network, replay the batch stream deterministically to restore its weights, and continue training solo:
Profiling
Numbers-first (no profiler needed — works everywhere, incl. WSL2 without a PMU):
Flamegraph (native Linux, perf required — full setup in SETUP.md §4):
|
The example runs a bounded, self-contained race with logging and file I/O off,
so the profile shows compute, not formatting. --evolve answers the evolution
path; the default answers the batch-stream question.
License
gras is licensed under the MIT License.