gras
Neural Architecture Search via Genetic Programming
Bring your data, your training loop, your fitness function — gras handles the rest.
The engine evolves neural network topologies using genetic algorithms.
You provide a Trainer (how to train) and a Fitness (how to rank),
and gras searches for the best architecture in parallel.
Why gras?
Hand-designing networks is slow and misses non-obvious architectures. gras discovers better structures automatically:
Before vs After
Hand-designed — same width, same activation, straight pipeline:
%%{init: {'theme': 'base', 'themeVariables': {
'primaryColor': '#1a3a2a',
'primaryTextColor': '#fff',
'lineColor': '#2d6a4f',
'primaryBorderColor': '#40916c'
}}}%%
graph LR
I([Input 100]) --> H1[64 · relu]
H1 --> H2[64 · relu]
H2 --> O([Output 1])
style I fill:#1b4332,stroke:#40916c,stroke-width:3px,color:#fff
style H1 fill:#2d6a4f,stroke:#52b788,stroke-width:3px,color:#fff
style H2 fill:#2d6a4f,stroke:#52b788,stroke-width:3px,color:#fff
style O fill:#1b4332,stroke:#40916c,stroke-width:3px,color:#fff
gras evolved — variable dims, diverse activations, rich wiring:
%%{init: {'theme': 'base', 'themeVariables': {
'primaryColor': '#1a3a2a',
'primaryTextColor': '#fff',
'lineColor': '#2d6a4f',
'primaryBorderColor': '#40916c'
}}}%%
graph LR
I([Input 100]) ==> H1[64 · relu]
H1 ==> H2[[96 · gelu]]
H1 ==> H3(32 · silu)
I ==> H4[48 · mish]
H2 ==> H4
H3 ==> H4
H2 ==> H5[64 · tanh]
H4 ==> H5
H5 ==> O([Output 1])
style I fill:#1b4332,stroke:#40916c,stroke-width:3px,color:#fff
style H1 fill:#2d6a4f,stroke:#52b788,stroke-width:3px,color:#fff
style H2 fill:#1b4332,stroke:#40916c,stroke-width:3px,color:#fff
style H3 fill:#40916c,stroke:#95d5b2,stroke-width:3px,color:#fff
style H4 fill:#2d6a4f,stroke:#52b788,stroke-width:3px,color:#fff
style H5 fill:#1b4332,stroke:#40916c,stroke-width:3px,color:#fff
style O fill:#1b4332,stroke:#40916c,stroke-width:3px,color:#fff
Each hidden node has its own width, activation, and standardization — all discovered by evolution, not hand-tuned.
How evolution works
%%{init: {'theme': 'base', 'themeVariables': {
'primaryColor': '#1a2744',
'primaryTextColor': '#fff',
'lineColor': '#2563eb',
'primaryBorderColor': '#3b82f6'
}}}%%
graph TD
A([Initial Population]) --> B[Evaluate<br/>train + score]
B --> C{Best enough?}
C -->|No| D[Select<br/>tournament + elitism]
C -->|Yes| E([Done ✓])
D ==> F[Crossover<br/>swap subtrees]
F ==> G[Dedup & Refill<br/>remove duplicates]
G ==> H[Mutate<br/>activation / combine / std]
H ==> B
style A fill:#1e3a5f,stroke:#3b82f6,stroke-width:3px,color:#fff
style B fill:#1e40af,stroke:#60a5fa,stroke-width:3px,color:#fff
style C fill:#1e3a5f,stroke:#3b82f6,stroke-width:3px,color:#fff
style D fill:#1e40af,stroke:#60a5fa,stroke-width:3px,color:#fff
style E fill:#1e3a5f,stroke:#3b82f6,stroke-width:3px,color:#fff
style F fill:#2563eb,stroke:#93c5fd,stroke-width:3px,color:#fff
style G fill:#1e40af,stroke:#60a5fa,stroke-width:3px,color:#fff
style H fill:#2563eb,stroke:#93c5fd,stroke-width:3px,color:#fff
Features
- Evolutionary NAS — evolve hidden layer count, dims, activations, combine ops, standardize ops per node
- Variable hidden dims — each node draws its own output width with configurable pool and stride
- Parallel evaluation — rayon-powered population scoring
- Deterministic — one seed reproduces the entire run: population, weights, batches
- Built-in metrics — MSE, MAE, RMSE, R2, accuracy, F1, precision, cross-entropy
- Custom fitness — drop-in closures for any metric
- Checkpointing — engine.json + per-generation snapshots with full topology
- Deduplication — removes duplicate topologies via full Spec comparison
- Elitism — configurable number of top individuals preserved each generation
- Selection — tournament with elitism
- Crossover — one-point (matching-node pivot, subtree-like on DAG) or uniform (per-node swap)
- Mutation — one random type per individual: activation, combine op, or standardize op
- 16 activations — Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos
- 4 combine ops — Add, Mean, Max, Min
- 2 standardize ops — Identity, LayerNorm
- Dropout — configurable per-hidden-node regularization
- CSV support — auto-detects CSV and converts to binary on first run
- Robustness tracking — tracks topology appearances, fitness stats, identifies most/least reliable architectures
Installation
[]
= "0.1"
# Optional CUDA
= { = "0.1", = ["cuda"] }
Setup
Requirements
- Rust (edition 2024+) — install via rustup
- C compiler — gcc/clang (needed by the flodl build system)
- libtorch — the PyTorch C++ runtime, precompiled for your platform
CPU
CUDA
- NVIDIA driver —
nvidia-smishould work - CUDA toolkit — matching your driver version
- libtorch (CUDA variant) — compiled against the same CUDA version
auto_device() returns Device::CUDA(0) when compiled with --features cuda, Device::CPU otherwise.
What You Bring
gras is flexible — it evolves over your training setup:
| You provide | What gras does |
|---|---|
Data — .bin or .csv files |
Loads, splits, batches |
Trainer — wrap a closure with trainer::from_fn, or implement the Trainer trait |
Trains each network per generation |
| Fitness — your ranking metric | Ranks individuals for selection |
The engine handles topology creation, evolution loop, selection, crossover, mutation, logging, and robustness tracking. You stay in control of training and evaluation.
Quick Start
use ;
use from_fn;
// 1. Prepare your data: inputs.csv + targets.csv (or .bin)
let data_dir = new;
// 2. Define how to rank individuals
let fitness = new;
// 3. Define how to train — one closure owns your whole loop
let trainer = from_fn;
// 4. Configure evolution
let opts = builder
.set_pop_size
.set_num_generations
.set_selection
.set_crossover
.set_mutation
.build
.unwrap;
// 5. Run
let mut engine = new.unwrap;
engine.run.unwrap;
engine.show_robustness;
Custom Training
from_fn covers quick custom loops. For stateful or complex trainers — early
stopping, RL, segmentation, anything with its own config — implement the
[Trainer] trait directly (see examples/custom_trainer.rs). A built-in
SupervisedTrainer with a TrainingConfig also ships at
gras::trainer::supervised (re-exported at gras::trainer for short) for
standard SGD/Adam supervised learning; it is a convenience, not the intended
default.
Training hyperparameters (optimizer, lr, epochs, batch sizes, dropout)
live on the TrainingConfig — the evolution engine owns no training knobs.
Dropout is a TrainingConfig field (default 0.0); the engine bakes it into
every network it builds and records it per individual, so saved graphs stay
self-describing.
Options Reference
Mandatory Fields
| Field | Builder method | Constraint |
|---|---|---|
pop_size |
.set_pop_size(n) |
≥ 2 |
num_generations |
.set_num_generations(n) |
≥ 1 |
selection |
.set_selection(kind) |
Required |
crossover |
.set_crossover(kind) |
Required |
mutation |
.set_mutation(kind) |
Required |
Defaults
Everything else has conservative defaults. Set only what your experiment needs:
| Option | Default | Builder method |
|---|---|---|
seed |
None (random) | .set_seed(Some(42)) |
num_threads |
1 | .set_num_threads(0) (0 = auto) |
elite_count |
2 | .set_elite_count(5) |
dedup_pop_and_fill |
false | .set_dedup_pop_and_fill(true) |
Topology
| Option | Default | Builder method |
|---|---|---|
hidden_dim_pool |
8..=64 | .set_hidden_dim_pool(16..=64) |
hidden_dim_stride |
8 | .set_hidden_dim_stride(16) |
min_hidden_num_nodes |
2 | .set_min_hidden_num_nodes(5) |
max_hidden_num_nodes |
8 | .set_max_hidden_num_nodes(20) |
min_hidden_inputs_per_node |
1 | .set_min_hidden_inputs_per_node(5) |
max_hidden_inputs_per_node |
8 | .set_max_hidden_inputs_per_node(20) |
min_hidden_outputs_per_node |
1 | .set_min_hidden_outputs_per_node(5) |
max_hidden_outputs_per_node |
8 | .set_max_hidden_outputs_per_node(20) |
Warm Start
| Option | Default | Builder method |
|---|---|---|
prior_topology_paths |
[] (empty) |
.set_prior_topology("some_topology.json") — add one path |
.set_prior_topologies(vec!["a.json", "b.json"]) — add multiple |
Resume
| Option | Default | Builder method |
|---|---|---|
resume_from |
None | .set_resume_from("results/<run_id>") — continue a previous run |
Every run writes checkpoint.json (at start and after each generation) with
the frontier population, its topology RNG states, and the run_seed. Point
set_resume_from at a run directory to continue it exactly where it left
off: the population is restored, the generation counter continues, and
num_generations counts additional generations. Useful when a search
stalled, got interrupted, or simply didn't converge within its budget:
let opts = builder
.set_pop_size
.set_num_generations // 30 MORE generations
.set_resume_from // previous run dir
.set_selection
.set_crossover
.set_mutation
.build?;
Because genetics are seeded from the run's run_seed + generation counter
(and the checkpoint restores each topology's RNG position), a resumed run is
bit-identical to one uninterrupted run with the combined generation count —
same scores, population, history, and robustness. Notes:
- Keep
pop_size/selection/crossover/mutationthe same as the source run to stay on the original trajectory (a mismatch onpop_sizewarns; differing genetics deliberately branch the search from the frontier). - Resumed artifacts go to a new run dir — the source run's files are never touched.
- Mutually exclusive with
set_prior_topology(ies).
Fitness
You bring your own fitness function — it ranks individuals for selection:
let fitness = new;
The closure receives (&Variable, &Variable) → Result<f32>. Direction is Maximize or Minimize.
Trainer
You bring your own training loop — it trains and scores each network:
let trainer = from_fn;
The closure receives (Network, u64) → Result<(f32, Option<f32>, usize)>: the
network to train and the generation seed (split your data with it to keep runs
reproducible). Return (score, eval_loss, param_count) — the engine ranks on
score; eval_loss and param_count are used for logging and robustness.
For stateful or complex trainers, implement the Trainer trait instead (see
examples/custom_trainer.rs).
Data Format
Supported Formats
| Format | Detection | Conversion |
|---|---|---|
.bin (native) |
inputs.bin + targets.bin exist |
None — used directly |
.csv |
inputs.csv + targets.csv exist |
Auto-converts to .bin in flodl_data/ on first load, then reads the cache |
Required Layout
data/your_problem/
├── inputs.csv # [n, input_dim] — feature tensor (CSV or .bin)
├── targets.csv # [n, output_dim] — label tensor (CSV or .bin)
└── flodl_data/ # auto-created on first run (cached .bin)
├── inputs.bin
└── targets.bin
input_dimandoutput_dimare provided by your trainer (they must match the data shapes)- CSV headers are auto-detected (skipped if non-numeric)
- Lines starting with
#are treated as comments
The .bin format
inputs.bin / targets.bin each hold one tensor in a tiny custom binary
format — no external dependency (see src/utils/data.rs for the exact
code):
magic "GRA1" (4 bytes) | dtype tag (1 byte) | ndim (u64 LE)
| shape (ndim × u64 LE) | row-major tensor bytes
- dtype tag:
0= f32,1= f64,2= i64 — only these three dtypes are storable; anything else is rejected with an error. Datasets are read back in whatever dtype was saved (trainers usually cast to f32 on load). - Header fields are little-endian; the payload is the tensor's raw row-major
bytes, so the file is exactly
header + numel × element_sizebytes. save_datasetalso writes a small human-readablemeta.jsonbeside the tensors with the input/target shapes (informational only — loading does not depend on it).
How data is loaded
resolve_dataset(dir) picks the data source, in priority order:
{dir}/flodl_data/inputs.bin— the cached.bin(created when a.csvwas converted on a previous run){dir}/inputs.bin— direct native.bin{dir}/inputs.csv— parsed and auto-converted to.binin{dir}/flodl_data/, so the CSV is only parsed once
Reading & writing data in code
The one-stop helpers read either format and write either or both:
use ;
// Write a Dataset (inputs [n, in_dim] + targets [n, out_dim]) to a directory:
save_dataset_as.unwrap; // .bin only
save_dataset_as.unwrap; // .csv only
save_dataset_as.unwrap; // both at once
// Read it back from .bin OR .csv — no need to know which was written:
let ds: Dataset = load_dataset_auto.unwrap;
Datasetis just{ inputs: Tensor, targets: Tensor }.resolve_dataset(dir)is the engine's loader: it adds aflodl_data/cache — CSV is parsed once, converted to.bin, and re-read from the cache on later runs.- Lower-level siblings, all public and documented:
save_dataset/load_dataset(.bin),save_csv_dataset/load_csv_dataset(.csv),save_tensor(path, &t)/load_tensor(path)(one raw tensor file). - The engine itself only calls
resolve_dataset— yourTrainerprovidesinput_dim/output_dim, which must match the loaded shapes.
Outputs
Your experience
Running engine.run() gives you three layers, from most to least durable:
- All data saved automatically — every run writes
results/<run_id>/:robustness.csvwith per-topology stats,engine.jsonwith the full config + whole-run record, and onegen_XXX.jsonsnapshot per generation. This is the source of truth — everything you see in the terminal comes from these files. - Terminal logs (via env var) — logs are printed through
env_logger, controlled by theRUST_LOGenvironment variable. Default isinfo(RUST_LOG=debugfor per-generation detail,RUST_LOG=warnto quiet it). - Tables for convenience — the run log is three phases: an
initialization summary, one table per generation, and a done section.
Post-run analysis is yours:
robustness.csvholds every row, andengine.show_robustness(...)(see Quick Start) prints the same data as a table in code.
Run Directory Structure
results/<run_id>/
├── engine.json # Full run config + whole-run robustness record
├── checkpoint.json # Frontier population + RNG states — resume target (see Resume)
├── robustness.csv # Every repeated topology, one row each — the analysis artifact
├── improvements/
│ ├── gen_00.json # gen_seed + individuals (seed, dropout_prob, fitness_mean,
│ │ # eval_loss_mean, params, per-step train_losses +
│ │ # per-epoch eval_losses, topology, topo_hash)
│ └── ...
The engine writes snapshots only — it does not judge topologies for you.
Per-generation files are raw data; which topology is worth inspecting (and
how it truly performed) is answered by robustness.csv, which aggregates
appearances of each topology across the whole run. Terminal logs cover three
phases: an initialization summary, one table per generation, and a done
section pointing at engine.json and robustness.csv.
Analyzing an individual (.md)
Every topology carries one canonical hash — 16 hex chars (xxh3) — everywhere
it appears: topo_hash on each individual in the gen_XX.json snapshots and
topo_id in robustness.csv. To look at a specific topology, copy its hash
from the CSV and render it:
# or write to a file: ... results/<run_id> <topo_id> analysis.md
The example finds every generation that topology appeared in (reported as
gen_<NNN> · idx <i>) and prints the analysis of the latest occurrence:
- a nodes table (kind, in/out ports, linear dims, activation, combine, standardize, sources)
- an edge list with distance markers
- an ASCII wiring diagram — Manhattan-style right-angle arrows, like a circuit schematic
- a Mermaid flowchart of the full graph
The same output comes from
gras::utils::markdown::topology_markdown(&topo, net) — pass the built
Network to enrich the nodes table with linear dims and source wiring.
The Mermaid block renders natively on GitHub, or locally with mmdc
(mermaid-cli).
Other runnable examples: cargo run --example custom_trainer (implementing
the Trainer trait) and cargo run --example categorical_showcase (quick
categorical / continuous showcase).
robustness.csv
One row per repeated topology — this is the complete record of what the run saw, in plain CSV:
| Column | Meaning |
|---|---|
topo_id |
canonical topology hash (16 hex chars) — same as topo_hash in the snapshots |
appearances |
how many times it appeared across generations |
fit_mean, fit_sd, fit_min, fit_max |
fitness distribution across appearances |
loss_mean, loss_sd, loss_min, loss_max |
loss distribution (present only when trainers report loss) |
params |
parameter count |
Robustness Tracking
In code, engine.show_robustness(10, RobustnessFilter::Both) prints the
most-appeared topologies as a table; the CSV above is the same data in
plain text. Repeated topologies look like:
── repeated topologies (top 20) ──
rank appearances mean std_dev min max params topo_id
#1 44 0.6103 0.0873 0.4611 0.7855 121610 3a7f2b1c9d4e5f06
Reproducibility
The contract: same seed + same options ⇒ same training. Every piece of the chain is seeded and recorded, so a saved run can be replayed bit-for-bit:
| Component | Seeded by | Recorded in gen_XX.json |
|---|---|---|
| Initial weights | topology.options.seed (inside the embedded topology) |
individual seed |
| Dropout | trainer's TrainingConfig → topology.options.dropout_prob (applied at build) |
individual dropout_prob |
| Train/eval split | gen_seed → split_indices (eval set: gen_seed + EVAL_SEED_OFFSET) |
top-level gen_seed |
| Batch sequence | gen_seed via sample_batches_from_indices |
derived from gen_seed |
| Dropout masks | flodl::manual_seed(gen_seed) per training call |
derived from gen_seed |
| Loss curves | train per step, eval as per-epoch pass means (eval mode each epoch) | individual train_losses / eval_losses |
Notes:
Network::build(topo, device)derivesseedanddropout_probfrom the topology itself, so rebuilding a saved topology reproduces the exact net the engine evolved (no silent seed-0/dropout-0.05 divergence).- Dropout is configured on the trainer, not the engine:
from_fn/custom trainers default to 0.0 (they own regularization) and opt in viaClosureTrainer::with_dropout(p)or theTrainingConfig.dropout_probfield. Engine::gen_seed_for(generation)exposes the per-generation trainer seed;train_networkseeds libtorch's RNG from it so dropout masks replay too.- Overfitting is visible per individual:
train_lossesholds one value per training step (8 batches → 8 values);eval_lossesholds the mean of each per-epoch eval pass (all eval batches see the same fixed model, so a per-batch eval vector would be pure noise) — withnum_epochs: 10you get 10 eval points to plot against the per-step train line.fitness_mean/eval_loss_meanare the means of the final eval pass (ranking + parity); the lasteval_lossesentry equalseval_loss_mean. cargo run --example train_from_gen -- results/<run_id> <gen_idx> <idx>replays one individual from its gen JSON and self-checks the achieved fitness/loss/curves against the recordedfitness_mean/eval_loss_meanand per-batch curves — a mismatch means the reconstruction (config, loss, fitness) differs from the original run.
Citing gras
If you use gras in your research or project, please mention it:
Or simply reference it in your paper or README:
Topologies were discovered using gras, a genetic programming framework for neural architecture search.
Contributing
Contributions are welcome! Please open an issue or PR on GitHub.
License
MIT