๐งฌ gras: Neural Architecture Search via Genetic Programming
Bring your data, your training loop, your fitness function โ gras handles the rest.
The engine evolves neural network topologies using genetic algorithms.
You provide a Trainer (how to train) and a Fitness (how to rank),
and gras searches for the best architecture in parallel.
Why gras?
Hand-designing networks is slow and misses non-obvious architectures. gras discovers better structures automatically:
Before vs After
Hand-designed โ same width, same activation, straight pipeline:
graph LR
I[Input 100] --> H1[64 ยท relu]
H1 --> H2[64 ยท relu]
H2 --> O[Output 1]
gras evolved โ variable dims, diverse activations, rich wiring:
graph LR
I[Input 100] --> H1
H1[64 ยท relu] --> H2[[96 ยท gelu]]
H1 --> H3(32 ยท silu)
I --> H4[48 ยท mish]
H2 --> H4
H3 --> H4
H2 --> H5[64 ยท tanh]
H4 --> H5
H5 --> O[Output 1]
Each hidden node has its own width, activation, and standardization โ all discovered by evolution, not hand-tuned.
How evolution works
graph TD
A[Initial Population<br/>random topologies] --> B[Evaluate<br/>train + score all nets]
B --> C{Best enough?}
C -->|No| D[Select<br/>tournament + elitism]
C -->|Yes| E[Done โ]
D --> F[Crossover<br/>swap subtrees]
F --> G[Dedup & Refill<br/>remove duplicates]
G --> H[Mutate<br/>activation / combine / std]
H --> B
Features
- Evolutionary NAS โ evolve hidden layer count, dims, activations, combine ops, standardize ops per node
- Variable hidden dims โ each node draws its own output width with configurable pool and stride
- Parallel evaluation โ rayon-powered population scoring
- Deterministic โ one seed reproduces the entire run: population, weights, batches
- Built-in metrics โ MSE, MAE, RMSE, R2, accuracy, F1, precision, cross-entropy
- Custom fitness โ drop-in closures for any metric
- Checkpointing โ engine.json + per-generation snapshots with full topology
- Deduplication โ removes duplicate topologies via full Spec comparison
- Elitism โ configurable number of top individuals preserved each generation
- Selection โ tournament with elitism
- Crossover โ one-point (matching-node pivot, subtree-like on DAG) or uniform (per-node swap)
- Mutation โ one random type per individual: activation, combine op, or standardize op
- 16 activations โ Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos
- 4 combine ops โ Add, Mean, Max, Min
- 2 standardize ops โ Identity, LayerNorm
- Dropout โ configurable per-hidden-node regularization
- CSV support โ auto-detects CSV and converts to binary on first run
- Robustness tracking โ tracks topology appearances, fitness stats, identifies most/least reliable architectures
Installation
[]
= "0.1.6"
# Optional CUDA
= { = "0.1.6", = ["cuda"] }
Setup
Requirements
- Rust (edition 2024+) โ install via rustup
- C compiler โ gcc/clang (needed by the flodl build system)
- libtorch โ the PyTorch C++ runtime, precompiled for your platform
CPU
CUDA
- NVIDIA driver โ
nvidia-smishould work - CUDA toolkit โ matching your driver version
- libtorch (CUDA variant) โ compiled against the same CUDA version
auto_device() returns Device::CUDA(0) when compiled with --features cuda, Device::CPU otherwise.
What You Bring
gras is flexible โ it evolves over your training setup:
| You provide | What gras does |
|---|---|
Data โ .bin or .csv files (auto-detected) |
Loads, splits, batches |
Trainer โ implement the Trainer trait (we provide SupervisedTrainer) |
Trains each network per generation |
| Fitness โ your ranking metric | Ranks individuals for selection |
The engine handles topology creation, evolution loop, selection, crossover, mutation, logging, and robustness tracking. You stay in control of training and evaluation.
Quick Start
use ;
use ;
// 1. Prepare your data: inputs.csv + targets.csv (or .bin)
let data_dir = new;
// 2. Define how to rank individuals
let fitness = new;
// 3. Define how to train (or use SupervisedTrainer)
let trainer = new.unwrap;
// 4. Configure evolution
let opts = builder
.set_pop_size
.set_num_generations
.set_selection
.set_crossover
.set_mutation
.build
.unwrap;
// 5. Run
let mut engine = new.unwrap;
engine.run.unwrap;
engine.show_robustness;
Options Reference
Mandatory Fields
| Field | Builder method | Constraint |
|---|---|---|
pop_size |
.set_pop_size(n) |
โฅ 2 |
num_generations |
.set_num_generations(n) |
โฅ 1 |
selection |
.set_selection(kind) |
Required |
crossover |
.set_crossover(kind) |
Required |
mutation |
.set_mutation(kind) |
Required |
Defaults
Everything else has conservative defaults. Set only what your experiment needs:
| Option | Default | Builder method |
|---|---|---|
seed |
None (random) | .set_seed(Some(42)) |
num_threads |
1 | .set_num_threads(0) (0 = auto) |
dropout_prob |
0.0 | .set_dropout_prob(0.1) |
elite_count |
2 | .set_elite_count(5) |
dedup_pop_and_fill |
false | .set_dedup_pop_and_fill(true) |
Topology
| Option | Default | Builder method |
|---|---|---|
hidden_dim_pool |
8..=64 | .set_hidden_dim_pool(16..=64) |
hidden_dim_stride |
8 | .set_hidden_dim_stride(16) |
min_hidden_num_nodes |
2 | .set_min_hidden_num_nodes(5) |
max_hidden_num_nodes |
8 | .set_max_hidden_num_nodes(20) |
min_hidden_inputs_per_node |
1 | .set_min_hidden_inputs_per_node(5) |
max_hidden_inputs_per_node |
8 | .set_max_hidden_inputs_per_node(20) |
min_hidden_outputs_per_node |
1 | .set_min_hidden_outputs_per_node(5) |
max_hidden_outputs_per_node |
8 | .set_max_hidden_outputs_per_node(20) |
Warm Start
| Option | Default | Builder method |
|---|---|---|
prior_topology_paths |
[] (empty) |
.set_prior_topology("engine.json") โ add one path |
.set_prior_topologies(vec!["a.json", "b.json"]) โ add multiple |
Fitness
You bring your own fitness function โ it ranks individuals for selection:
let fitness = new;
The closure receives (&Variable, &Variable) โ Result<f32>. Direction is Maximize or Minimize.
Data Format
Supported Formats
| Format | Detection | Conversion |
|---|---|---|
.bin (native) |
inputs.bin + targets.bin exist |
None โ used directly |
.csv |
inputs.csv + targets.csv exist |
Auto-converts to .bin in flodl_data/ |
Required Layout
data/your_problem/
โโโ inputs.csv # [n, input_dim] โ feature tensor (CSV or .bin)
โโโ targets.csv # [n, output_dim] โ label tensor (CSV or .bin)
โโโ flodl_data/ # auto-created on first run (cached .bin)
โโโ inputs.bin
โโโ targets.bin
input_dimandoutput_dimare auto-detected from data shapes- CSV headers are auto-detected (skipped if non-numeric)
- Lines starting with
#are treated as comments
Outputs
Run Directory Structure
results/<run_id>/
โโโ engine.json # Full run config + best topology + robustness
โโโ robustness.csv # Topology appearances, fitness stats (top 20 best/worst)
โโโ improvements/
โ โโโ gen_00.json # All individuals (seed, fitness, loss, params, topology)
โ โโโ gen_00.md # Markdown for best topology
โ โโโ ...
Robustness Tracking
At run end, the engine logs repeated topologies:
โโ repeated topologies (top 20) โโ
rank appearances mean std_dev min max params topo_id
#1 44 0.6103 0.0873 0.4611 0.7855 121610 3a7f2b1c
Contributing
Contributions are welcome! Please open an issue or PR on GitHub.
License
MIT