🧬 gras: Neural Architecture Search via Genetic Programming
Bring your data, your training loop, your fitness function — gras handles the rest.
The engine evolves neural network topologies using genetic algorithms.
You provide a Trainer (how to train) and a Fitness (how to rank),
and gras searches for the best architecture in parallel.
Features
- Evolutionary NAS — evolve hidden layer count, dims, activations, combine ops, standardize ops per node
- Variable hidden dims — each node draws its own output width with configurable pool and stride
- Parallel evaluation — rayon-powered population scoring
- Deterministic — one seed reproduces the entire run: population, weights, batches
- Built-in metrics — MSE, MAE, RMSE, R2, accuracy, F1, precision, cross-entropy
- Custom fitness — drop-in closures for any metric
- Checkpointing — engine.json + per-generation snapshots with full topology
- Deduplication — removes duplicate topologies via full Spec comparison
- Elitism — configurable number of top individuals preserved each generation
- Selection — tournament with elitism
- Crossover — one-point (matching-node pivot, subtree-like on DAG) or uniform (per-node swap)
- Mutation — one random type per individual: activation, combine op, or standardize op
- 16 activations — Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos
- 4 combine ops — Add, Mean, Max, Min
- 2 standardize ops — Identity, LayerNorm
- Dropout — configurable per-hidden-node regularization
- CSV support — auto-detects CSV and converts to binary on first run
- Robustness tracking — tracks topology appearances, fitness stats, identifies most/least reliable architectures
Installation
[]
= "0.1"
# Optional CUDA
= { = "0.1", = ["cuda"] }
Setup
Requirements
- Rust (edition 2024+) — install via rustup
- C compiler — gcc/clang (needed by the flodl build system)
- libtorch — the PyTorch C++ runtime, precompiled for your platform
CPU
CUDA
- NVIDIA driver —
nvidia-smishould work - CUDA toolkit — matching your driver version
- libtorch (CUDA variant) — compiled against the same CUDA version
auto_device() returns Device::CUDA(0) when compiled with --features cuda, Device::CPU otherwise.
What You Bring
gras is flexible — it evolves over your training setup:
| You provide | What gras does |
|---|---|
Data — .bin or .csv files (auto-detected) |
Loads, splits, batches |
Trainer — your training loop (or use SupervisedTrainer) |
Trains each network per generation |
| Fitness — your ranking metric (or use built-in scorers) | Ranks individuals for selection |
The engine handles topology creation, evolution loop, selection, crossover, mutation, logging, and robustness tracking. You stay in control of training and evaluation.
Quick Start
use Path;
use Device;
use ;
use ;
Options Reference
Mandatory Fields
| Field | Builder method | Constraint |
|---|---|---|
pop_size |
.set_pop_size(n) |
≥ 2 |
num_generations |
.set_num_generations(n) |
≥ 1 |
selection |
.set_selection(kind) |
Required |
crossover |
.set_crossover(kind) |
Required |
mutation |
.set_mutation(kind) |
Required |
Defaults
Everything else has conservative defaults. Set only what your experiment needs:
| Option | Default | Builder method |
|---|---|---|
seed |
None (random) | .set_seed(Some(42)) |
num_threads |
1 | .set_num_threads(0) (0 = auto) |
dropout_prob |
0.0 | .set_dropout_prob(0.1) |
elite_count |
2 | .set_elite_count(5) |
dedup_pop_and_fill |
false | .set_dedup_pop_and_fill(true) |
Topology
| Option | Default | Builder method |
|---|---|---|
hidden_dim_pool |
8..=64 | .set_hidden_dim_pool(16..=64) |
hidden_dim_stride |
8 | .set_hidden_dim_stride(16) |
min_hidden_num_nodes |
2 | .set_min_hidden_num_nodes(5) |
max_hidden_num_nodes |
8 | .set_max_hidden_num_nodes(20) |
min_hidden_inputs_per_node |
1 | .set_min_hidden_inputs_per_node(5) |
max_hidden_inputs_per_node |
8 | .set_max_hidden_inputs_per_node(20) |
min_hidden_outputs_per_node |
1 | .set_min_hidden_outputs_per_node(5) |
max_hidden_outputs_per_node |
8 | .set_max_hidden_outputs_per_node(20) |
Warm Start
| Option | Default | Builder method |
|---|---|---|
prior_topology |
None | .set_prior_topology("engine.json") |
prior_topologies |
None | .set_prior_topologies(vec!["a.json", "b.json"]) |
Fitness
Your fitness function ranks individuals. Use built-ins or write your own:
// Built-in
let fitness = new;
// Custom
let fitness = new;
Built-in scorers
Continuous (targets [n, 1]):
| Function | Direction |
|---|---|
mse_loss_score |
Minimize |
rmse_score |
Minimize |
l1_loss_score |
Minimize |
r2_score |
Maximize |
Categorical (targets [n, C] one-hot):
| Function | Direction |
|---|---|
accuracy_score |
Maximize |
f1_score |
Maximize |
precision_score |
Maximize |
cross_entropy_onehot |
Minimize |
cross_entropy_onehot_loss |
Minimize (Variable, for backward) |
Custom — any closure fn(&Variable, &Variable) -> Result<f32>:
new
Data Format
Supported Formats
| Format | Detection | Conversion |
|---|---|---|
.bin (native) |
inputs.bin + targets.bin exist |
None — used directly |
.csv |
inputs.csv + targets.csv exist |
Auto-converts to .bin in flodl_data/ |
Required Layout
data/your_problem/
├── inputs.csv # [n, input_dim] — feature tensor (CSV or .bin)
├── targets.csv # [n, output_dim] — label tensor (CSV or .bin)
└── flodl_data/ # auto-created on first run (cached .bin)
├── inputs.bin
└── targets.bin
input_dimandoutput_dimare auto-detected from data shapes- CSV headers are auto-detected (skipped if non-numeric)
- Lines starting with
#are treated as comments
Framing Your Problem
Regression — 1 output, continuous target:
inputs: [n, features] targets: [n, 1]
Binary classification — 1 output, target 0.0 or 1.0:
inputs: [n, features] targets: [n, 1]
Multi-class — C outputs, one-hot targets:
inputs: [n, features] targets: [n, C]
Multi-output regression — multiple continuous targets:
inputs: [n, features] targets: [n, outputs]
Synthetic Datasets
| Generator | Task | Input dim | Target dim |
|---|---|---|---|
synthetic_sine(n, seed, device) |
Regression | 1 | 1 |
synthetic_poly3(n, seed, device) |
Regression | 1 | 1 |
synthetic_sigmoid(n, seed, device) |
Regression | 1 | 1 |
synthetic_multi_sine(n, seed, device) |
Regression | 5 | 1 |
synthetic_xor(n, seed, device) |
Classification | 2 | 2 (one-hot) |
synthetic_blobs(n, seed, device) |
Classification | 2 | 3 (one-hot) |
synthetic_spiral(n, seed, device) |
Classification | 2 | 3 (one-hot) |
synthetic_iris_like(n, seed, device) |
Classification | 4 | 3 (one-hot) |
synthetic_classification(n, in, out, seed, dev) |
Classification | custom | custom |
use synthetic;
use save_dataset;
let ds = synthetic_sine.unwrap;
save_dataset.unwrap;
Outputs
Run Directory Structure
results/<run_id>/
├── engine.json # Full run config + best topology + robustness
├── improvements/
│ ├── gen_00.json # All individuals (seed, fitness, loss, params, topology)
│ ├── gen_00.md # Markdown for best topology
│ └── ...
Robustness Tracking
At run end, the engine logs repeated topologies:
── repeated topologies (top 20) ──
rank appearances mean std_dev min max params topo_id
#1 44 0.6103 0.0873 0.4611 0.7855 121610 3a7f2b1c
Examples
| Example | Description |
|---|---|
generate_md_from_gen |
Generate markdown for any topology from gen_XX.json |
train_from_gen |
Fully train a specific network and see results |
# Generate MD for a specific topology
# Via results directory + topo hash
# Fully train a specific network
Contributing
Contributions are welcome! Please open an issue or PR on GitHub.
License
MIT