gras 0.1.7

A Genetic Programming framework for Neural Architecture Search in Rust
Documentation
gras-0.1.7 has been yanked.

๐Ÿงฌ gras: Neural Architecture Search via Genetic Programming

Crates.io License: MIT

Bring your data, your training loop, your fitness function โ€” gras handles the rest.

The engine evolves neural network topologies using genetic algorithms. You provide a Trainer (how to train) and a Fitness (how to rank), and gras searches for the best architecture in parallel.

Why gras?

Hand-designing networks is slow and misses non-obvious architectures. gras discovers better structures automatically:

Before vs After

Hand-designed โ€” same width, same activation, straight pipeline:

graph LR
    I[Input 100] --> H1[64 ยท relu]
    H1 --> H2[64 ยท relu]
    H2 --> O[Output 1]

gras evolved โ€” variable dims, diverse activations, rich wiring:

graph LR
    I[Input 100] --> H1
    H1[64 ยท relu] --> H2[[96 ยท gelu]]
    H1 --> H3(32 ยท silu)
    I --> H4[48 ยท mish]
    H2 --> H4
    H3 --> H4
    H2 --> H5[64 ยท tanh]
    H4 --> H5
    H5 --> O[Output 1]

Each hidden node has its own width, activation, and standardization โ€” all discovered by evolution, not hand-tuned.

How evolution works

graph TD
    A[Initial Population<br/>random topologies] --> B[Evaluate<br/>train + score all nets]
    B --> C{Best enough?}
    C -->|No| D[Select<br/>tournament + elitism]
    C -->|Yes| E[Done โœ“]
    D --> F[Crossover<br/>swap subtrees]
    F --> G[Dedup & Refill<br/>remove duplicates]
    G --> H[Mutate<br/>activation / combine / std]
    H --> B

Features

  • Evolutionary NAS โ€” evolve hidden layer count, dims, activations, combine ops, standardize ops per node
  • Variable hidden dims โ€” each node draws its own output width with configurable pool and stride
  • Parallel evaluation โ€” rayon-powered population scoring
  • Deterministic โ€” one seed reproduces the entire run: population, weights, batches
  • Built-in metrics โ€” MSE, MAE, RMSE, R2, accuracy, F1, precision, cross-entropy
  • Custom fitness โ€” drop-in closures for any metric
  • Checkpointing โ€” engine.json + per-generation snapshots with full topology
  • Deduplication โ€” removes duplicate topologies via full Spec comparison
  • Elitism โ€” configurable number of top individuals preserved each generation
  • Selection โ€” tournament with elitism
  • Crossover โ€” one-point (matching-node pivot, subtree-like on DAG) or uniform (per-node swap)
  • Mutation โ€” one random type per individual: activation, combine op, or standardize op
  • 16 activations โ€” Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos
  • 4 combine ops โ€” Add, Mean, Max, Min
  • 2 standardize ops โ€” Identity, LayerNorm
  • Dropout โ€” configurable per-hidden-node regularization
  • CSV support โ€” auto-detects CSV and converts to binary on first run
  • Robustness tracking โ€” tracks topology appearances, fitness stats, identifies most/least reliable architectures

Installation

[dependencies]
gras = "0.1.6"

# Optional CUDA
gras = { version = "0.1.6", features = ["cuda"] }

Setup

Requirements

  • Rust (edition 2024+) โ€” install via rustup
  • C compiler โ€” gcc/clang (needed by the flodl build system)
  • libtorch โ€” the PyTorch C++ runtime, precompiled for your platform

CPU

export LIBTORCH_PATH=/path/to/libtorch
export LD_LIBRARY_PATH=$LIBTORCH_PATH/lib:$LD_LIBRARY_PATH
cargo build

CUDA

  1. NVIDIA driver โ€” nvidia-smi should work
  2. CUDA toolkit โ€” matching your driver version
  3. libtorch (CUDA variant) โ€” compiled against the same CUDA version
export LIBTORCH_PATH=/path/to/libtorch-cuda
export CUDA_HOME=/usr/local/cuda
export LD_LIBRARY_PATH=$LIBTORCH_PATH/lib:$CUDA_HOME/lib64:$LD_LIBRARY_PATH
cargo build --features cuda

auto_device() returns Device::CUDA(0) when compiled with --features cuda, Device::CPU otherwise.

What You Bring

gras is flexible โ€” it evolves over your training setup:

You provide What gras does
Data โ€” .bin or .csv files (auto-detected) Loads, splits, batches
Trainer โ€” implement the Trainer trait (we provide SupervisedTrainer) Trains each network per generation
Fitness โ€” your ranking metric Ranks individuals for selection

The engine handles topology creation, evolution loop, selection, crossover, mutation, logging, and robustness tracking. You stay in control of training and evaluation.

Quick Start

use gras::{Engine, EngineOptions, Fitness, Direction, SelectionMethod, CrossoverMethod, MutationMethod};
use gras::trainer::{SupervisedTrainer, TrainingConfig};

// 1. Prepare your data: inputs.csv + targets.csv (or .bin)
let data_dir = std::path::Path::new("data/my_problem");

// 2. Define how to rank individuals
let fitness = Fitness::new(
    |pred, target| { /* your metric */ Ok(0.0) },
    Direction::Maximize,
    "my_metric",
);

// 3. Define how to train (or use SupervisedTrainer)
let trainer = SupervisedTrainer::new(
    data_dir,
    10,    // input_dim
    2,     // output_dim
    TrainingConfig::default(),
    |pred, target| { /* your loss: e.g. MSE, cross-entropy */ Ok(pred.clone()) },
).unwrap();

// 4. Configure evolution
let opts = EngineOptions::builder()
    .set_pop_size(50)
    .set_num_generations(10)
    .set_selection(SelectionMethod::Tournament { tournament_size: 2 })
    .set_crossover(CrossoverMethod::OnePoint { action_prob: 0.5 })
    .set_mutation(MutationMethod::Activation { prob: 0.1 })
    .build()
    .unwrap();

// 5. Run
let mut engine = Engine::new(opts, fitness, Box::new(trainer)).unwrap();
engine.run().unwrap();
engine.show_robustness(10, "both");

Options Reference

Mandatory Fields

Field Builder method Constraint
pop_size .set_pop_size(n) โ‰ฅ 2
num_generations .set_num_generations(n) โ‰ฅ 1
selection .set_selection(kind) Required
crossover .set_crossover(kind) Required
mutation .set_mutation(kind) Required

Defaults

Everything else has conservative defaults. Set only what your experiment needs:

Option Default Builder method
seed None (random) .set_seed(Some(42))
num_threads 1 .set_num_threads(0) (0 = auto)
dropout_prob 0.0 .set_dropout_prob(0.1)
elite_count 2 .set_elite_count(5)
dedup_pop_and_fill false .set_dedup_pop_and_fill(true)

Topology

Option Default Builder method
hidden_dim_pool 8..=64 .set_hidden_dim_pool(16..=64)
hidden_dim_stride 8 .set_hidden_dim_stride(16)
min_hidden_num_nodes 2 .set_min_hidden_num_nodes(5)
max_hidden_num_nodes 8 .set_max_hidden_num_nodes(20)
min_hidden_inputs_per_node 1 .set_min_hidden_inputs_per_node(5)
max_hidden_inputs_per_node 8 .set_max_hidden_inputs_per_node(20)
min_hidden_outputs_per_node 1 .set_min_hidden_outputs_per_node(5)
max_hidden_outputs_per_node 8 .set_max_hidden_outputs_per_node(20)

Warm Start

Option Default Builder method
prior_topology_paths [] (empty) .set_prior_topology("engine.json") โ€” add one path
.set_prior_topologies(vec!["a.json", "b.json"]) โ€” add multiple

Fitness

You bring your own fitness function โ€” it ranks individuals for selection:

let fitness = Fitness::new(
    |pred, y| my_metric(pred, y),
    Direction::Maximize,
    "my_metric",
);

The closure receives (&Variable, &Variable) โ†’ Result<f32>. Direction is Maximize or Minimize.

Data Format

Supported Formats

Format Detection Conversion
.bin (native) inputs.bin + targets.bin exist None โ€” used directly
.csv inputs.csv + targets.csv exist Auto-converts to .bin in flodl_data/

Required Layout

data/your_problem/
โ”œโ”€โ”€ inputs.csv     # [n, input_dim]  โ€” feature tensor (CSV or .bin)
โ”œโ”€โ”€ targets.csv    # [n, output_dim] โ€” label tensor (CSV or .bin)
โ””โ”€โ”€ flodl_data/    # auto-created on first run (cached .bin)
    โ”œโ”€โ”€ inputs.bin
    โ””โ”€โ”€ targets.bin
  • input_dim and output_dim are auto-detected from data shapes
  • CSV headers are auto-detected (skipped if non-numeric)
  • Lines starting with # are treated as comments

Outputs

Run Directory Structure

results/<run_id>/
โ”œโ”€โ”€ engine.json                    # Full run config + best topology + robustness
โ”œโ”€โ”€ robustness.csv                 # Topology appearances, fitness stats (top 20 best/worst)
โ”œโ”€โ”€ improvements/
โ”‚   โ”œโ”€โ”€ gen_00.json                # All individuals (seed, fitness, loss, params, topology)
โ”‚   โ”œโ”€โ”€ gen_00.md                  # Markdown for best topology
โ”‚   โ””โ”€โ”€ ...

Robustness Tracking

At run end, the engine logs repeated topologies:

โ”€โ”€ repeated topologies (top 20) โ”€โ”€
  rank   appearances      mean   std_dev      min      max    params  topo_id
  #1             44    0.6103    0.0873   0.4611   0.7855   121610  3a7f2b1c

Contributing

Contributions are welcome! Please open an issue or PR on GitHub.

License

MIT