gras 1.0.2

A Genetic Programming framework for Neural Architecture Search in Rust
Documentation
# gras

### Neural Architecture Search via a Continuous Step-Race

`gras` is a lightweight, high-performance Genetic Programming library for Neural Architecture Search in Rust. It evolves neural network topologies under a **step-race**: a unified, continuous evolutionary loop where all networks train and evaluate synchronously on a shared deterministic stream, culling and birthing individuals dynamically.

## Why gras?

Hand-designing network architectures is slow, error-prone, and biased. 
`gras` automates discovery: each hidden node independently evolves its own width, activation function, merge (combine) operation, and normalization structure—all dynamically wired and evaluated in real-time.

### Representation: Hand-designed vs. Evolved

**Hand-designed** — uniform, straight pipeline:

```mermaid
graph LR
    I([Input 100]) --> H1[64 · relu]
    H1 --> H2[64 · relu]
    H2 --> O([Output 1])
```

**gras evolved** — diverse widths, heterogeneous activations, complex multi-path routing:

```mermaid
graph LR
    I([Input 100]) ==> H1[64 · relu]
    H1 ==> H2[[96 · gelu]]
    H1 ==> H3(32 · silu)
    I ==> H4[48 · mish]
    H2 ==> H4
    H3 ==> H4
    H2 ==> H5[64 · tanh]
    H4 ==> H5
    H5 ==> O([Output 1])
```

---

### How the Step-Race Loop Works

Evolution is continuous, always active, and gated by a **historical checkpoint ledger**:

```mermaid
graph TD
    A([Seed Initial Population]) --> B[Group Step:<br/>Every live net trains on same batch]
    B --> C[Every Step:<br/>Trigger Crossover & Mutation Rolls]
    C --> D{Crossover child<br/>clears checkpoint gates?}
    D -->|Yes| E[Cull worst net &<br/>Insert crossover child]
    D -->|No| F[Discard child<br/>pop unchanged]
    C --> G[Mutation Roll:<br/>Select victim via inverse fitness]
    G --> H[Cull victim &<br/>Insert random immigrant]
    E --> I{clock > 0 &&<br/>clock % checkpoint_every == 0?}
    F --> I
    H --> I
    I -->|Yes| J[Write pop-mean fitness<br/>to checkpoints.json ledger]
    I -->|No| B
    J --> B
```

---

## Features

- **Continuous Step-Race Engine** — Dynamic culling and insertion on every step instead of rigid generation boundaries.
- **Checkpoint-Gated Crossover** — Crossover children must prove themselves during a history-replay catch-up by clearing a gate of historical population means.
- **Data-Agnostic Trainer Interface** — The evolutionary loop handles selection, culling, and metadata. You own the training recipe (Adam, SGD, schedules, custom losses).
- **100% Deterministic** — Identical `run_seed` and config guarantees identical weight initializations, batches, and evolutionary histories.
- **Bulletproof Resume** — `RaceEngine::resume` reloads active networks, ignores culled tombstones, validates population counts, and replays each net with bit-identical metric parity checked upon revival.
- **16 Activations** — Identity, ReLU, GeLU, SiLU, SELU, Tanh, Sigmoid, Mish, LeakyReLU, ELU, GeluTanh, Softplus, HardSwish, HardSigmoid, Sin, Cos.
- **Combine (Merge) Operators** — Add, Mean, Max, Min (+ Multiply, Subtract, Divide).
- **Standardize Operators** — Identity, LayerNorm.
- **Rich Diagnostic Visualization** — Topology markdown tables, edge lists, ASCII diagrams, and native Mermaid flowchart generation.

---

## Installation

Add this to your `Cargo.toml`:

```toml
[dependencies]
gras = "1.0"

# Compile with CUDA support
gras = { version = "1.0", features = ["cuda"] }
```

### Requirements

- **Rust** (edition 2024+)
- **C/C++ compiler** — gcc/clang (needed by the underlying `flodl` autograd compilation)
- **libtorch** — PyTorch C++ runtime precompiled for your platform

Before compiling or running, initialize the environment:
```bash
# CPU setup
source env_setup.sh
cargo build

# CUDA setup
source env_setup.sh cuda
cargo build $GRAS_FEATURES
```

---

## Quick Start Showcase (`examples/mnist.rs`)

We ship a complete, first-class documented showcase example under `examples/mnist.rs` illustrating the full capability of the library and config builder.

```bash
# Run the MNIST race showcase on CPU
source env_setup.sh && cargo run --release --example mnist -- --pop 15 --steps 200 --checkpoint-every 10

# Run on GPU (if CUDA features are enabled)
source env_setup.sh cuda && cargo run --release -F cuda --example mnist -- --seed 16 --steps 100
```

### CLI Options Showcase

The example supports quick CLI configuration parameters:

| Flag | Default | Description |
|------|---------|-------------|
| `--seed N` | random (recorded) | Run seed for 100% reproducible execution |
| `--pop N` | 20 | Population size (active live networks in the race) |
| `--steps N` | 10 | Maximum step budget for the race |
| `--checkpoint-every N` | 10 | Cadence (in steps) of checkpoint gate logs |
| `--log-level L` | "summ" | Verbosity level (`summ`, `minimal`, `full`, `none`) |

### CLI Options Showcase

The example supports quick CLI configuration parameters:

| Flag | Default | Description |
|------|---------|-------------|
| `--seed N` | random (recorded) | Run seed for 100% reproducible execution |
| `--pop N` | 20 | Population size (active live networks in the race) |
| `--steps N` | 10 | Maximum step budget for the race |
| `--checkpoint-every N` | 10 | Cadence (in steps) of checkpoint gate logs |
| `--log-level L` | "summ" | Verbosity level (`summ`, `minimal`, `full`, `none`) |

---

## Library Quick Start

```rust
use gras::engine::{Direction, Fitness, RaceConfig, RaceEngine, RunMode};
use gras::trainer::TabularTrainer;
use gras::utils::{data, score};

fn main() -> flodl::tensor::Result<()> {
    // 1. Resolve dataset
    let data_dir = std::path::Path::new("data/mnist/train");
    let dataset = data::resolve_dataset(data_dir)?;

    // 2. Define Ranking Fitness (engine culling metric)
    let fitness = Fitness::new(
        score::accuracy_score,
        Direction::Maximize,
        "accuracy",
    );

    // 3. Define Supervised Trainer (Adam, linear gradient clipping)
    let loss_fn = |pred: &gras::Variable, y: &gras::Variable| {
        score::cross_entropy_onehot_loss(pred, y)
    };
    let trainer = TabularTrainer::new(loss_fn)
        .with_learning_rate(1e-3)
        .with_grad_clip(1.0);

    // 4. Configure the Race
    let config = RaceConfig::builder()
        .set_pop_size(10)
        .set_max_steps(100)
        .set_checkpoint_every(10)
        
        // --- Operation Pools (Pristine, Boilerplate-free String slice API!) ---
        .set_combine_ops(&["Mean", "Min", "Max"])
        .set_activations(&["ReLU", "SELU", "GELU"])
        .set_standardize_ops(&["Identity"])
        
        // --- Gating, Mode & Sidecar exports ---
        .set_crossover_gating(gras::engine::config::CrossoverGating::Soft) // Checkpoint gate strictness for crossover children
        .set_mode(RunMode::Tabular)                       // Paradigm mode
        .set_csv_export(true)                             // Writes the history.csv unified event log (metric + attempt rows)
        
        // --- Structural Constraints ---
        .set_min_hidden_num_nodes(5)
        .set_max_hidden_num_nodes(10)
        .build();

    // 5. Build and Run the RaceEngine
    let mut engine = RaceEngine::new(gras::engine::RunSpec {
        data_dir: data_dir.to_path_buf(),
        config,
        fitness,
        trainer: Box::new(trainer),
        seed: Some(42),
        run_dir: None,
    })?;

    let stop_reason = engine.run()?;
    println!("Race stopped: {:?}", stop_reason);
    Ok(())
}
```

---

## Detailed Option Surface

### 1. Budgets and Halts
First budget hit cleanly stops the race:

| Option | Default | Builder method | Description |
|--------|---------|----------------|-------------|
| `max_steps` | None | `.set_max_steps(n)` | Total global step budget limit |
| `max_target_fitness` | None | `.set_max_target_fitness(v)` | Stop once best network hits this fitness |

**Stop criteria race each other.** Any combination of `max_steps` / `max_target_fitness` may be set — at every step all set criteria are evaluated in priority order (steps → target) and the **first to fire** ends the run; the stop log names which one fired. With only one set, that criterion is simply the only racer. (`custom_stop` joins the race too, always evaluated last.)

**Post-race pruner (optional).** With `.set_pop_pruner(true)` + `.set_pop_pruner_method(PopPrunerMethod::Hard, steps)`, a stop reason becomes a **transition** instead of an exit: the engine culls every net except the top-`elite_count` (default 1: the champion; each cull is recorded as a `pruner` attempt row + a `pruned` tombstone) and keeps training the survivors for `steps` more steps — same trainer, optimizer state, LR, and shared stream; evolution and stop criteria are off (they are evolution-phase concerns). Every solo step lands in `history.csv` and the survivors' net JSONs like a race step.

### 2. Genetic Evolution Parameters

| Option | Default | Builder method | Description |
|--------|---------|----------------|-------------|
| `pop_size` | 5 | `.set_pop_size(n)` | Number of active networks in population |
| `checkpoint_every` | 10 | `.set_checkpoint_every(n)` | Step cadence of ledger checkpoints |
| `crossover_rolls` | 1 | `.set_crossover_rolls(n)` | Crossover attempts rolled per step |
| `crossover_retries` | 0 | `.set_crossover_retries(n)` | Extra full retries per crossover roll after a gate rejection (fresh parents + generate + gate each retry; every attempt recorded in `history.csv`) |
| `mutate_rolls` | 1 | `.set_mutate_rolls(n)` | Immigrant rolls processed per step |
| `crossover_prob` | 0.5 | `.set_crossover_prob(v)` | Crossover execution probability |
| `mutate_prob` | 0.2 | `.set_mutate_prob(v)` | Mutation execution probability |

---

## Data-Agnostic "Bring Your Own Trainer"

The `RaceEngine` acts as an orchestrator. It knows nothing about backpropagation or loss computations. By implementing the `Trainer` trait (`src/trainer/mod.rs`), you can configure custom training schemas entirely from user space:

```rust
pub trait Trainer {
    /// Make the optimizer for a newly built or reloaded network.
    fn make_optimizer(&self, net: &Network) -> Box<dyn flodl::nn::optim::Optimizer>;

    /// Train the network for one clock-step in place.
    fn train_step(
        &mut self,
        net: &mut Network,
        optimizer: &mut dyn flodl::nn::optim::Optimizer,
        step: usize,
        ctx: &StepContext<'_>,
    ) -> flodl::tensor::Result<StepReport>;
}
```

Contract rules enforced at runtime:
1. **Replay Determinism** — Call `seed_step_randomness(ctx.net_seed, step, 0)` in `train_step` to synchronize random dropout masks.
2. **In-place updates** — The engine owns the networks in memory; you borrow, apply gradients, and return.

We ship `TabularTrainer` out of the box as a reference recipe, and `examples/custom_trainer.rs` showcases momentum, linear warmup schedules, and delayed evaluation strategies.

---

## Outputs and Tooling

### Run Directory Layout
Each run creates a dedicated run directory containing:
```
results/<timestamp>/
├── engine.json         # Run identity + the COMPLETE config snapshot (every knob)
├── checkpoints.json    # The evolution gate ledger history
├── history.csv         # Unified event log, typed by the `type` column.
│                       #   Individual identity = (hash, net_seed): the same
│                       #   topology hash can reappear across eras (re-born or
│                       #   regenerated children); net_seed disambiguates.
│                       #   `metric` rows (one per step × live net): step, hash,
│                       #   net_seed, origin, entered_at_step, train_loss,
│                       #   eval_loss, fitness, <informative metrics...>
│                       #   `attempt` rows (one per evolution event): branch,
│                       #   attempt, outcome (inserted / rejected_gate),
│                       #   gate_index, child_fitness, bar, victim,
│                       #   victim_net_seed, pop_size
└── nets/
    ├── <hash>.json         # Live net: topology, net_seed, step, last_metrics, meta
    └── <culled_hash>.json  # Tombstone: is_alive=false, culled_at_step, cull_reason,
                            #   final_smoothed_fitness
```

**Precision.** Every numeric field in these artifacts is the raw `f32` (serde /
`Display` round-trip, i.e. full precision). Only the console log rounds —
per-step metrics to 2 decimals, as `mean ± std`. There is no `options.csv`: run
settings live in `engine.json`, and the unified history lives in `history.csv`,
which covers **every net that ever lived** (culled ones included, via `metric`
rows) **and every evolution attempt** (inserted or gate-rejected, via `attempt`
rows) — not just the surviving frontier.

### Stop summary & champion topology

At stop (any log level), the run writes two extra things:

- **`elite-<hash>.md`** — the champion's (elite rank #1) full topology as
  Markdown: nodes table, edge list, ASCII wiring diagram, and a Mermaid
  flowchart. Written unconditionally — it's an artifact, not a log line. In
  the Mermaid diagram, **box borders scale with the node's `hidden_dim`** and
  **wire thickness scales with hop distance** (long-range skips render thick).
- A condensed stop report: frontier snapshot count, the resume command, and
  the final elite list (top-k by smoothed fitness with origin, birth step,
  and parameter count).

### Solo-Net Recovery Tooling (`examples/train_by_hash.rs`)
Old engines required complex analysis to train an interesting candidate further. In `gras`, simply point our training tool to any run directory and network hash. It will automatically load the blueprint, rebuild the network, replay the batch stream deterministically to restore its weights, and continue training solo:

```bash
cargo run --example train_by_hash [RUN_DIR] [NET_HASH]
```

---

## Profiling

Numbers-first (no profiler needed — works everywhere, incl. WSL2 without a PMU):
```bash
cargo bench --bench stream        # stream vs train/eval cost split
```

Flamegraph (native Linux, `perf` required — full setup in **SETUP.md §4**):
```bash
sudo apt install -y linux-tools-common linux-tools-generic   # generic perf (WSL2-safe)
cargo install flamegraph
echo 0 | sudo tee /proc/sys/kernel/perf_event_paranoid        # allow sampling

source env_setup.sh
cargo flamegraph --profile profiling --bench flamegraph -- --steps 300          # bare group step
cargo flamegraph --profile profiling --bench flamegraph -- --steps 300 --evolve # + catch-up replay
```
The example runs a bounded, self-contained race with logging and file I/O off,
so the profile shows compute, not formatting. `--evolve` answers the evolution
path; the default answers the batch-stream question.

---

## License

`gras` is licensed under the MIT License.