tensorlogic_train/nas/
mod.rs

1//! Neural Architecture Search (NAS) for TensorLogic.
2//!
3//! This module provides infrastructure for automatically discovering high-performing
4//! neural network architectures without manual design.  The search follows the
5//! **ask/tell** convention used throughout this crate: callers request a candidate
6//! architecture via `ask()`, evaluate it externally, then report the score via
7//! `tell()`.  No objective closure is stored.
8//!
9//! ## Algorithms
10//!
11//! | Type | Struct | Notes |
12//! |------|--------|-------|
13//! | Random search | [`RandomArchSearch`] | Uniform baseline, no state |
14//! | Regularized evolution | [`RegularizedEvolution`] | Real et al. 2019 [^1] |
15//!
16//! ## Regularized (Aging) Evolution
17//!
18//! Based on Real et al. (2019) *"Regularized Evolution for Image Classifier
19//! Architecture Search"* (AAAI 2019, <https://arxiv.org/abs/1802.01548>).
20//!
21//! The algorithm maintains a **cyclic population** of fixed size
22//! (`population_size`).  Once the population is full:
23//!
24//! 1. Draw `tournament_size` members uniformly at random.
25//! 2. Select the highest-scoring member (the *winner*).
26//! 3. Apply a single random mutation to the winner to produce a child.
27//! 4. Evaluate the child externally (ask/tell cycle).
28//! 5. Add the child to the population and evict the **oldest** member
29//!    (front of the deque) — regardless of its score.
30//!
31//! Aging pressure prevents premature convergence and keeps exploration alive
32//! throughout the search budget.
33//!
34//! ## Search Space
35//!
36//! [`ArchSearchSpace`] constrains:
37//! - Depth range (`min_depth`..`max_depth`)
38//! - Per-layer width options (discrete set of `usize`)
39//! - Per-layer activation options (e.g. `"relu"`, `"gelu"`, `"tanh"`)
40//! - Per-layer operation options (e.g. `"linear"`, `"conv"`, `"attention"`)
41//!
42//! [`ArchSampler`] draws uniformly from this space and implements the four
43//! mutation operators (change op, change width, change activation, add/remove
44//! layer).
45//!
46//! ## Quick Start
47//!
48//! ```rust,no_run
49//! use tensorlogic_train::nas::{ArchSearchSpace, RegularizedEvolution};
50//!
51//! let space = ArchSearchSpace::new(
52//!     2, 6,
53//!     vec![64, 128, 256],
54//!     vec!["relu".to_string(), "gelu".to_string()],
55//!     vec!["linear".to_string(), "conv".to_string()],
56//! ).unwrap();
57//!
58//! let mut evo = RegularizedEvolution::new(space, 20, 5, 42).unwrap();
59//!
60//! for _ in 0..100 {
61//!     let arch = evo.ask().unwrap();
62//!     // …evaluate arch externally…
63//!     let score = 0.9_f64; // placeholder
64//!     evo.tell(arch, score);
65//! }
66//!
67//! if let Some((best, score)) = evo.best() {
68//!     println!("Best depth={}, score={score:.4}", best.depth());
69//! }
70//! ```
71//!
72//! [^1]: Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019). Regularized
73//!       evolution for image classifier architecture search. *Proceedings of the
74//!       AAAI Conference on Artificial Intelligence*, 33(01), 4780–4789.
75
76pub mod evolution;
77pub mod random_search;
78pub mod sampler;
79pub mod space;
80
81#[cfg(test)]
82mod tests;
83
84pub use evolution::{NasResult, RegularizedEvolution};
85pub use random_search::RandomArchSearch;
86pub use sampler::ArchSampler;
87pub use space::{ArchSearchSpace, Architecture, LayerSpec};
tensorlogic_train/nas/mod.rs

tensorlogic_train/nas/
mod.rs
