# 7.1. Reproducibility and Random Seeds
Every component with randomness in RustyML draws it through 1 chokepoint, the crate-level `random` module. A single seed can pin an entire experiment. The rules are simple, but 2 consequences differ from NumPy's process-global `np.random` behavior. These consequences are order-sensitivity and thread-locality. RustyML supports scikit-learn's per-estimator `random_state` style, and a global seed that covers every component without one. This page explains how the 2 interact.
## 7.1.1. The 2 public entry points
The public surface is 2 free functions, re-exported at the crate root:
```rust,ignore
pub fn set_global_seed(seed: u64);
pub fn clear_global_seed();
```
`set_global_seed` installs a seed for the calling thread. `clear_global_seed` removes it and restores entropy-based behavior. There is no getter and no global object to pass around. You do not construct a per-call RNG yourself.
Instead, each component takes an `Option<u64>` seed, usually as a `.with_random_state(seed)` builder method or a `random_state` argument. An internal resolver reconciles that value against the global seed. Call `set_global_seed` once, before you build the models whose randomness you want to fix. Everything downstream then becomes reproducible too.
2 internal resolvers reconcile a component's seed with the global one. `make_rng(random_state)` always returns a concrete RNG. It falls back to OS entropy when no seed applies. Almost every component uses it.
`make_rng_opt(random_state)` returns `Option<StdRng>` instead. It yields `None` when neither a local nor a global seed is in effect. That `None` means no randomization was requested. The `DecisionTree` uses this second form (see [2.4. Decision Trees](../Chapter-02/2.4._Decision_Trees.md)). This keeps split tie-breaking fully deterministic unless you ask for randomness. With `make_rng_opt`, an unseeded tree never randomizes ties, not even from entropy.
## 7.1.2. The 3-way seed resolution rule
Given a component's `random_state: Option<u64>` and the thread-local global seed, the rule resolves as follows:
| Component `random_state` | Global seed set? | Result |
| --- | --- | --- |
| `Some(seed)` | either | Use `seed` directly. The global stream is **ignored and left untouched**. |
| `None` | yes | Derive an independent sub-seed by advancing the global stream 1 step. |
| `None` | no | Seed from OS entropy. **Not reproducible**. |
The following code shows the mechanism:
```rust,ignore
match random_state {
Some(seed) => StdRng::seed_from_u64(seed), // independent, global untouched
None => match global_seed_rng {
Some(global) => StdRng::seed_from_u64(global.next_u64()), // sub-seed from the stream
None => StdRng::from_rng(&mut rng()), // no seed anywhere: OS entropy
},
}
```
The 2 branches are asymmetric. An explicit `Some(seed)` builds its RNG from that number alone. It never calls `next_u64()` on the global stream. A `None` under a global seed *consumes 1 draw* from the stream instead.
The global seed acts like a generator of sub-seeds. It hands them out in the order that unseeded draws request them. An explicit `Some` seed resolves without touching the stream at all, so the component it belongs to is inert against the stream. Section 7.1.3 explains when each component draws.
## 7.1.3. Order-sensitivity and when a seed is really inert
This rule has 2 consequences that matter in practice. One of them is a common trap.
The first consequence is order-sensitivity. Unseeded components draw their sub-seeds from the shared stream in the order they ask for them. Their reproducibility therefore depends on the order of those requests. Draw for model A, then for model B, and each gets a specific sub-seed. Draw in the other order instead, and the sub-seeds swap between them. A global seed therefore reproduces a run only when the draw order also stays the same.
The second consequence is inertness. The resolver guarantees that `make_rng(Some(seed))` never calls `next_u64()` on the global stream. An explicit seed consumes nothing from it. Every component resolves its seed at 1 late, named moment, so inertness holds end to end:
- **Estimators resolve inside `fit`**: `KMeans`, `SVC`, `LinearSVC`, `IsolationForest`, `DecisionTree`, `t-SNE`, `train_test_split`, and the `Sequential` shuffle seed. Each one stores `random_state` as a plain field and reads it exactly once, inside `fit`. Construction with `.with_random_state(s)` touches nothing.
- **Layers resolve inside `build`**: `Dense`, the dropout and noise layers, and the convolutional and recurrent layers. A constructor takes the layer configuration alone and draws nothing. A kernel extent comes from the input, and the input is not there yet. `UnaryLayer::build` allocates every array and spends the seed the layer recorded. `SequentialBuilder::build` calls it once per layer, from the input, so a stack draws in stack order. `GraphBuilder::build` does the same over the nodes of a graph model, and a layer that several nodes call draws once.
5 layers draw more than 1 array from that 1 generator, and the order of those draws is fixed. `SeparableConv1D` and `SeparableConv2D` draw the depthwise kernel first and the pointwise kernel second. `SimpleRNN`, `LSTM`, and `GRU` draw the fused input kernel first, then 1 orthogonal recurrent block per gate, in gate order. A different order changes every value from the second draw onward. The draw order inside such a layer is as much a part of the reproducibility contract as the order across layers. See [3.2.3](../Chapter-03/3.2._Dense_Layers_and_Activations.md#323-weight-initialization).
`.with_random_state(s)` on an unbuilt layer records the seed and draws nothing. On a layer that is already built it draws the arrays again from the new seed. So the order of those 2 calls does not matter.
The order of the draws is therefore what matters, and not the order of the constructor calls. The program below builds a seeded layer between 2 unseeded ones. The unseeded layer after it does not move, because the seeded layer resolved seed 999 on its own:
```rust
use ndarray::Array2;
use rustyml::neural_network::layers::Activation;
use rustyml::neural_network::layers::dense::Dense;
use rustyml::neural_network::traits::UnaryLayer;
use rustyml::neural_network::{Ctx, Shape, Tensor};
use rustyml::{clear_global_seed, set_global_seed};
fn row() -> Tensor {
Array2::from_shape_vec((1, 4), vec![0.5, -1.0, 2.0, 0.25])
.unwrap()
.into_dyn()
}
fn built(seed: Option<u64>) -> Dense {
let mut layer = Dense::new(3, Activation::Linear).unwrap();
if let Some(seed) = seed {
layer = layer.with_random_state(seed);
}
// The draw happens here, not in `Dense::new`.
layer.build(&Shape::with_free_batch(&[1, 4])).unwrap();
layer
}
// `forward` takes `&self` and an inference context, so it draws nothing at all.
fn output(layer: &Dense, x: &Tensor) -> Tensor {
layer.forward(x, &mut Ctx::inference()).unwrap()
}
fn max_abs_diff(a: &Tensor, b: &Tensor) -> f32 {
a.iter()
.zip(b.iter())
.map(|(x, y)| (x - y).abs())
.fold(0.0_f32, f32::max)
}
fn main() {
let x = row();
// Run A: 2 consecutive unseeded builds draw sub-seed #1 then sub-seed #2.
set_global_seed(42);
let a1 = built(None);
let a2 = built(None);
// Run B: identical, except a seeded layer is built in between.
set_global_seed(42);
let b1 = built(None);
let _seeded = built(Some(999));
let b2 = built(None);
clear_global_seed();
let (pa1, pa2) = (output(&a1, &x), output(&a2, &x));
let (pb1, pb2) = (output(&b1, &x), output(&b2, &x));
// The first unseeded layer is unaffected: same seed, same build position.
assert_eq!(max_abs_diff(&pa1, &pb1), 0.0);
// 2 consecutive unseeded builds differ: the stream advanced between them.
assert!(max_abs_diff(&pa1, &pa2) > 1e-4);
// b2 DOES match a2. The spliced layer resolved seed 999 on its own,
// and it took nothing from the global stream.
assert_eq!(max_abs_diff(&pa2, &pb2), 0.0);
println!("a build draws a sub-seed, and an explicit seed draws none");
}
```
The lesson applies to both families. To make a run survive refactoring that inserts or reorders stochastic components, give each one its own explicit `random_state`. Do not rely on the global stream's ordering.
An explicit seed decouples a component completely, whether it draws in `fit` or in `build`. Without one, keep the order of the draws stable between your seed call and each draw. The global seed suits a fixed, linear pipeline. Explicit per-component seeds are the reliable choice.
### A forward pass draws nothing into the layer
A build is not the only draw of a layer. A dropout layer and a noise layer draw a mask on every training forward pass, and that draw follows a second rule. `forward` takes `&self`, so it never advances the random stream that the layer holds. The layer copies the stream out of the context and draws the mask from the copy. The advanced copy then waits in the state channel of the context.
The stream reaches the layer at 1 place only. A model applies the state of the pass after each forward pass, and the layer takes its stream back there. `Sequential::train_batch` does this for every layer it calls. `UnaryLayer::forward_mut` does it for a caller that drives 1 layer by hand. 2 forward passes with no such step between them draw the same mask. An inference pass proposes no state at all, and a dropout layer passes its input through unchanged:
```rust
use ndarray::Array;
use rustyml::neural_network::layers::Dropout;
use rustyml::neural_network::traits::UnaryLayer;
use rustyml::neural_network::{Ctx, Shape};
fn main() {
let x = Array::ones((4, 8)).into_dyn();
// `forward` takes `&self`. It draws from a copy of the stream that the
// context holds, so the stream of the layer itself never moves.
let mut pure = Dropout::new(0.5).unwrap().with_random_state(7);
pure.build(&Shape::known(&[4, 8])).unwrap();
let first = pure.forward(&x, &mut Ctx::training()).unwrap();
let second = pure.forward(&x, &mut Ctx::training()).unwrap();
assert_eq!(first, second, "a pure pass draws the same mask twice");
// `forward_mut` completes the pass: it moves the advanced stream into the
// layer, which is the step a model takes after every training forward pass.
let mut advancing = Dropout::new(0.5).unwrap().with_random_state(7);
let mut ctx = Ctx::training();
let one = advancing.forward_mut(&x, &mut ctx).unwrap();
assert_eq!(ctx.pending_states(), 0, "the state reached the layer");
let two = advancing.forward_mut(&x, &mut Ctx::training()).unwrap();
assert_ne!(one, two, "the applied state advanced the stream");
// Both layers start from seed 7, so the first mask of each is the same mask.
assert_eq!(first, one);
println!("a forward pass repeats the mask, and an applied state advances it");
}
```
The rule holds for every layer whose behavior depends on the mode. It is also what lets several threads run inference against 1 model, because a pass that writes nothing into a layer needs no exclusive access.
## 7.1.4. Thread-local semantics
The global seed lives in a `thread_local!` cell. This keeps `set_global_seed` lock-free and free of contention. It also has a hard consequence. **The seed only affects the thread that called `set_global_seed`.** Set it on the same thread that constructs your models, and everything works. Move construction to another thread, and the seed becomes invisible there.
This matters in 3 settings.
**Spawned threads and async runtimes.** A worker thread you start with `std::thread::spawn`, or a task inside `tokio` or `rayon`, starts with no global seed. It falls back to entropy. Call `set_global_seed` at the top of each worker. A better option is to give each component an explicit `random_state`, since that is thread-independent by construction.
**Internal parallelism.** Some estimators build their RNGs on the calling thread before they parallelize. `KMeans` builds a fresh k-means++ RNG once per restart, always before any parallel work. `n_init` defaults to 10. So a global seed reaches every restart.
`IsolationForest` is the exception. It constructs 1 RNG *inside each per-tree closure*. Once `n_estimators >= 10`, those closures run on Rayon worker threads through `into_par_iter()`.
On the `None` path, each worker calls the resolver. The worker finds no thread-local global, because the seed lives on your thread, not on the worker's thread. It falls back to entropy. **A global seed alone does not make a parallel `IsolationForest` reproducible.**
The explicit path avoids this problem. `.with_random_state(s)` gives tree `i` the seed `s + i`. It does not reference any thread-local state. So it reproduces identically, no matter which worker runs which tree.
For any component that builds randomness on worker threads, use an explicit `random_state` instead of the global seed. See [7.3. Performance Tuning and Parallelism](./7.3._Performance_Tuning_and_Parallelism.md) for when parallelism starts.
**The test harness.** Rust's default test harness spawns a fresh thread per test. Each test therefore starts with the global seed unset, which gives clean isolation. Under `cargo test -- --test-threads=1`, every test instead runs on 1 shared thread. A test that calls `set_global_seed` then leaks that seed into every later test that expected unseeded (entropy) behavior. Clear the seed afterward with a drop guard, so it clears even on panic. The crate's own integration tests use this pattern:
```rust,ignore
#[must_use]
pub struct GlobalSeedGuard;
impl GlobalSeedGuard {
pub fn set(seed: u64) -> Self {
rustyml::set_global_seed(seed);
GlobalSeedGuard
}
}
impl Drop for GlobalSeedGuard {
fn drop(&mut self) {
rustyml::clear_global_seed(); // runs even on panic/unwind
}
}
```
Bind it to a variable, for example `let _seed = GlobalSeedGuard::set(123);`. This keeps it alive for the test body and clears it on the way out. An unbound call, `GlobalSeedGuard::set(123);`, drops immediately. It clears the seed before you use it. This is why the type carries `#[must_use]`.
## 7.1.5. What draws randomness across the crate
Everything below routes through the same resolver. 1 global seed covers all of it on the constructing thread, with the parallelism exception from the previous section. Each row also lists the per-component override. Use that override when you want independence from construction order or from the calling thread.
| Component | How to seed it | Reached by the global seed? | Notes |
| --- | --- | --- | --- |
| NN layer weight init (`Dense`, conv, recurrent, ...) | `.with_random_state(seed)` | Yes | `UnaryLayer::build` draws the arrays, and this records the seed it spends. `Initializer::GlorotUniform` draws almost every kernel, a uniform draw over `[-0.05, 0.05]` draws the `Embedding` table, and `Initializer::Orthogonal` draws a recurrent kernel. See 7.1.3 for the draw order inside a layer, and [3.2.3](../Chapter-03/3.2._Dense_Layers_and_Activations.md#323-weight-initialization) for the full rule. |
| Dropout / spatial dropout / gaussian noise masks | `.with_random_state(seed)` | Yes | The layer holds the stream, and a forward pass draws from a copy of it. The stream advances when the model applies the state of the pass. See [3.8](../Chapter-03/3.8._Regularization_and_Normalization_Layers.md). |
| `Sequential` and `Graph` minibatch shuffle | `Sequential::set_seed(seed)`, `SequentialBuilder::new_with_seed(seed)`, or `GraphBuilder::new_with_seed(seed)` | Yes (seed field defaults to `None`) | Only affects `fit_with_batches`. Does not touch layer weights. See [3.1](../Chapter-03/3.1._The_Sequential_Model.md) and [3.10](../Chapter-03/3.10._Graph_Models_and_Merge_Layers.md). |
| `KMeans` (k-means++ init) | `.with_random_state(seed)` | Yes | RNG rebuilt on the calling thread once per k-means++ restart (`n_init` defaults to 10). See [2.7](../Chapter-02/2.7._KMeans_Clustering.md). |
| `SVC` / `LinearSVC` | `.with_random_state(seed)` | Yes | Working-set selection (`SVC`) and minibatch shuffling (`LinearSVC`). See [2.5](../Chapter-02/2.5._Support_Vector_Machines.md). |
| `estimate_bandwidth` (Mean Shift helper) | `estimate_bandwidth(x, quantile, n_samples, Some(seed))` | Yes | Randomness is in the subsampling. `MeanShift::fit` itself, including bin seeding, is deterministic. See [2.9](../Chapter-02/2.9._Mean_Shift.md). |
| `IsolationForest` | `.with_random_state(seed)` | **Not on the parallel path** | Explicit seed required for reproducibility once `n_estimators >= 10`. Per-tree seed is `seed + i`. See [2.13](../Chapter-02/2.13._Isolation_Forest.md). |
| Decision tree split tie-breaking | `.with_random_state(seed)` | Yes | Uses `make_rng_opt`. Ties are randomized *only* when a seed is in effect, otherwise fully deterministic. |
| `t-SNE` random init | `.with_random_state(seed)` **and** `.with_init(Init::Random)` | Yes | The default `Init::PCA` is deterministic and ignores `random_state`. See [2.12](../Chapter-02/2.12._t-SNE.md). |
| `train_test_split` / `train_test_split_stratified` | `random_state: Option<u64>` argument | Yes | Controls the index shuffle. See [4.1](../Chapter-04/4.1._Train_Test_Split.md). |
2 rows need a closer look. The `t-SNE` seed does nothing on the default code path. `Init::PCA` initializes the embedding from the top principal components, and this is deterministic. So `random_state` only matters after you switch to `.with_init(Init::Random)`.
Mean Shift is often mistaken for a seeded estimator. The `MeanShift` struct has no `random_state` field at all. The only draw in that module is the optional subsampling inside the free `estimate_bandwidth` helper. You call that helper yourself to pick a bandwidth.
### Intentional exclusions
Not every pseudo-random draw in the crate routes through this module. Only draws with a lasting effect on the result do. The `pca` and `kernel_pca` dimensionality reducers are left out on purpose.
Their iterative eigensolvers (power iteration, Lanczos) seed a random *starting vector* with a fixed constant. These methods converge to the same eigenvectors regardless of the starting vector. So the seed is observationally inert. It only pins an otherwise-arbitrary eigenvector sign. Using global state here would make that sign choice *less* reproducible, for no benefit.
Randomized SVD (`pca`'s `SVDSolver::Randomized(u64)`) takes its seed inside the public solver variant. The caller always pins it explicitly. There is no `None` path for the global seed to fill. The general rule: route a draw through this module only when it makes a pseudo-random choice that changes the result.
## 7.1.6. What a seed does not freeze
A seed makes the *pseudo-random choices* reproducible. It does not make every part of a run bit-identical.
**Floating-point reduction order is a separate axis.** Summing a vector or a matrix product in parallel can produce results that differ in the last bits from a serial sum. Floating-point addition is not associative, and this has nothing to do with seeding. For bit-stable numeric results, use the parallelism controls, not the seed. See [6.3. Parallel Reductions](../Chapter-06/6.3._Parallel_Reductions.md) and [7.3. Performance Tuning and Parallelism](./7.3._Performance_Tuning_and_Parallelism.md).
**Cross-machine bit-identity is not guaranteed.** With the same seed, 2 machines make identical random choices. Differences in floating-point rounding, SIMD width, thread count, and the BLAS-free matmul backend's reduction order can still make the final weights differ in low-order bits. The same seed gives the same *sequence of decisions*. It does not give byte-for-byte identical floats across architectures.
**A seed does not survive a thread hop, and it does not persist across a save or load by itself.** Reloading a trained model from disk gives you its frozen weights, not the RNG stream that produced them. If you continue training a reloaded model, set the seed again on the current thread. See [7.2. Model Persistence in Depth](./7.2._Model_Persistence_in_Depth.md).
## 7.1.7. Recipes
### Reproducible experiment template
For a single, linear construction sequence on 1 thread, call `set_global_seed` once at the top. This is the simplest way to make an entire pipeline reproducible. Leave every component unseeded (`random_state == None`), and let each one draw its sub-seed from the stream in order. This includes the `Sequential` shuffle, whose seed field defaults to `None` and is therefore covered by the global seed too:
```rust
use ndarray::Array2;
use rustyml::neural_network::Shape;
use rustyml::neural_network::Tensor;
use rustyml::neural_network::layers::Activation;
use rustyml::neural_network::layers::dense::Dense;
use rustyml::neural_network::losses::MeanSquaredError;
use rustyml::neural_network::optimizers::SGD;
use rustyml::neural_network::sequential::SequentialBuilder;
use rustyml::set_global_seed;
fn t2(rows: usize, cols: usize, data: Vec<f32>) -> Tensor {
Array2::from_shape_vec((rows, cols), data).unwrap().into_dyn()
}
fn main() {
// One call up front. Every unseeded draw below derives from this, in order.
set_global_seed(2026);
#[rustfmt::skip]
let x = t2(4, 4, vec![
0.5, -1.0, 2.0, 0.25,
1.0, 0.0, -0.5, 1.5,
-2.0, 0.5, 1.0, -1.0,
0.25, 2.0, -1.5, 0.0,
]);
let y = t2(4, 1, vec![1.0, 0.0, -1.0, 0.5]);
// No per-layer seeds, no explicit set_seed. `build` draws sub-seed #1 for
// the first layer and sub-seed #2 for the second, in that order.
let mut model = SequentialBuilder::new()
.add(Dense::new(3, Activation::ReLU).unwrap())
.add(Dense::new(1, Activation::Linear).unwrap())
.build(&Shape::known(x.shape()))
.unwrap();
model.compile(SGD::new(0.05, 0.0, false, 0.0).unwrap(), MeanSquaredError::new());
// batch_size < n_samples exercises the per-epoch shuffle (sub-seed #3).
model.fit_with_batches(&x, &y, 5, 2).unwrap();
let p = model.predict(&t2(1, 4, vec![0.5, -1.0, 2.0, 0.25])).unwrap();
println!("prediction shape: {:?}", p.shape());
}
```
Run this program twice, and the trained weights come out byte-identical. The draw order, 2 layer builds then the shuffle RNG, pulls the same 3 sub-seeds from the `2026` stream each time. Reorder those `.add` calls, and the sub-seed assignment changes. That is the order-sensitivity from section 7.1.3, in concrete form.
### Per-component `random_state` overrides
Set `random_state` directly on each component when you want it pinned independently of construction order and of which thread runs it. This is the reliable choice for library code, parallel estimators, and code you refactor often. Explicit seeds ignore the global stream entirely. This also lets you reproduce a single component while leaving the rest free:
```rust
use ndarray::{Array1, Array2, array};
use rustyml::prelude::*;
fn main() {
// 2 well-separated blobs, 6 samples, 2 features.
let x: Array2<f64> = array![
[0.0, 0.0], [0.2, 0.1], [0.1, -0.2],
[5.0, 5.0], [5.2, 4.9], [4.8, 5.1],
];
let y: Array1<usize> = array![0, 0, 0, 1, 1, 1];
// Same random_state => same index shuffle => same split, every run.
let split_a = train_test_split(x.clone(), y.clone(), Some(0.5), Some(42)).unwrap();
let split_b = train_test_split(x.clone(), y.clone(), Some(0.5), Some(42)).unwrap();
assert_eq!(split_a.0, split_b.0); // x_train identical
assert_eq!(split_a.2, split_b.2); // y_train identical
// Same random_state => same k-means++ init => same labels, independent of the above.
let labels_1 = KMeans::new(2, 100, 1e-4)
.unwrap()
.with_random_state(7)
.fit_predict(&x)
.unwrap();
let labels_2 = KMeans::new(2, 100, 1e-4)
.unwrap()
.with_random_state(7)
.fit_predict(&x)
.unwrap();
assert_eq!(labels_1, labels_2);
println!("split and k-means both reproducible under explicit seeds");
}
```
You can mix the 2 styles freely. A common pattern uses `set_global_seed` for the ambient defaults. It adds an explicit `random_state` on the one estimator you need to hold fixed, while you sweep everything else. Explicit seeds are inert against the global stream. So pinning that one estimator does not disturb the sub-seeds every other component receives.