# 2.13. Isolation Forest
`IsolationForest` is the unsupervised anomaly detector in RustyML.
Most anomaly detectors model where normal data lives, then flag points that fall outside that region. `IsolationForest` inverts this approach. It uses the fact that anomalies are *few and different*. A tree of random axis-aligned splits isolates an anomaly into its own leaf after only a few cuts. A point buried in a dense cluster needs many more cuts before the tree isolates it. RustyML averages this cut count over a forest of trees, then normalizes the result into an anomaly score.
This estimator matches `sklearn.ensemble.IsolationForest`, the same subsampling-based ensemble from Liu, Ting, and Zhou. The API now lines up with scikit-learn method for method. `score_samples`, `decision_function`, and `predict` mean what they mean in Python, including the sign of the scores. If you port code written against an older RustyML, read [Section 2.13.4](#2134-fitting-and-reading-scores) first. Both the score sign and the method names changed there.
## 2.13.1. The isolation principle and the anomaly score
Each tree grows on a random subsample of the data. The build step repeats one simple move. It picks a feature at random. It picks a split threshold at random, between that feature's minimum and maximum value in the current node. It sends points below the threshold to the left child and the rest to the right child.
The tree recurses until a node holds 1 point. It also stops when every value of the chosen feature is equal in that node, because there is nothing left to split. It stops too when it hits the depth cap. A point far from the bulk of the data needs only a handful of random cuts to isolate. A point in the middle of a dense cluster needs many more cuts.
The tree structure that records this stays minimal. An isolation tree does not need class counts or impurity. It only needs where it split and how many points reached each leaf:
```rust,ignore
pub enum IsolationTree {
Leaf { size: usize },
Internal {
feature: usize,
threshold: f64,
left: Box<IsolationTree>,
right: Box<IsolationTree>,
},
}
```
The raw signal for a sample is its **path length**, `h(x)`. This is the number of edges from the root to the leaf where the sample lands. A tree has a depth cap, so a leaf can still hold several points the tree did not fully separate. The path length adds a correction for the subtree that would have continued below that leaf. A leaf at depth `d` holding `size` points contributes `d + c(size)` to the path length.
`c(n)` estimates the average extra depth needed to isolate `n` points. `c(n)` is also the normalization constant. It equals the expected path length of a failed search in a binary search tree of `n` points:
`c(n) = 2 * H(n-1) - 2 * (n-1) / n`, where `H(m)` is the m-th harmonic number.
RustyML computes `H(m)` exactly, as a running sum, when `n <= 50`. Above that, it switches to the asymptotic form `ln(m) + gamma + 1 / (2 * m)`. Here `gamma` is the Euler-Mascheroni constant, and `m = n - 1`. This approximation keeps the error under `1e-3`. RustyML pins 2 edge cases: `c(n) = 0` when `n <= 1`, and `c(2) = 1`. The final score for a sample averages `h(x)` over every tree, then applies Liu et al.'s formula, negated:
`s(x) = -2^(-E[h(x)] / c(n))`.
The score lands in the range `[-1, 0)`. A *lower* score means more anomalous. This is the reverse of the earlier RustyML convention.
A *short* average path is easy to isolate, so it is anomalous. It drives the score toward **-1**. A *long* average path is hard to isolate, so it is normal. It drives the score toward **0**.
The negation puts the score on the usual convention where negative marks the rejected class. It also makes `score_samples` numerically identical to scikit-learn's version.
The neutral point is `s = -0.5`. At this point, a sample's expected path length equals `c(n)`, so its isolation cost matches the tree average. Liu et al. call this the "no distinct anomaly" regime. [`Contamination::Auto`](#2136-turning-scores-into-labels-contamination) uses `-0.5` as its cutoff. Treat `-0.5` as *no signal*, not as *confirmed normal*. A point earns "normal" only when it scores clearly above `-0.5`.
The `c(n)` in the denominator uses `n = sample_size`. This is the realized per-tree subsample size, covered in the next 2 sections. It is not `max_samples`. This distinction matters when your dataset is smaller than `max_samples`, because normalizing by the wrong `n` would shift every score.
RustyML's test suite checks the closed form directly. It fits identical points with `max_samples >= n_rows`. Every score comes out exactly `-0.5`, because `E[h(x)] = c(sample_size)` cancels the exponent.
## 2.13.2. Constructing the forest
`IsolationForest` has 2 entry points. `IsolationForest::default()` gives the standard configuration. `IsolationForest::new(n_estimators, max_samples)` sets the 2 structural parameters. It validates them and returns `Result<Self, Error>`. 3 builder methods refine the result further. Each one consumes and returns `self`, so you can chain them:
```rust,ignore
impl IsolationForest {
pub fn new(n_estimators: usize, max_samples: usize) -> Result<Self, Error>;
pub fn with_max_depth(self, max_depth: usize) -> Result<Self, Error>;
pub fn with_random_state(self, seed: u64) -> Self;
pub fn with_contamination(self, contamination: Contamination) -> Result<Self, Error>;
}
```
```rust
use rustyml::machine_learning::IsolationForest;
fn main() {
// Standard config: 100 trees, subsample of 256 rows each,
// depth auto = ceil(log2(256)) = 8, no seed.
let _a = IsolationForest::default();
// Explicit config, with a seed and a hand-set depth cap that overrides the auto value.
let forest = IsolationForest::new(100, 256)
.unwrap()
.with_max_depth(10)
.unwrap()
.with_random_state(42);
assert_eq!(forest.get_n_estimators(), 100);
assert_eq!(forest.get_max_samples(), 256);
assert_eq!(forest.get_max_depth(), 10);
assert_eq!(forest.get_random_state(), Some(42));
}
```
`new` rejects `n_estimators == 0` or `max_samples == 0`. It returns [`Error::InvalidParameter`](../Chapter-01/1.6._Error_Handling.md), naming the field that failed. `with_max_depth(0)` rejects the same way.
Note one asymmetry. An explicit depth of 0 is invalid. The *auto-computed* depth can still be 0. For example, `max_samples == 1` gives `ceil(log2(1)) = 0`. This is fine, because a subsample of 1 point has nothing to split anyway.
| `n_estimators` | `new` / `default` | `100` | greater than 0 |
| `max_samples` | `new` / `default` | `256` | greater than 0 |
| `max_depth` | auto or `with_max_depth` | `ceil(log2(max_samples))` = `8` | greater than 0 if set explicitly |
| `random_state` | `with_random_state` | `None` (entropy-seeded) | any `u64` |
| `contamination` | `with_contamination` | `Contamination::Auto` | a `Fraction(c)` must be finite and in `(0.0, 0.5]` |
Every field has a getter. `get_n_estimators`, `get_max_samples`, `get_max_depth`, `get_random_state` (returns `Option<u64>`), and `get_contamination` work right after construction. 4 more getters make sense only after fitting: `get_n_features`, `get_sample_size`, `get_offset` (returns `Option<f64>`, the resolved score cutoff), and `get_trees` (returns `Option<&Vec<IsolationTree>>`, `None` until fitted).
Understand the auto depth before you override it. `ceil(log2(max_samples))` estimates the height of a balanced tree over the subsample. Anomalies are roughly isolated by that height already. Splitting deeper mostly separates points inside dense normal clusters. The `c(size)` leaf correction already accounts for that effect.
A bigger `max_depth` buys almost no accuracy, and it costs more build time. Leave `max_depth` on auto, unless you have a specific reason to change it.
## 2.13.3. Subsampling, `sample_size`, and why 256
Isolation Forest makes an unusual design choice. Each tree trains on a small random subsample, not on the whole dataset. This is a feature, not a shortcut. RustyML draws `sample_size = min(max_samples, n_rows)` rows for each tree, without replacement, through a partial Fisher-Yates shuffle.
The default `max_samples = 256` comes directly from the original paper. Small subsamples defeat 2 failure modes of density-based detectors. **Swamping** happens when normal points near a cluster of anomalies start to look anomalous. **Masking** happens when a dense clump of anomalies hides its own members from each other. Both problems get worse as each tree sees more data, because large samples let anomalies form their own mini-clusters. Those mini-clusters are no longer easy to isolate.
A 256-row subsample is large enough to profile the shape of the normal region. It is also small enough that anomalies stay sparse and stay easy to isolate. A bigger subsample mostly buys higher build cost and *worse* detection quality. That is why `max_samples` is a subsample budget, not a "use more data" knob.
This design has 2 consequences. First, when your dataset has fewer rows than `max_samples`, `sample_size` clamps to `n_rows`. Every tree then sees the whole dataset. The forest still works, because each tree still differs through its random splits. It just no longer subsamples. `fit` checks for this case and never panics.
Second, normalization uses `c(sample_size)`, not `c(max_samples)`. RustyML reads `sample_size` back from the fitted model instead of assuming `max_samples`. `get_sample_size()` reports the realized value after `fit`.
## 2.13.4. Fitting and reading scores
`fit` takes a 2-D `f64` array, where rows are samples and columns are features. It returns `Result<&mut Self, Error>`. `fit` records `n_features` and `sample_size`, builds every tree, then resolves the `contamination` rule into a stored score cutoff, the `offset`. Scoring then exposes 4 surfaces. Each one shifts or thresholds the surface before it, in a chain:
```rust,ignore
pub fn fit<S>(&mut self, x: &ArrayBase<S, Ix2>) -> Result<&mut Self, Error>
where S: Data<Elem = f64> + Send + Sync;
pub fn score_samples<S>(&self, x: &ArrayBase<S, Ix2>) -> Result<Array1<f64>, Error>
where S: Data<Elem = f64>; // 1 score per row, in [-1, 0). Lower means more anomalous
pub fn score_sample(&self, sample: &[f64]) -> Result<f64, Error>; // 1 row, as a slice
pub fn decision_function<S>(&self, x: &ArrayBase<S, Ix2>) -> Result<Array1<f64>, Error>
where S: Data<Elem = f64>; // score_samples(x) minus offset. Negative marks an outlier
pub fn predict<S>(&self, x: &ArrayBase<S, Ix2>) -> Result<Array1<i32>, Error>
where S: Data<Elem = f64>; // sign of decision_function: {-1 outlier, +1 inlier}
```
Each name means exactly what it means in scikit-learn. `score_samples` gives the raw anomaly score. `decision_function` shifts that score so zero marks the decision boundary. `predict` gives the sign of the decision value. A strictly negative value gives `-1`. Everything else gives `+1`, so a sample that lands *exactly* on the cutoff counts as an inlier.
Pick the method the task needs. Use `score_samples` to rank samples, set your own cutoff, or feed a downstream calibrator. Use `decision_function` for a signed margin. Use `predict` for a hard `{-1, +1}` decision.
Earlier RustyML versions differ here in 2 ways. Both changes alter results silently, instead of failing to compile.
First, the scores are now **negated**. The old scale was `[0, 1]`, where a higher score meant more anomalous. The new scale is `[-1, 0)`, where a *lower* score means more anomalous. Flip every comparison that ranks or thresholds a score.
Second, the per-sample slice method is now called **`score_sample`** (singular). It replaces the old `anomaly_score` method. The old batch form, where `predict` returned scores, is gone. The old `predict_labels(&x, contamination)` method is gone too. `score_samples` replaces the first. `predict`, together with the fitted `contamination` rule, replaces the second.
`score_sample` scores one point at a time, for a live stream or an ad-hoc query. It takes a `&[f64]` slice instead of a matrix. `fit_predict` runs `fit`, then `predict`, on the same matrix. It returns labels, the standard "label the training set" convenience for unsupervised models.
```rust
use ndarray::array;
use rustyml::machine_learning::IsolationForest;
fn main() {
let train = array![[0.0, 0.0], [0.1, 0.0], [0.0, 0.1], [0.1, 0.1], [30.0, 30.0]];
let mut forest = IsolationForest::new(80, 32).unwrap().with_random_state(1);
forest.fit(&train).unwrap();
// Score points one at a time through the slice API.
let normal = forest.score_sample(&[0.05, 0.05]).unwrap();
let weird = forest.score_sample(&[30.0, 30.0]).unwrap();
assert!(weird < normal); // lower score means more anomalous
// The batch form, and the same values shifted by the fitted cutoff.
let scores = forest.score_samples(&train).unwrap();
let decision = forest.decision_function(&train).unwrap();
let offset = forest.get_offset().unwrap(); // -0.5 under Contamination::Auto
assert!((decision[0] - (scores[0] - offset)).abs() < 1e-12);
}
```
Every entry point validates its input and returns a typed [`Error`](../Chapter-01/1.6._Error_Handling.md). `fit` returns `Error::EmptyInput` on zero rows. `fit` returns `Error::NonFinite` on any `NaN` or infinite value. `score_samples` also checks `Error::DimensionMismatch`, when the feature count differs from training, and `Error::NotFitted`, before `fit` runs.
`decision_function` and `predict` propagate those same errors unchanged. `score_sample` checks only `Error::NotFitted` and `Error::DimensionMismatch` for its single slice. It does not check the slice for `NaN` or infinite values.
The batch calls accept any `ArrayBase` backing. This includes non-contiguous views, such as a transposed array. You do not need to copy `.t()` into a fresh buffer before scoring.
## 2.13.5. A worked example: injecting outliers into a blob
This is the standard sanity check for an anomaly detector. Build a tight cluster of inliers. Drop in a few points that clearly do not belong. Confirm that the forest ranks those points at the top.
```rust
use ndarray::Array2;
use rustyml::machine_learning::{Contamination, IsolationForest};
fn main() {
// 20 inliers in a tight blob near (5, 5)...
let mut rows: Vec<f64> = Vec::new();
for i in 0..20 {
let t = i as f64;
rows.push(5.0 + 0.15 * t.sin());
rows.push(5.0 + 0.15 * t.cos());
}
// ...plus 3 injected outliers far from the cluster.
for &(x, y) in &[(30.0, 30.0), (-20.0, 40.0), (50.0, -10.0)] {
rows.push(x);
rows.push(y);
}
let n = rows.len() / 2; // 23 rows
let data = Array2::from_shape_vec((n, 2), rows).unwrap();
// A 15% contamination budget, resolved into a fixed cutoff at fit time.
let mut forest = IsolationForest::new(100, 256)
.unwrap()
.with_random_state(42)
.with_contamination(Contamination::Fraction(0.15))
.unwrap();
let labels = forest.fit_predict(&data).unwrap();
let scores = forest.score_samples(&data).unwrap();
// The 3 injected rows (indices 20..23) should score well below the blob.
for i in (n - 3)..n {
println!("outlier row {i}: score {:.3}", scores[i]);
}
let flagged: Vec<usize> = labels
.iter()
.enumerate()
.filter(|&(_, &l)| l == -1)
.map(|(i, _)| i)
.collect();
println!("cutoff (offset): {:.3}", forest.get_offset().unwrap());
println!("flagged as outliers (-1): {flagged:?}");
}
```
The 3 outlier points land near the bottom of the score range. The blob rows sit close to zero:
```text
outlier row 20: score -0.735
outlier row 21: score -0.779
outlier row 22: score -0.788
cutoff (offset): -0.419
flagged as outliers (-1): [2, 20, 21, 22]
```
Note what the cutoff is, and what it is not. `Contamination::Fraction(0.15)` sets the offset to the 15th percentile of the *training* scores. It separates off roughly the lowest 15% of samples. Here that is 4 rows out of 23, 1 more than the 3 rows actually injected.
The exact count depends on where the scores fall. It does not depend on a fixed `ceil(0.15 * n)`, so an inlier can get swept in. This is what contamination *means*, and it motivates the next section.
## 2.13.6. Turning scores into labels: contamination
`contamination` is a rule you set on the builder. It is not an argument to a prediction call. `fit` resolves it into a single stored number, the `offset`:
```rust,ignore
pub enum Contamination {
Auto, // the paper's cutoff: offset = -0.5
Fraction(f64), // offset = the 100*c-th percentile of the TRAINING scores. c is in (0.0, 0.5]
}
```
`Contamination::Auto` is the default. It pins the cutoff at `-0.5`, the score where a sample isolates exactly as fast as the forest average. `Contamination::Fraction(c)` instead reads the cutoff from the training scores, at the `100*c`-th percentile. It uses NumPy's linear interpolation for that percentile. This choice makes `get_offset()` equal scikit-learn's `offset_` numerically. It does more than flag a comparable number of rows.
A `Fraction` value must be finite and inside `(0.0, 0.5]`. Otherwise, `with_contamination` returns `Error::InvalidParameter`. The upper bound of `0.5` encodes the assumption that anomalies are the minority.
Resolving the cutoff at fit time is the whole point of this design. The rule becomes model state. A sample gets the same label whether you score it alone, in a slice, or in the full batch.
The earlier design worked differently. It took a quantile of whatever batch it received, so it was transductive. A single-row call always came back `-1`. Splitting a test set in half changed its labels.
Changing the rule after fitting requires a new call to `fit`, because RustyML computes the offset from the training scores.
Contamination is still a *budget*, not a discovery. You tell the model what proportion of the training data you expect to be anomalous. `fit` places the cutoff accordingly, whether or not that many anomalous points actually exist. If you overestimate, the cutoff sweeps in normal points as false positives, as the worked example showed. If you underestimate, the cutoff labels real anomalies as inliers. The model cannot know the true rate, so there is no way around this tradeoff.
When you have ground-truth labels for a validation set, sweep the fraction. Pick the value that best trades precision against recall for your cost structure. The tools in [5.2. Classification Metrics](../Chapter-05/5.2._Classification_Metrics.md) apply here, if you treat `-1` as the positive class.
When you do not have ground truth, prefer the raw `score_samples` values. Threshold them against a level you can justify. For example, use a percentile from a clean reference window or an absolute score cutoff. Do not just commit to a fixed proportion. If your idea of "outlier" is really "sparse region," rather than "few and different," compare against [DBSCAN](./2.8._DBSCAN.md), which labels low-density points as noise.
## 2.13.7. Seeding and reproducible forests
An unseeded forest draws its randomness from entropy, so 2 fits differ. Pass `with_random_state(seed)` to make the whole forest reproducible. The same seed and the same data give bit-identical scores.
```rust
use ndarray::array;
use rustyml::machine_learning::IsolationForest;
fn main() {
let data = array![[0.0, 0.0], [0.5, 0.5], [1.0, 1.0], [2.0, 2.0], [50.0, 50.0]];
let mut a = IsolationForest::new(30, 20).unwrap().with_random_state(13);
a.fit(&data).unwrap();
let sa = a.score_samples(&data).unwrap();
let mut b = IsolationForest::new(30, 20).unwrap().with_random_state(13);
b.fit(&data).unwrap();
let sb = b.score_samples(&data).unwrap();
assert_eq!(sa, sb); // bit-identical, regardless of thread scheduling
}
```
This determinism is stronger than it looks, and the reason matters for the parallel build described in the next section. RustyML does not share a single RNG across the trees. Tree `i` gets its own generator, seeded with `seed.wrapping_add(i)`. Each tree's seed depends only on its index.
So the forest comes out identical whether the trees build serially or across rayon threads, in any order the scheduler picks. Parallelism can never change the result. This is a deliberate contrast with the neural-network components in this crate, where reproducibility can depend on order.
You can fix randomness globally instead of passing a seed to every constructor. `rustyml::set_global_seed` seeds an unseeded forest from the process-global, thread-local, random stream:
```rust
use ndarray::array;
use rustyml::machine_learning::IsolationForest;
use rustyml::set_global_seed;
fn main() {
set_global_seed(2026);
let data = array![[0.0, 0.0], [0.5, 0.5], [1.0, 1.0], [50.0, 50.0]];
// No with_random_state: per-tree seeds derive from the global stream.
let mut forest = IsolationForest::new(30, 16).unwrap();
let _labels = forest.fit_predict(&data).unwrap();
}
```
An explicit `with_random_state` always wins over the global seed. It also never consumes the global stream, so mixing the two stays predictable. The global seed is thread-local. Set it on the same thread that builds the model. See [7.1. Reproducibility and Random Seeds](../Chapter-07/7.1._Reproducibility_and_Random_Seeds.md) for the full seed-resolution rules.
## 2.13.8. Parallel training and prediction
Training and scoring both parallelize through rayon. A gate keeps tiny workloads serial, to avoid fork/join overhead. `fit` builds trees across a `into_par_iter` once `n_estimators` reaches its threshold of 10 trees. The default 100-tree forest always trains in parallel. Each tree is an independent subsample-and-build step, with its own index-seeded RNG. The work parallelizes cleanly, with no shared mutable state and no effect on the result.
`score_samples` parallelizes over *rows* once the estimated traversal work clears the calibrated tree-traversal gate. That work estimate is samples times trees times average path length. Below the gate, small batches score serially. `decision_function` and `predict` inherit this behavior, because they are thin wrappers over `score_samples`. `score_sample` always runs serially, because it handles only a single sample.
Isolation Forest scales well on wide forests and large scoring batches. Training cost is roughly `n_estimators * sample_size * log(sample_size)`, and it is embarrassingly parallel across trees. Per-sample scoring is `O(n_estimators * tree_depth)`, and it is independent across rows. See [7.3. Performance Tuning and Parallelism](../Chapter-07/7.3._Performance_Tuning_and_Parallelism.md) to tune the gates or study the cost model.
## 2.13.9. Saving and loading a forest
A fitted forest serializes to a compact postcard binary, through `save_to_path` and `load_from_path`. Every RustyML estimator exposes this same pair of methods. The whole state travels: every tree, the hyperparameters, `n_features`, `sample_size`, and the fitted `offset`. A reloaded model scores *and labels* identically to the original, byte for byte.
```rust
use ndarray::array;
use rustyml::machine_learning::IsolationForest;
fn main() {
let data = array![[0.0, 0.0], [0.1, 0.1], [0.2, 0.0], [10.0, 10.0]];
let mut forest = IsolationForest::new(50, 32).unwrap().with_random_state(7);
forest.fit(&data).unwrap();
let before = forest.score_samples(&data).unwrap();
let path = "isolation_forest_model.bin";
forest.save_to_path(path).unwrap();
let loaded = IsolationForest::load_from_path(path).unwrap();
let after = loaded.score_samples(&data).unwrap();
assert_eq!(before, after); // scoring survives the round trip exactly
std::fs::remove_file(path).unwrap();
}
```
`load_from_path` returns `Error::Io` when the file is missing, or when the bytes are not a valid serialized forest. RustyML persists `sample_size`, so a loaded model keeps the correct `c(n)` normalization, even though it never sees the training data again. Its scores stay on the same scale as before you saved it.
One migration note applies. The stored `offset` changed sign along with the scores. A forest saved by an older version resolves its cutoff on the wrong side of zero. Re-fit and re-save any persisted forest. For versioning and format details across the crate, see [7.2. Model Persistence in Depth](../Chapter-07/7.2._Model_Persistence_in_Depth.md).