# 2. Classical Machine Learning
Classical machine learning covers everything in RustyML that is not a neural network: linear models, trees, kernel methods, clustering, and dimensionality reduction. These algorithms train fast, need little data, and produce models you can inspect. Reach for them first. Move to [Chapter 3](../Chapter-03/3.0._Neural_Networks.md) only when a problem needs a deep network. Every estimator in this chapter lives under `rustyml::machine_learning` and shares one small contract. Construct it with `new`, which validates its arguments and returns `Result`. `LinearRegression::new` is the one exception: it cannot fail, so it returns `Self` directly. Train the model with `fit`, and run inference with `predict`, unless the model reduces dimensionality. The dimensionality-reduction transformers use `transform` and `fit_transform` instead of `predict`. Learn this rhythm once, and it carries across all 14 models in this chapter.
Read [Chapter 1](../Chapter-01/1.0._Getting_Started.md) before this chapter. Read [Working with ndarray](../Chapter-01/1.3._Working_with_ndarray.md) first, since every model consumes an `Array2<f64>` feature matrix. Read [Error Handling](../Chapter-01/1.6._Error_Handling.md) too, since constructors and `fit`/`predict` all return the crate's `Result`. [Chapter 4](../Chapter-04/4.0._Data_Preprocessing.md) covers encoding and scaling your features. [Chapter 5](../Chapter-05/5.0._Model_Evaluation.md) covers the accuracy, silhouette, and R^2 scores you use to judge these models.
Every model in this chapter follows the shape below, with only the details changing:
```rust
use rustyml::machine_learning::LinearRegression;
use ndarray::array;
fn main() {
// construct -> fit -> predict, the rhythm every estimator repeats
let mut model = LinearRegression::new(true);
let x = array![[1.0, 2.0], [2.0, 3.0], [3.0, 4.0]];
let y = array![6.0, 9.0, 12.0];
model.fit(&x, &y).unwrap();
let preds = model.predict(&array![[4.0, 5.0]]).unwrap();
println!("prediction: {:?}", preds);
}
```
Each section stands alone, so jump straight to the model you need. Reading in order moves from the simplest estimators to the most involved.
Supervised learning predicts a target from labeled examples. [Linear Regression](./2.1._Linear_Regression.md) fits a continuous target, with optional L1/L2 regularization and a choice of a gradient-descent or a closed-form solver. It is the best place to learn the fit/predict loop. [Logistic Regression](./2.2._Logistic_Regression.md) reuses that gradient machinery for binary classification. [K-Nearest Neighbors](./2.3._K_Nearest_Neighbors.md) skips training and classifies by proximity, with a selectable distance metric and weighting scheme. [Decision Trees](./2.4._Decision_Trees.md) split the feature space into readable if/else rules, using the ID3, C4.5, or CART algorithm, with pruning. [Support Vector Machines](./2.5._Support_Vector_Machines.md) covers 2 models: a kernelized `SVC` (SMO solver) for curved boundaries, and a fast `LinearSVC` for wide, high-dimensional data. [Linear Discriminant Analysis](./2.6._Linear_Discriminant_Analysis.md) classifies and reduces dimensions at the same time, by modeling each class as a Gaussian with a shared covariance.
Clustering groups unlabeled data. [KMeans](./2.7._KMeans_Clustering.md) partitions points into a fixed number of clusters, using k-means++ initialization. It is fast, and the usual default choice. [DBSCAN](./2.8._DBSCAN.md) finds clusters of any shape by density, and labels outliers as noise. It needs no cluster count in advance. [Mean Shift](./2.9._Mean_Shift.md) also finds the number of clusters on its own, by climbing a density surface. Score all 3 methods with the [clustering metrics](../Chapter-05/5.3._Clustering_Metrics.md).
Dimensionality reduction compresses features while it keeps their structure. [Principal Component Analysis](./2.10._Principal_Component_Analysis.md) is the linear default for decorrelation and compression. [Kernel PCA](./2.11._Kernel_PCA.md) extends it to nonlinear structure, through RBF, polynomial, and other kernels. [t-SNE](./2.12._t-SNE.md) embeds high-dimensional data into 2 or 3 dimensions, for visualization only. It learns no reusable projection. It exposes `fit_transform` alone, with no out-of-sample `transform`.
Anomaly detection stands as its own family. [Isolation Forest](./2.13._Isolation_Forest.md) scores how easily each point isolates under random splits, and flags outliers without needing any labels.