# 2.5. Support Vector Machines
A support vector machine finds the decision boundary that sits as far as possible from the nearest points of each class. This is the maximum-margin hyperplane. RustyML ships 2 implementations that reach this goal from opposite directions.
`SVC` is the kernelized version. It lifts the data into a higher-dimensional space through a kernel function and finds a linear boundary there. Back in the original space, that boundary becomes curved. `LinearSVC` skips the kernel. It fits a straight hyperplane by minimizing hinge loss with stochastic gradient descent.
Both models are strictly binary classifiers, with no built-in one-vs-rest wrapper. Both use the same `{0.0, 1.0}` label domain, so you can swap one for the other without a rewrite of the target array. Past that shared goal, the 2 models differ in solver, regularization, and cost. The rest of this page helps you pick the right one for a problem.
## 2.5.1. When to reach for which
The choice comes down to 2 numbers: the sample count (`n_samples`) and the feature count (`n_features`). It also depends on whether a straight line can separate the classes.
`SVC` builds an `n_samples` x `n_samples` kernel matrix, also called the Gram matrix, before training starts. The Sequential Minimal Optimization (SMO) solver then works over that matrix. The matrix sets a hard limit: it costs `O(n_samples^2)` in memory and `O(n_samples^2 * n_features)` to build. At 10,000 samples, the Gram matrix alone takes about 800 MB of `f64` values. At 260,000 samples, it would need hundreds of gigabytes.
`SVC` fits well when the boundary is truly nonlinear and `n_samples` stays modest, from hundreds to a few thousand rows. It does not scale to large datasets. That limit comes from the algorithm, not from this implementation.
`LinearSVC` never forms a kernel matrix. Each epoch runs a few matrix-vector passes over the data. That costs `O(n_samples * n_features)` in time and only `O(n_features)` in memory for the weight vector. It scales linearly in both dimensions, which makes it the default choice for wide, high-dimensional, or large-sample problems. Text classification with tens of thousands of sparse features is the classic case.
The catch is that `LinearSVC` can only fit a linear boundary. If the classes are not linearly separable, no amount of training fixes that. Use `SVC` with a kernel instead.
| | `SVC` | `LinearSVC` |
|---|---|---|
| Boundary | linear or nonlinear (via kernel) | linear only |
| Solver | Sequential Minimal Optimization (dual) | minibatch SGD (primal) |
| Label domain | `0.0` / `1.0` | `0.0` / `1.0` |
| Memory | `O(n_samples^2)` Gram matrix | `O(n_features)` weights |
| Scales with `n_samples` | poorly, the `n^2` memory wall | linearly |
| Regularization knob | `C` (bounds the dual variables) | L1 / L2 penalty with strength `lambda` |
| Loss | hinge (solved in the dual) | `Hinge` or `SquaredHinge` |
Watch for one trap. If you need only a linear boundary on a large dataset, do not use `SVC` with `KernelType::Linear`. That choice still pays the full `O(n_samples^2)` Gram-matrix cost, for a problem `LinearSVC` solves in linear memory. `SVC` with a linear kernel earns its cost only on small data, where you want the exact max-margin dual solution.
Both estimators are sensitive to feature scale. The RBF kernel measures squared Euclidean distance, and the SGD in `LinearSVC` takes fixed-size steps in feature space. Standardize your columns first (see [Standardization and Normalization](../Chapter-04/4.2._Standardization_and_Normalization.md)), unless they already share a range.
Both estimators also share one hard rule for `y`: every entry must be exactly `0.0` or `1.0`. Any other value, such as a `-1.0` from a textbook derivation, a `2.0`, or a stray `0.5`, causes an `Error::InvalidInput` at `fit`. Neither estimator remaps labels silently. If your labels are strings or arbitrary integers, run them through [Label Encoding](../Chapter-04/4.3._Label_Encoding.md) first.
## 2.5.2. SVC: kernels and the SMO solver
Construct an `SVC` with `SVC::new(kernel, regularization_param, tol, max_iter)`. This call validates the arguments and returns a `Result`. Set the seed for reproducible training in a separate builder step, with `with_random_state`.
| Parameter | Type | Meaning | Validation |
|---|---|---|---|
| `kernel` | `KernelType` | kernel function (see below) | n/a |
| `regularization_param` (C) | `f64` | trades margin width against training error | must be positive and finite |
| `tol` | `f64` | KKT stopping tolerance for SMO | must be positive and finite |
| `max_iter` | `usize` | iteration cap for the SMO outer loop | must be non-zero |
`C` is the regularization knob. It sets the upper bound on each dual coefficient, `0 <= alpha <= C`. A large `C` lets the solver push the alphas high, fitting every training point with a narrow margin that tolerates few violations. A small `C` keeps the alphas small, widening the margin and accepting more slack. This is the opposite of the `lambda` intuition from linear models: a larger `C` means less regularization. Any non-positive or non-finite value fails at construction with `Error::InvalidParameter`.
`SVC::default()` gives an RBF kernel with `gamma = 0.1`, `C = 1.0`, `tol = 0.001`, and `max_iter = 1000`. This is a reasonable start when you have no prior information, though `gamma` almost always needs tuning.
Training runs Sequential Minimal Optimization (SMO). It repeatedly picks a pair of dual variables that violate the Karush-Kuhn-Tucker (KKT) conditions. It optimizes that pair analytically, holding the rest fixed. This repeats until the whole set satisfies the KKT conditions within `tol`, or until training hits the iteration cap.
The dual problem uses `+1`/`-1` targets, because the `y_i * y_j` products in its objective only make sense with a sign. `fit` converts your `{0.0, 1.0}` column to `+1`/`-1` once, on entry, and keeps that signed form internal. `predict` maps the sign back to `{0.0, 1.0}`. You never pass `+1`/`-1` labels in, and, with one exception noted below, you never get them out either.
The Gram matrix and the initial error cache can build in parallel. The SMO inner loop stays sequential on purpose, so the optimization path stays reproducible for a given seed.
This example runs the whole construct-fit-predict loop with a linear kernel, on data a straight line separates cleanly:
```rust
use rustyml::machine_learning::{KernelType, SVC};
use ndarray::array;
fn main() {
// Class 1 sits upper-right. Class 0 sits lower-left. A line separates them.
let x = array![
[2.0, 2.0], [3.0, 2.0], [2.0, 3.0], [3.0, 3.0],
[-2.0, -2.0], [-3.0, -2.0], [-2.0, -3.0], [-3.0, -3.0],
];
let y = array![1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0];
let mut svc = SVC::new(KernelType::Linear, 10.0, 1e-3, 1000)
.unwrap()
.with_random_state(42);
svc.fit(&x, &y).unwrap();
// predict emits labels in {0.0, 1.0}, the same domain fit received
let preds = svc.predict(&x).unwrap();
println!("predictions: {:?}", preds);
// decision_function returns the raw signed distance to the hyperplane
let scores = svc.decision_function(&x).unwrap();
println!("scores: {:?}", scores);
}
```
`predict` sets the threshold at zero. It maps `>= 0.0` to `1.0` and everything else to `0.0`. `decision_function` hands back the raw signed score. A positive score means the class-`1.0` side, and its size shows how far the point sits from the boundary. Use `decision_function` when you want a confidence-like ordering instead of a hard label. There is also `fit_predict`, which fits and then predicts on the same matrix in 1 call.
### The kernel zoo and the gamma coefficient
`KernelType` is the same enum that [Kernel PCA](./2.11._Kernel_PCA.md) uses, so learning it here pays off twice. It has 5 variants.
| Variant | `K(x, y)` | Fields |
|---|---|---|
| `Linear` | `x*y` | none |
| `Poly { degree, gamma, coef0 }` | `(gamma*x*y + coef0)^degree` | `degree: u32`, `gamma: Gamma`, `coef0: f64` |
| `RBF { gamma }` | `exp(-gamma*\|\|x-y\|\|^2)` | `gamma: Gamma` |
| `Sigmoid { gamma, coef0 }` | `tanh(gamma*x*y + coef0)` | `gamma: Gamma`, `coef0: f64` |
| `Cosine` | `(x*y) / (\|\|x\|\| * \|\|y\|\|)` | none |
`RBF` is the default kernel and the one to try first for nonlinear data. It is a smooth, local similarity measure that only needs `gamma` tuned. `Poly` adds explicit interaction terms up to `degree`, but high degrees overflow quickly. A degree-400 polynomial on modestly large inputs pushes the decision value to infinity, which surfaces as `Error::NonFinite` at predict time.
`Sigmoid` mimics a two-layer network, but it is not positive-definite for all parameters, so it can behave erratically. `Cosine` measures angle rather than distance. It guards against zero vectors by returning `0.0` for them.
The `gamma` field is not a bare `f64`. It is a `Gamma` enum with 3 variants, matching scikit-learn's `'scale'`, `'auto'`, and explicit choices:
- `Gamma::Value(v)`: an explicit coefficient you supply.
- `Gamma::Scale`: resolved at fit time to `1 / (n_features * X.var())`, where `X.var()` is the population variance of all entries of the training matrix.
- `Gamma::Auto`: resolved at fit time to `1 / n_features`.
`Scale` and `Auto` are data-dependent, so they stay placeholders until `fit` sees the data. `fit` resolves them once and stores the concrete value. So a call to `get_kernel()` after `fit` returns a `KernelType` whose `gamma` is now a `Gamma::Value`. `Scale` fails with `Error::InvalidInput` if the data has zero variance, for example when every feature is constant, because the formula would divide by zero.
`gamma` controls how far a single training point's influence reaches. A large `gamma` makes the RBF bumps tight and the boundary wiggly, a fast route to overfitting. A small `gamma` makes the bumps broad and the boundary smooth. It is the parameter you will tune most.
### RBF on a problem no line can split
A kernel exists to separate data that a hyperplane cannot. Two concentric rings are the canonical example: an inner ring of one class, surrounded by an outer ring of the other. No straight line divides them, and a linear kernel is helpless here. The RBF kernel, which scores points by proximity, classifies every one correctly.
```rust
use rustyml::machine_learning::{Gamma, KernelType, SVC};
use ndarray::array;
fn main() {
// Inner ring (radius 1) is class 1, outer ring (radius 5) is class 0.
let x = array![
[1.0, 0.0], [-1.0, 0.0], [0.0, 1.0], [0.0, -1.0],
[5.0, 0.0], [-5.0, 0.0], [0.0, 5.0], [0.0, -5.0],
];
let y = array![1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0];
let mut svc = SVC::new(
KernelType::RBF { gamma: Gamma::Value(0.5) },
10.0, // C
1e-3, // tol
5000, // max_iter, nonlinear problems need more SMO passes
)
.unwrap()
.with_random_state(42);
svc.fit(&x, &y).unwrap();
let preds = svc.predict(&x).unwrap();
let correct = preds.iter().zip(y.iter()).filter(|&(p, t)| p == t).count();
println!("RBF accuracy on rings: {}/{}", correct, y.len());
}
```
Swap `KernelType::RBF { .. }` for `KernelType::Linear` on this dataset, and the classifier can no longer place every point on the right side. The rings are not linearly separable. That is exactly the failure mode a kernel exists to fix. Nonlinear problems also generally need a higher `max_iter` than linear ones, because the SMO solver has more support vectors to settle.
### Inspecting the fitted model
After `fit`, `SVC` exposes the pieces of the trained model through getters. This is more than most estimators in this guide offer. The support vectors are the only training rows that matter to prediction, the ones whose dual coefficient came out non-zero. You can read them back directly:
```rust
use rustyml::machine_learning::{Gamma, KernelType, SVC};
use ndarray::array;
fn main() {
let x = array![
[2.0, 2.0], [3.0, 2.0], [2.0, 3.0], [3.0, 3.0],
[-2.0, -2.0], [-3.0, -2.0], [-2.0, -3.0], [-3.0, -3.0],
];
let y = array![1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0];
let mut svc = SVC::new(
KernelType::RBF { gamma: Gamma::Value(0.5) },
5.0, 1e-3, 1000,
)
.unwrap()
.with_random_state(42);
svc.fit(&x, &y).unwrap();
// Only rows with a non-zero alpha survive as support vectors.
let support_vectors = svc.get_support_vectors().unwrap();
let alphas = svc.get_alphas().unwrap();
let labels = svc.get_support_vector_labels().unwrap();
println!(
"kept {} of {} rows as support vectors",
support_vectors.nrows(),
x.nrows()
);
println!("dual coefficients (alphas): {:?}", alphas);
println!("support-vector labels (SMO's internal +/-1): {:?}", labels);
println!("bias: {:?}", svc.get_bias());
println!("SMO iterations actually run: {:?}", svc.get_actual_iterations());
println!("resolved kernel: {:?}", svc.get_kernel());
}
```
`get_support_vectors` returns `Option<&Array2<f64>>`. `get_alphas` and `get_support_vector_labels` return `Option<&Array1<f64>>`. `get_bias` returns `Option<f64>`. All 4 are `None` before `fit` and `Some` after.
`get_support_vector_labels` is the one crack in the wall around the dual. It hands back the internal `+1`/`-1` encoding, not the `{0.0, 1.0}` you handed to `fit`. So a `-1.0` in that array denotes class `0.0`. The label stays signed because `predict` folds it straight into the `alpha_i * y_i` coefficients. Threshold it yourself if you want the caller's domain back.
`get_actual_iterations` reports how many SMO outer passes actually ran, always a number in `[1, max_iter]`. That count tells you whether the solver converged or hit the cap. A count equal to `max_iter` is a hint to raise the cap or loosen `tol`. `get_kernel` returns the resolved kernel. That is how you read back the concrete `gamma` that `Gamma::Scale` or `Gamma::Auto` produced.
Training can find no support vectors at all, for instance with single-class data where there is nothing to separate. In that case, `fit` returns `Error::NotConverged` instead of a degenerate all-zero model.
## 2.5.3. LinearSVC: hinge loss in the primal
`LinearSVC` solves the primal problem directly with minibatch stochastic gradient descent, minimizing hinge loss plus a regularization penalty. Construct it with `LinearSVC::new(max_iter, learning_rate, penalty, fit_intercept, tol)`.
| Parameter | Type | Meaning | Validation |
|---|---|---|---|
| `max_iter` | `usize` | maximum epochs | must be non-zero |
| `learning_rate` | `f64` | SGD step size | must be positive and finite |
| `penalty` | `RegularizationType` | `L1(lambda)` or `L2(lambda)` | `lambda` must be non-negative and finite |
| `fit_intercept` | `bool` | whether to learn a bias term | n/a |
| `tol` | `f64` | convergence tolerance on parameter change | must be positive and finite |
The labels are the same `0.0`/`1.0` that `SVC` takes. The story underneath is the same too: hinge loss needs signed targets, so `fit` remaps them to `-1`/`+1` on entry. The difference is that nothing on `LinearSVC`'s public surface ever leaks that encoding back. The fitted state is a weight vector and a bias, and both live in feature space, not label space.
```rust
use rustyml::machine_learning::{LinearSVC, RegularizationType};
use ndarray::array;
fn main() {
// The same 0.0 / 1.0 labels SVC takes. No remapping between the two.
let x = array![
[-5.0, 0.0], [-6.0, 0.0], [-7.0, 0.0], [-4.0, 0.0],
[5.0, 0.0], [6.0, 0.0], [7.0, 0.0], [4.0, 0.0],
];
let y = array![0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0];
let mut model = LinearSVC::new(
5000, // max_iter
0.01, // learning_rate
RegularizationType::L2(0.1), // penalty
true, // fit_intercept
1e-5, // tol
)
.unwrap()
.with_random_state(42);
model.fit(&x, &y).unwrap();
let preds = model.predict(&x).unwrap();
let scores = model.decision_function(&x).unwrap();
println!("labels: {:?}", preds);
println!("scores: {:?}", scores);
println!("weights: {:?}", model.get_weights().unwrap());
println!("bias: {:?}", model.get_bias().unwrap());
println!("stopped after {:?} epochs", model.get_actual_iterations());
}
```
`fit` shapes training with 2 details. It picks the minibatch size automatically, as `clamp(n_samples / 10, 32, 512)`. You do not set that size. Each epoch shuffles the sample order before it slices the data into minibatches. This is why the seed matters (see below).
`fit` checks convergence on the root-mean-square change in the weights and bias between epochs. Training stops early once that change drops below `tol`. The default `Loss::Hinge` keeps a constant-size gradient near the margin, so a fixed learning rate often leaves the weights oscillating instead of settling. The example above shows this: it reaches the full 5000-epoch cap instead of stopping early. `Loss::SquaredHinge` has a gradient that shrinks to zero near the margin. Training with that loss usually stops well under `max_iter`, as the example below shows.
Left unchecked, a runaway `learning_rate` can push the weights to non-finite values. `fit` catches that and reports `Error::NonFinite` mid-training instead of returning garbage. Use a smaller `learning_rate` or stronger regularization to fix it.
Watch one threshold difference from `SVC`. `LinearSVC::predict` maps a decision value `> 0.0` to class `1.0`, and everything else, including exactly `0.0`, to class `0.0`. `SVC` maps `>= 0.0` to `1.0` instead. Because both models share the same label domain, this is a real behavioral difference, not a bookkeeping detail. The 2 models break a zero score toward opposite classes.
`decision_function` here returns `x * weights + bias`. With `fit_intercept = false`, the bias stays exactly `0.0`, and the score is a pure dot product.
### Penalty and loss
`RegularizationType` is `L1(lambda)` or `L2(lambda)`. L2, or ridge, applies a `weights *= (1 - learning_rate * lambda)` shrink at each step. This keeps all weights small but non-zero. L1, or lasso, applies a constant, sign-based subgradient pull that drives the weights of irrelevant features toward zero. Unlike `LinearRegression` and `LogisticRegression`, `LinearSVC` applies this pull as a plain subgradient step, not a proximal (soft-thresholding) step. So a weight rarely lands on exactly `0.0`, though it still lands close to it.
If a feature carries no signal, a strong L1 penalty shrinks its weight closer to zero than it shrinks an informative feature's weight. This helps when you suspect that many columns are noise. `lambda = 0.0` is legal. It disables the penalty entirely.
2 builder methods extend the estimator past what `new` sets. `with_loss` switches between `Loss::Hinge`, the default, `max(0, 1 - y*f(x))`, and `Loss::SquaredHinge`, `max(0, 1 - y*f(x))^2`. `Loss::SquaredHinge` penalizes margin violations quadratically and is differentiable everywhere. That smoother objective lets some datasets converge more cleanly.
`with_learning_rate_decay` turns on an inverse-scaling schedule, where the effective rate at epoch `t` is `learning_rate / (1 + decay * t)`. This lets SGD settle closer to the optimum, instead of hovering a fixed step away from it. It returns a `Result`, because the decay must be non-negative and finite.
```rust
use rustyml::machine_learning::{LinearSVC, Loss, RegularizationType};
use ndarray::array;
fn main() {
let x = array![
[-5.0, 0.0], [-6.0, 0.0], [-7.0, 0.0], [-4.0, 0.0],
[5.0, 0.0], [6.0, 0.0], [7.0, 0.0], [4.0, 0.0],
];
let y = array![0.0, 0.0, 0.0, 0.0, 1.0, 1.0, 1.0, 1.0];
let mut model = LinearSVC::new(10_000, 0.01, RegularizationType::L1(0.5), true, 1e-6)
.unwrap()
.with_loss(Loss::SquaredHinge)
.with_learning_rate_decay(0.001) // returns Result and checks the decay
.unwrap()
.with_random_state(0);
model.fit(&x, &y).unwrap();
println!("penalty: {:?}", model.get_penalty());
println!("loss: {:?}", model.get_loss());
println!("preds: {:?}", model.predict(&x).unwrap());
}
```
The getters mirror `SVC`'s: `get_weights` (`Option<&Array1<f64>>`), `get_bias`, `get_penalty`, `get_loss`, `get_learning_rate`, `get_learning_rate_decay`, `get_tolerance`, `get_max_iterations`, `get_actual_iterations`, and `get_random_state`. The weight vector length always matches `n_features`. So `get_weights` doubles as a feature-importance readout, once your columns share a common scale. `LinearSVC` is the linear-classifier cousin of [Logistic Regression](./2.2._Logistic_Regression.md). The difference is the loss: hinge instead of log-loss. Hinge cares only about points near or across the margin, so it tends to give a boundary that ignores confidently-correct points entirely.
## 2.5.4. Reproducibility, persistence, and errors
Both estimators are non-deterministic by default. Both take a fixed seed through `with_random_state(u64)`. For `SVC`, the seed drives the SMO working-set fallback, the randomized offset used when scanning for a second alpha to optimize. The same seed reproduces the same support vectors, bias, and predictions, bit for bit.
For `LinearSVC`, the seed drives the per-epoch minibatch shuffle. That fixes the exact sequence of gradient steps, and so the final weights too. Different seeds can land on different solutions, especially for `LinearSVC` on data where the shuffle order matters. Set the seed whenever you need runs to be comparable. See [Reproducibility and Random Seeds](../Chapter-07/7.1._Reproducibility_and_Random_Seeds.md) for how this fits the crate-wide seeding story.
```rust
use rustyml::machine_learning::{Gamma, KernelType, SVC};
use ndarray::array;
fn main() {
let x = array![
[2.0, 2.0], [3.0, 2.0], [2.0, 3.0], [3.0, 3.0],
[-2.0, -2.0], [-3.0, -2.0], [-2.0, -3.0], [-3.0, -3.0],
];
let y = array![1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0];
let train = || {
let mut svc = SVC::new(KernelType::RBF { gamma: Gamma::Value(0.5) }, 5.0, 1e-3, 1000)
.unwrap()
.with_random_state(42);
svc.fit(&x, &y).unwrap();
svc.predict(&x).unwrap()
};
// Same seed, same data -> identical predictions.
assert_eq!(train(), train());
println!("same seed reproduces the model exactly");
}
```
Both models implement `save_to_path` and `load_from_path`. These methods serialize the entire fitted state to a compact postcard binary blob. That state is the support vectors and alphas for `SVC`, or the weights and bias for `LinearSVC`, plus every hyperparameter.
You can choose any file extension. The format stays binary regardless of the name. A round-trip reproduces predictions and decision scores exactly. This is the shallow end of [Model Persistence in Depth](../Chapter-07/7.2._Model_Persistence_in_Depth.md).
```rust
use rustyml::machine_learning::{Gamma, KernelType, SVC};
use ndarray::array;
use std::fs::remove_file;
fn main() {
let x = array![
[2.0, 2.0], [3.0, 2.0], [2.0, 3.0], [3.0, 3.0],
[-2.0, -2.0], [-3.0, -2.0], [-2.0, -3.0], [-3.0, -3.0],
];
let y = array![1.0, 1.0, 1.0, 1.0, 0.0, 0.0, 0.0, 0.0];
let mut svc = SVC::new(KernelType::RBF { gamma: Gamma::Value(0.5) }, 5.0, 1e-3, 1000)
.unwrap()
.with_random_state(42);
svc.fit(&x, &y).unwrap();
svc.save_to_path("svc_model.bin").unwrap(); // postcard binary
let loaded = SVC::load_from_path("svc_model.bin").unwrap();
assert_eq!(svc.predict(&x).unwrap(), loaded.predict(&x).unwrap());
println!("loaded model reproduces the original's predictions");
remove_file("svc_model.bin").unwrap();
}
```
Every fallible entry point returns the crate's `Error` (see [Error Handling](../Chapter-01/1.6._Error_Handling.md)). The variants you will meet in practice:
| When | Error | Trigger |
|---|---|---|
| `new` | `Error::InvalidParameter` | non-positive `C`/`learning_rate`/`tol`, zero `max_iter`, negative `lambda`, non-finite anything |
| `fit` | `Error::InvalidInput` | a label outside `{0.0, 1.0}` (same rule for both models), or `Gamma::Scale` on zero-variance data |
| `fit` | `Error::EmptyInput` / `Error::DimensionMismatch` | empty matrix, or `y.len()` != `x.nrows()` |
| `fit` | `Error::NotConverged` | `SVC` found no support vectors, for example with single-class data |
| `fit` | `Error::NonFinite` | kernel matrix or weights went non-finite during training |
| `predict` / `decision_function` | `Error::NotFitted` | called before `fit` |
| `predict` / `decision_function` | `Error::DimensionMismatch` | input feature count != training feature count |
| `predict` / `decision_function` | `Error::NonFinite` | a decision value overflowed, for example with a high-degree `Poly` kernel |
| `load_from_path` | `Error::Io` | missing file or corrupt/incompatible bytes |
The label domain still trips people, just not in the direction the SMO literature suggests. Both models want `0.0`/`1.0`, so a `y` copied from a textbook derivation or from another library's signed convention causes an `InvalidInput` at `fit`. Neither model converts it silently. Encode your labels into `{0.0, 1.0}` before training. After that, the only `+1`/`-1` values you will meet are whatever `get_support_vector_labels` hands back.