Struct AdaBoost 

pub struct AdaBoost<M, L> {
    pub models: Vec<M>,
    pub model_weights: Vec<f64>,
    pub classes: Vec<L>,
}

A fitted AdaBoost ensemble classifier.

Structure

AdaBoost (Adaptive Boosting) is an ensemble learning method that combines multiple weak learners into a strong classifier. Unlike bagging methods (like Random Forest), AdaBoost trains learners sequentially, where each new learner focuses more on examples that previous learners misclassified.

Each fitted model M has an associated weight (alpha) that represents its contribution to the final prediction. Models that perform better on their training data receive higher weights.

Algorithm Overview

Given a DatasetBase denoted as D with n samples:

1. Initialize sample weights uniformly: w_i = 1/n for all samples
2. For each iteration t from 1 to T (number of estimators): a. Train base learner on weighted dataset b. Calculate weighted error rate c. Compute model weight (alpha) based on error d. Update sample weights: increase weights for misclassified samples e. Normalize sample weights

Prediction Algorithm

The final prediction is computed using weighted majority voting:

• Each model’s prediction is weighted by its alpha value
• The class with the highest weighted vote is selected

Example

use linfa::prelude::{Fit, Predict};
use linfa_ensemble::AdaBoostParams;
use linfa_trees::DecisionTree;
use ndarray_rand::rand::SeedableRng;
use rand::rngs::SmallRng;

// Load Iris dataset
let mut rng = SmallRng::seed_from_u64(42);
let (train, test) = linfa_datasets::iris()
    .shuffle(&mut rng)
    .split_with_ratio(0.8);

// Train AdaBoost with decision tree stumps
let adaboost_model = AdaBoostParams::new(DecisionTree::params().max_depth(Some(1)))
    .n_estimators(50)
    .learning_rate(1.0)
    .fit(&train)
    .unwrap();

// Make predictions on the test set
let predictions = adaboost_model.predict(&test);

References

• Freund, Y., & Schapire, R. E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences, 55(1), 119-139.
• Scikit-Learn AdaBoost Documentation
• An Introduction to Statistical Learning, Chapter 8

Fields

models: Vec<M>

The fitted base learner models

model_weights: Vec<f64>

The weight (alpha) for each model in the ensemble

classes: Vec<L>

The unique class labels seen during training

Implementations

impl<M, L> AdaBoost<M, L>

pub fn n_estimators(&self) -> usize

Returns the number of estimators in the ensemble

pub fn weights(&self) -> &[f64]

Returns the model weights (alpha values)

Trait Implementations

impl<M: Clone, L: Clone> Clone for AdaBoost<M, L>

fn clone(&self) -> AdaBoost<M, L>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<M: Debug, L: Debug> Debug for AdaBoost<M, L>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<F: Clone, T, M, L> PredictInplace<ArrayBase<OwnedRepr<F>, Dim<[usize; 2]>>, T> for AdaBoost<M, L>

where M: PredictInplace<Array2<F>, T>, <T as AsTargets>::Elem: Copy + Eq + Hash + Debug + Into<usize>, T: AsTargets + AsTargetsMut<Elem = <T as AsTargets>::Elem>, usize: Into<<T as AsTargets>::Elem>,

fn predict_inplace(&self, x: &Array2<F>, y: &mut T)

Predict something in place

fn default_target(&self, x: &Array2<F>) -> T

Create targets that predict_inplace works with.