Module kernel_ridge_regression 

Kernel Ridge Regression Module

This module provides comprehensive kernel ridge regression implementations with various approximation methods, regularization strategies, and robust variants.

Overview

Kernel ridge regression extends linear ridge regression to nonlinear problems by using kernel methods. This module includes:

• Basic Kernel Ridge Regression: Standard implementation with multiple solvers
• Multi-Task Learning: Joint learning across multiple related regression tasks
• Robust Variants: Resistance to outliers using robust loss functions
• Kernel Approximation: Efficient large-scale methods using feature approximations

Kernel Approximation Methods

All implementations support multiple kernel approximation methods for scalability:

• Nyström Method: Landmark-based kernel approximation
• Random Fourier Features: Approximate shift-invariant kernels
• Structured Random Features: Computational improvements to RFF
• Fastfood Transform: Fast Walsh-Hadamard based approximation

Architecture

The module is organized into focused submodules:

• core_types - Shared types, enums, and utility functions
• basic_regression - Standard kernel ridge regression
• multitask_regression - Multi-task learning capabilities
• robust_regression - Robust variants for outlier resistance

Examples

Basic Usage

use sklears_kernel_approximation::kernel_ridge_regression::{
    KernelRidgeRegression, ApproximationMethod
};
use sklears_core::{Fit, Predict};
use scirs2_core::ndarray::array;

// Prepare data
let X = array![[1.0, 2.0], [2.0, 3.0], [3.0, 4.0], [4.0, 5.0]];
let y = array![1.0, 4.0, 9.0, 16.0];

// Create approximation method
let approximation = ApproximationMethod::RandomFourierFeatures {
    n_components: 100,
    gamma: 1.0,
};

// Create and fit model
let krr = KernelRidgeRegression::new(approximation)
    .alpha(0.1)
    .random_state(42);

let fitted_model = krr.fit(&X, &y)?;
let predictions = fitted_model.predict(&X)?;

Multi-Task Learning

use sklears_kernel_approximation::kernel_ridge_regression::{
    MultiTaskKernelRidgeRegression, ApproximationMethod, TaskRegularization
};
use sklears_core::{Fit, Predict};
use scirs2_core::ndarray::array;

let X = array![[1.0, 2.0], [2.0, 3.0], [3.0, 4.0]];
let Y = array![[1.0, 2.0], [4.0, 5.0], [9.0, 10.0]]; // Two related tasks

let approximation = ApproximationMethod::RandomFourierFeatures {
    n_components: 50,
    gamma: 1.0,
};

let mtkrr = MultiTaskKernelRidgeRegression::new(approximation)
    .alpha(0.1)
    .task_regularization(TaskRegularization::L2 { beta: 0.01 });

let fitted_model = mtkrr.fit(&X, &Y)?;
let predictions = fitted_model.predict(&X)?;

Robust Regression

use sklears_kernel_approximation::kernel_ridge_regression::{
    RobustKernelRidgeRegression, ApproximationMethod, RobustLoss
};
use sklears_core::{Fit, Predict};
use scirs2_core::ndarray::array;

let X = array![[1.0, 2.0], [2.0, 3.0], [3.0, 4.0], [10.0, 10.0]]; // Last point is outlier
let y = array![1.0, 2.0, 3.0, 100.0]; // Last target is outlier

let approximation = ApproximationMethod::RandomFourierFeatures {
    n_components: 50,
    gamma: 1.0,
};

let robust_krr = RobustKernelRidgeRegression::new(approximation)
    .alpha(0.1)
    .robust_loss(RobustLoss::Huber { delta: 1.0 });

let fitted_model = robust_krr.fit(&X, &y)?;
let predictions = fitted_model.predict(&X)?;

Re-exports

pub use crate::Untrained;

pub use core_types::ApproximationMethod;

pub use core_types::FastfoodTransform;

pub use core_types::FeatureTransformer;

pub use core_types::Kernel;

pub use core_types::Nystroem;

pub use core_types::RBFSampler;

pub use core_types::SamplingStrategy;

pub use core_types::Solver;

pub use core_types::StructuredRandomFeatures;

pub use basic_regression::IntoUntrained;

pub use basic_regression::KernelRidgeRegression;

pub use basic_regression::OnlineKernelRidgeRegression;

pub use multitask_regression::MultiTaskKernelRidgeRegression;

pub use multitask_regression::TaskRegularization;

pub use robust_regression::RobustKernelRidgeRegression;

pub use robust_regression::RobustLoss;

Modules

basic_regression

Basic Kernel Ridge Regression Implementation

core_types

Core Types for Kernel Ridge Regression

multitask_regression

Multi-Task Kernel Ridge Regression Implementation

prelude

Prelude module for convenient imports

robust_regression

Robust Kernel Ridge Regression Implementation

utils

Utility functions for kernel ridge regression

Structs

Trained

Marker trait for trained models

Type Aliases

Float

Default floating point type for the library