Crate scirs2_linalg

Expand description

§SciRS2 Linear Algebra - High-Performance Matrix Operations

scirs2-linalg provides comprehensive linear algebra operations with SciPy/NumPy-compatible APIs, leveraging native BLAS/LAPACK for peak performance and offering advanced features like SIMD acceleration, GPU support, and specialized solvers.

§🎯 Key Features

SciPy/NumPy Compatibility: Drop-in replacement for scipy.linalg and numpy.linalg
Native BLAS/LAPACK: Hardware-optimized through OpenBLAS, Intel MKL, or Apple Accelerate
SIMD Acceleration: AVX/AVX2/AVX-512 optimized operations for f32/f64
GPU Support: CUDA, ROCm, OpenCL, and Metal acceleration
Parallel Processing: Multi-threaded via Rayon for large matrices
Comprehensive Solvers: Direct, iterative, sparse, and specialized methods
Matrix Functions: Exponential, logarithm, square root, and trigonometric functions
Attention Mechanisms: Multi-head, flash, and sparse attention for transformer models

§📦 Module Overview

SciRS2 Module	SciPy/NumPy Equivalent	Description
Basic ops	`scipy.linalg.det`, `inv`	Determinants, inverses, traces
Decompositions	`scipy.linalg.lu`, `qr`, `svd`	LU, QR, SVD, Cholesky, Schur
Eigenvalues	`scipy.linalg.eig`, `eigh`	Standard and generalized eigenproblems
Solvers	`scipy.linalg.solve`, `lstsq`	Linear systems (direct & iterative)
Matrix functions	`scipy.linalg.expm`, `logm`	Matrix exponential, logarithm, etc.
Norms	`numpy.linalg.norm`, `cond`	Vector/matrix norms, condition numbers
Specialized	`scipy.linalg.solve_banded`	Banded, circulant, Toeplitz matrices
Attention	-	Multi-head, flash attention (PyTorch-style)
BLAS	`scipy.linalg.blas.*`	Low-level BLAS operations
LAPACK	`scipy.linalg.lapack.*`	Low-level LAPACK operations

§🚀 Quick Start

Add to your Cargo.toml:

[dependencies]
scirs2-linalg = "0.1.0-rc.1"
# Optional features
scirs2-linalg = { version = "0.1.0-rc.1", features = ["simd", "parallel", "gpu"] }

§Basic Matrix Operations

use scirs2_core::ndarray::array;
use scirs2_linalg::{det, inv, solve};

// Determinant and inverse
let a = array![[4.0, 2.0], [2.0, 3.0]];
let det_a = det(&a.view(), None).unwrap();
let a_inv = inv(&a.view(), None).unwrap();

// Solve linear system Ax = b
let b = array![6.0, 7.0];
let x = solve(&a.view(), &b.view(), None).unwrap();

§Matrix Decompositions

use scirs2_core::ndarray::array;
use scirs2_linalg::{lu, qr, svd, cholesky, eig};

let a = array![[1.0_f64, 2.0], [3.0, 4.0]];

// LU decomposition: PA = LU
let (p, l, u) = lu(&a.view(), None).unwrap();

// QR decomposition: A = QR
let (q, r) = qr(&a.view(), None).unwrap();

// SVD: A = UΣVᵀ
let (u, s, vt) = svd(&a.view(), true, None).unwrap();

// Eigenvalues and eigenvectors
let (eigenvalues, eigenvectors) = eig(&a.view(), None).unwrap();

// Cholesky decomposition for positive definite matrices
let spd = array![[4.0, 2.0], [2.0, 3.0]];
let l_chol = cholesky(&spd.view(), None).unwrap();

§Iterative Solvers (Large Sparse Systems)

use scirs2_core::ndarray::array;
use scirs2_linalg::{conjugate_gradient, gmres};

// Conjugate Gradient for symmetric positive definite systems
let a = array![[4.0_f64, 1.0], [1.0, 3.0]];
let b = array![1.0_f64, 2.0];
let x_cg = conjugate_gradient(&a.view(), &b.view(), 10, 1e-10, None).unwrap();

// GMRES for general systems
let x_gmres = gmres(&a.view(), &b.view(), 10, 1e-10, None).unwrap();

§Matrix Functions

use scirs2_core::ndarray::array;
use scirs2_linalg::{expm, logm, sqrtm, sinm, cosm};

let a = array![[1.0, 0.5], [0.5, 1.0]];

// Matrix exponential: exp(A)
let exp_a = expm(&a.view(), None).unwrap();

// Matrix logarithm: log(A)
let log_a = logm(&a.view(), None).unwrap();

// Matrix square root: √A
let sqrt_a = sqrtm(&a.view(), None).unwrap();

§Accelerated BLAS/LAPACK Operations

use scirs2_core::ndarray::array;
use scirs2_linalg::accelerated::{matmul, solve as fast_solve};

// Hardware-accelerated matrix multiplication
let a = array![[1.0_f64, 2.0], [3.0, 4.0]];
let b = array![[5.0_f64, 6.0], [7.0, 8.0]];
let c = matmul(&a.view(), &b.view()).unwrap();

// Fast linear system solver using LAPACK
let x = fast_solve(&a.view(), &b.view()).unwrap();

§Attention Mechanisms (Deep Learning)

use scirs2_core::ndarray::Array2;
use scirs2_linalg::attention::{multi_head_attention, flash_attention, AttentionConfig};

// Multi-head attention (Transformer-style)
let query = Array2::<f32>::zeros((32, 64));  // (batch_size, d_model)
let key = Array2::<f32>::zeros((32, 64));
let value = Array2::<f32>::zeros((32, 64));

let config = AttentionConfig {
    num_heads: 8,
    dropout: 0.1,
    causal: false,
};

let output = multi_head_attention(&query.view(), &key.view(), &value.view(), &config);

§🏗️ Architecture

scirs2-linalg
├── Basic Operations (det, inv, trace, rank)
├── Decompositions (LU, QR, SVD, Cholesky, Eigenvalues)
├── Solvers
│   ├── Direct (solve, lstsq)
│   ├── Iterative (CG, GMRES, BiCGSTAB)
│   └── Specialized (banded, triangular, sparse-dense)
├── Matrix Functions (expm, logm, sqrtm, trigonometric)
├── Accelerated Backends
│   ├── BLAS/LAPACK (native libraries)
│   ├── SIMD (AVX/AVX2/AVX-512)
│   ├── Parallel (Rayon multi-threading)
│   └── GPU (CUDA/ROCm/OpenCL/Metal)
├── Advanced Features
│   ├── Attention mechanisms
│   ├── Hierarchical matrices (H-matrices)
│   ├── Kronecker factorization (K-FAC)
│   ├── Randomized algorithms
│   ├── Mixed precision
│   └── Quantization-aware operations
└── Compatibility Layer (SciPy-compatible API)

§📊 Performance

Operation	Size	Pure Rust	BLAS/LAPACK	SIMD	GPU
Matrix Multiply	1000×1000	2.5s	15ms	180ms	5ms
SVD	1000×1000	8.2s	120ms	N/A	35ms
Eigenvalues	1000×1000	6.8s	95ms	N/A	28ms
Solve (direct)	1000×1000	1.8s	22ms	140ms	8ms

Note: Benchmarks on AMD Ryzen 9 5950X with NVIDIA RTX 3090. BLAS/LAPACK uses OpenBLAS.

§🔗 Integration

Works seamlessly with other SciRS2 crates:

scirs2-stats: Covariance matrices, statistical distributions
scirs2-optimize: Hessian computations, constraint Jacobians
scirs2-neural: Weight matrices, gradient computations
scirs2-sparse: Sparse matrix operations

§🔒 Version Information

Version: 0.1.0-rc.1
Release Date: October 03, 2025
MSRV (Minimum Supported Rust Version): 1.70.0
Documentation: docs.rs/scirs2-linalg
Repository: github.com/cool-japan/scirs

Re-exports§

pub use error::LinalgError;
pub use error::LinalgResult;
pub use self::eigen::advanced_precision_eig;
pub use self::eigen::eig;
pub use self::eigen::eig_gen;
pub use self::eigen::eigh;
pub use self::eigen::eigh_gen;
pub use self::eigen::eigvals;
pub use self::eigen::eigvals_gen;
pub use self::eigen::eigvalsh;
pub use self::eigen::eigvalsh_gen;
pub use self::eigen::power_iteration;
pub use self::quantization::calibration::calibrate_matrix;
pub use self::quantization::calibration::calibrate_vector;
pub use self::quantization::calibration::get_activation_calibration_config;
pub use self::quantization::calibration::get_weight_calibration_config;
pub use self::quantization::calibration::CalibrationConfig;
pub use self::quantization::calibration::CalibrationMethod;
pub use self::eigen_specialized::banded_eigen;
pub use self::eigen_specialized::banded_eigh;
pub use self::eigen_specialized::banded_eigvalsh;
pub use self::eigen_specialized::circulant_eigenvalues;
pub use self::eigen_specialized::largest_k_eigh;
pub use self::eigen_specialized::partial_eigen;
pub use self::eigen_specialized::smallest_k_eigh;
pub use self::eigen_specialized::tridiagonal_eigen;
pub use self::eigen_specialized::tridiagonal_eigh;
pub use self::eigen_specialized::tridiagonal_eigvalsh;
pub use self::complex::enhanced_ops::det as complex_det;
pub use self::complex::enhanced_ops::frobenius_norm;
pub use self::complex::enhanced_ops::hermitian_part;
pub use self::complex::enhanced_ops::inner_product;
pub use self::complex::enhanced_ops::is_hermitian;
pub use self::complex::enhanced_ops::is_unitary;
pub use self::complex::enhanced_ops::matrix_exp;
pub use self::complex::enhanced_ops::matvec;
pub use self::complex::enhanced_ops::polar_decomposition;
pub use self::complex::enhanced_ops::power_method;
pub use self::complex::enhanced_ops::rank as complex_rank;
pub use self::complex::enhanced_ops::schur as complex_schur;
pub use self::complex::enhanced_ops::skew_hermitian_part;
pub use self::complex::enhanced_ops::trace;
pub use self::complex::complex_inverse;
pub use self::complex::complex_matmul;
pub use self::complex::hermitian_transpose;
pub use self::decomposition_advanced::jacobi_svd;
pub use self::decomposition_advanced::polar_decomposition as advanced_polar_decomposition;
pub use self::decomposition_advanced::polar_decomposition_newton;
pub use self::decomposition_advanced::qr_with_column_pivoting;
pub use self::matrix_equations::solve_continuous_riccati;
pub use self::matrix_equations::solve_discrete_riccati;
pub use self::matrix_equations::solve_generalized_sylvester;
pub use self::matrix_equations::solve_stein;
pub use self::matrix_equations::solve_sylvester;
pub use self::matrix_factorization::cur_decomposition;
pub use self::matrix_factorization::interpolative_decomposition;
pub use self::matrix_factorization::nmf;
pub use self::matrix_factorization::rank_revealing_qr;
pub use self::matrix_factorization::utv_decomposition;
pub use self::matrix_functions::acosm;
pub use self::matrix_functions::asinm;
pub use self::matrix_functions::atanm;
pub use self::matrix_functions::coshm;
pub use self::matrix_functions::cosm;
pub use self::matrix_functions::expm;
pub use self::matrix_functions::geometric_mean_spd;
pub use self::matrix_functions::logm;
pub use self::matrix_functions::logm_parallel;
pub use self::matrix_functions::nuclear_norm;
pub use self::matrix_functions::signm;
pub use self::matrix_functions::sinhm;
pub use self::matrix_functions::sinm;
pub use self::matrix_functions::spectral_condition_number;
pub use self::matrix_functions::spectral_radius;
pub use self::matrix_functions::sqrtm;
pub use self::matrix_functions::sqrtm_parallel;
pub use self::matrix_functions::tanhm;
pub use self::matrix_functions::tanm;
pub use self::matrix_functions::tikhonov_regularization;
pub use self::matrixfree::block_diagonal_operator;
pub use self::matrixfree::conjugate_gradient as matrix_free_conjugate_gradient;
pub use self::matrixfree::diagonal_operator;
pub use self::matrixfree::gmres as matrix_free_gmres;
pub use self::matrixfree::jacobi_preconditioner;
pub use self::matrixfree::preconditioned_conjugate_gradient as matrix_free_preconditioned_conjugate_gradient;
pub use self::matrixfree::LinearOperator;
pub use self::matrixfree::MatrixFreeOp;
pub use self::solvers::iterative::bicgstab;
pub use self::solvers::iterative::conjugate_gradient as cg_solver;
pub use self::solvers::iterative::gmres;
pub use self::solvers::iterative::preconditioned_conjugate_gradient as pcg_solver;
pub use self::solvers::iterative::IterativeSolverOptions;
pub use self::solvers::iterative::IterativeSolverResult;
pub use self::specialized::specialized_to_operator;
pub use self::specialized::BandedMatrix;
pub use self::specialized::SpecializedMatrix;
pub use self::specialized::SymmetricMatrix;
pub use self::specialized::TridiagonalMatrix;
pub use self::structured::structured_to_operator;
pub use self::structured::CirculantMatrix;
pub use self::structured::HankelMatrix;
pub use self::structured::StructuredMatrix;
pub use self::structured::ToeplitzMatrix;
pub use self::extended_precision::*;
pub use self::stats::*;

Modules§

accelerated: Accelerated linear algebra operations using native BLAS/LAPACK
attention: Optimized attention mechanisms for transformer models
batch: Batch matrix operations for AI/ML workloads
blas: BLAS (Basic Linear Algebra Subprograms) interface
blas_accelerated: Accelerated BLAS (Basic Linear Algebra Subprograms) operations using ndarray-linalg
broadcast: NumPy-style broadcasting for linear algebra operations on higher-dimensional arrays
circulant_toeplitz: FFT-based solvers for circulant and Toeplitz matrices
compat: SciPy-compatible API wrappers for scirs2-linalg
compat_wrappers: Backward compatibility wrappers for functions that gained workers parameters
complex: Complex matrix operations
convolution: Specialized operations for convolutional neural networks
decomposition_advanced: Advanced matrix decomposition algorithms
eigen: Eigenvalue and eigenvector computations
eigen_specialized: Specialized eigenvalue solvers for structured matrices
error: Error types for the SciRS2 linear algebra module
extended_precision: Extended precision matrix operations
fft: Fast Fourier Transform (FFT) and spectral methods for signal processing
generic: Type-generic linear algebra operations
gradient: Gradient calculation utilities for neural networks
hierarchical: Hierarchical matrix factorizations for large-scale computational efficiency
kronecker: Kronecker product optimizations for neural network layers
lapack: LAPACK (Linear Algebra Package) interface
lapack_accelerated: LAPACK (Linear Algebra Package) operations
large_scale: Large-scale linear algebra algorithms
lowrank: Low-rank matrix operations and approximations
matrix_calculus: Matrix calculus utilities
matrix_dynamics: Matrix Differential Equations and Dynamical Systems
matrix_equations: Advanced matrix equation solvers
matrix_factorization: Advanced matrix factorization algorithms
matrix_functions: Matrix functions such as matrix exponential, logarithm, and square root
matrixfree: Matrix-free operations for iterative solvers in large models
mixed_precision: Mixed-precision linear algebra operations
optim: Optimized matrix operations for large parameter matrices
parallel: Parallel processing utilities for linear algebra operations
parallel_dispatch: Parallel algorithm dispatch for linear algebra operations
perf_opt: Performance optimizations for large matrices
preconditioners: Advanced preconditioners for iterative linear solvers
prelude: Common linear algebra operations for convenient importing
projection: Fast random projection methods for dimensionality reduction
quantization: Quantization-aware linear algebra operations Quantization-aware linear algebra operations
random: Random matrix generation utilities
random_matrices: Random matrix generation utilities
scalable: Scalable algorithms for tall-and-skinny or short-and-fat matrices
simd_ops: SIMD accelerated linear algebra operations
solvers: Linear system solvers
sparse_dense: Sparse-Dense matrix operations
special: Special matrix functions
specialized: Specialized matrix implementations with optimized storage and operations
stats: Statistical functions for matrices
structured: Structured matrices support (Toeplitz, circulant) for efficient representations
tensor_train: Tensor-Train (TT) decomposition for high-dimensional tensor computations

Structs§

LstsqResult: Solution to a least-squares problem

Functions§

basic_trace: Compute the trace of a square matrix.
bicgstab: Solve a linear system Ax = b using the BiCGSTAB (Biconjugate Gradient Stabilized) method.
cholesky: Compute the Cholesky decomposition of a matrix.
cholesky_default: Compute Cholesky decomposition using default thread count
cond: Compute the condition number of a matrix.
cond_defaultDeprecated: Compute the condition number without workers parameter (deprecated - use cond with workers)
conjugate_gradient: Solve a linear system Ax = b using the Conjugate Gradient method.
conjugate_gradient_default: Solve a linear system Ax = b using the Conjugate Gradient method (backward compatibility wrapper).
det: Compute the determinant of a square matrix.
det_default: Compute the determinant of a square matrix (backward compatibility wrapper).
gauss_seidel: Solve a linear system Ax = b using Gauss-Seidel iteration.
geometric_multigrid: Geometric Multigrid method for solving linear systems.
inv: Compute the inverse of a square matrix.
inv_default: Compute the inverse of a square matrix (backward compatibility wrapper).
jacobi_method: Solve a linear system Ax = b using Jacobi iteration.
lstsq: Compute least-squares solution to a linear matrix equation.
lstsq_default: Compute least-squares solution using default thread count
lu: Compute the LU decomposition of a matrix.
lu_default: Compute LU decomposition using default thread count
matrix_norm: Compute a matrix norm.
matrix_norm_defaultDeprecated: Compute a matrix norm without workers parameter (deprecated - use matrix_norm with workers)
matrix_power: Raise a square matrix to the given power.
matrix_power_default: Raise a square matrix to the given power (backward compatibility wrapper).
matrix_rank: Compute the rank of a matrix.
matrix_rank_defaultDeprecated: Compute matrix rank without workers parameter (deprecated - use matrix_rank with workers)
minres: Solve a linear system Ax = b using the MINRES (Minimal Residual) method.
qr: Compute the QR decomposition of a matrix.
qr_default: Compute QR decomposition using default thread count
schur: Compute the Schur decomposition of a matrix.
solve: Solve a linear system of equations.
solve_default: Solve linear system using default thread count
solve_multiple: Solve the linear system Ax = B for x with multiple right-hand sides.
solve_multiple_default: Solve multiple linear systems using default thread count
solve_triangular: Solve a linear system with a lower or upper triangular coefficient matrix.
successive_over_relaxation: Solve a linear system Ax = b using Successive Over-Relaxation (SOR).
svd: Compute the singular value decomposition of a matrix.
svd_default: Compute SVD using default thread count
vector_norm: Compute a vector norm.
vector_norm_parallel: Compute the norm of a vector with parallel processing support.

Crate scirs2_linalg

Crate scirs2_linalg Copy item path

§SciRS2 Linear Algebra - High-Performance Matrix Operations

§🎯 Key Features

§📦 Module Overview

§🚀 Quick Start

§Basic Matrix Operations

§Matrix Decompositions

§Iterative Solvers (Large Sparse Systems)

§Matrix Functions

§Accelerated BLAS/LAPACK Operations

§Attention Mechanisms (Deep Learning)

§🏗️ Architecture

§📊 Performance

§🔗 Integration

§🔒 Version Information

Re-exports§

Modules§

Structs§

Functions§

Crate scirs2_linalg