torsh-sparse

Sparse tensor operations for ToRSh, leveraging scirs2-sparse for efficient sparse matrix computations.

Status: Stable for CPU sparse formats, operations, neural-network layers, and linear algebra; GPU/cuSPARSE acceleration is a documented placeholder, not yet functional (see "GPU Acceleration" below). Tests: 259/259 passing (cargo nextest run --all-features).

Overview

This crate provides comprehensive sparse tensor support:

• Sparse Formats: COO, CSR, CSC, BSR, DIA, DSR, ELL, RLE, and Symmetric formats
• Operations: Sparse matrix multiplication, addition, elementwise ops, transpose, softmax
• Conversions: Dense to sparse, format conversions
• GPU Scaffolding: CudaSparseTensor/CudaSparseOps types exist but are currently placeholders (see "GPU Acceleration" below) — there is no working cuSPARSE backend yet
• Integration: Seamless integration with scirs2-sparse

All of the free functions below are called directly on the crate root / torsh_sparse::ops / torsh_sparse::linalg paths shown — there is no sparse:: namespace in this crate.

Usage

Creating Sparse Tensors

use torsh_sparse::{CooTensor, CsrTensor, sparse_from_dense, SparseFormat};
use torsh_core::Shape;
use torsh_tensor::prelude::*;

// From COO format (row indices, column indices, values, shape)
let row_indices = vec![0, 1, 1];
let col_indices = vec![2, 0, 2];
let values = vec![3.0f32, 4.0, 5.0];
let sparse_coo = CooTensor::new(row_indices, col_indices, values, Shape::new(vec![3, 4]))?;

// Or from (row, col, value) triplets
let coo2 = CooTensor::from_triplets(vec![(0, 0, 1.0f32), (1, 1, 2.0)], (2, 2))?;

// From a dense tensor (any target format, values below `threshold` are dropped)
let dense = tensor![[1.0, 0.0, 2.0],
                     [0.0, 0.0, 3.0],
                     [4.0, 5.0, 0.0]];
let sparse = sparse_from_dense(&dense, SparseFormat::Coo, Some(0.0))?;

// From CSR format (row pointers, column indices, values, shape)
let row_ptr = vec![0, 2, 3, 5]; // row pointers, len = rows + 1
let col_idx = vec![0, 2, 2, 0, 1];
let csr_values = vec![1.0f32, 2.0, 3.0, 4.0, 5.0];
let sparse_csr = CsrTensor::new(row_ptr, col_idx, csr_values, Shape::new(vec![3, 3]))?;

Sparse Operations

use torsh_sparse::ops;

// Sparse-sparse matrix multiplication
let c = ops::sparse_matmul(&a, &b)?;

// Sparse-dense multiplication (SpMM): sparse @ dense -> dense
let dense_vector = randn::<f32>(&[1000, 1])?;
let result = ops::spmm(&sparse_matrix, &dense_vector)?;

// Element-wise operations
let sum = ops::element_add(&sparse_a, &sparse_b)?;
let hadamard_product = ops::sphadamard(&sparse_a, &sparse_b)?;

// Transpose
let transposed = ops::transpose(&sparse_matrix)?;
// ...or, equivalently, the inherent method:
let transposed = sparse_matrix.transpose()?;

Format Conversions

// Convert between formats (methods on the SparseTensor trait)
let csr = sparse_coo.to_csr()?;
let csc = sparse_coo.to_csc()?;

// Convert to dense
let dense = sparse_tensor.to_dense()?;

Advanced Sparse Operations

// Sparse linear algebra
use torsh_sparse::linalg::{conjugate_gradient, gmres, bicgstab, SparseCholesky};

// Incomplete Cholesky decomposition (for symmetric positive-definite matrices)
let cholesky = SparseCholesky::new(&symmetric_sparse, /* fill_factor */ 1.5)?;

// Iterative solvers: Ax = b
// conjugate_gradient(matrix, b, x0, tol, max_iter) -> (x, iterations, residual)
let (x, iters, residual) = conjugate_gradient(&sparse_a, &b, None, 1e-6, 1000)?;
let (x, iters, residual) = gmres(&sparse_a, &b, None, 1e-6, 1000, 50)?;
let (x, iters, residual) = bicgstab(&sparse_a, &b, None, 1e-6, 1000)?;

Sparse Neural Network Layers

use torsh_sparse::nn::{SparseLinear, SparseEmbedding, GraphConvolution};

// Sparse Linear layer: new(in_features, out_features, sparsity, use_bias)
let sparse_linear = SparseLinear::new(1000, 100, 0.9, true)?; // 90% sparse

// Sparse Embedding
let sparse_embedding = SparseEmbedding::new(10_000, 300, 0.9)?;

// Graph Convolution (for GNNs): new(in_features, out_features, use_bias, add_self_loops, normalize)
let gcn = GraphConvolution::new(64, 32, true, true, true)?;

GPU Acceleration

torsh_sparse::gpu defines the shape of a future cuSPARSE-backed API (CudaSparseTensor, CudaSparseOps, CudaSparseTensorFactory), but as of this release it is a placeholder: CudaSparseTensor::is_cuda_available() always returns false, and operations like spmm, spgemm, dense↔sparse GPU conversion, and batched ops all return "not yet implemented" errors regardless of the cuda feature flag. Real cuSPARSE bindings are not currently wired up (the cudarc dependency behind the optional cuda feature is not yet used by this module). Treat this as a documented placeholder, not a working GPU backend, until this note is updated.

Sparse Gradients

// Sparse optimizer for sparse gradients (new(lr, beta1, beta2, eps, weight_decay, amsgrad))
use torsh_sparse::optimizers::SparseAdam;

let sparse_adam = SparseAdam::new(0.001, 0.9, 0.999, 1e-8, 0.0, false);

// Gradient accumulation for sparse tensors, keyed by tensor id
use torsh_sparse::autograd::SparseGradientAccumulator;

let mut grad_accumulator = SparseGradientAccumulator::new();
grad_accumulator.accumulate(tensor_id, sparse_gradient)?;

Utilities

use torsh_sparse::{analyze_sparse_tensor, compare_format_performance};

// Analyze sparsity
let stats = analyze_sparse_tensor(&tensor)?;
println!("Sparsity: {:.2}%", stats.sparsity * 100.0);
println!("NNZ: {}", stats.nnz);
println!("Detected pattern: {:?}", stats.pattern);
println!("Recommended format: {:?}", stats.recommended_format);

// Benchmark and compare all sparse formats for a given tensor
let comparison = compare_format_performance(&sparse_tensor, /* include_operations */ true)?;

Integration with SciRS2

This crate fully leverages scirs2-sparse for:

• Optimized sparse BLAS operations
• Efficient sparse matrix formats
• Hardware-accelerated sparse computations
• Advanced sparse linear algebra

Performance Tips

1. Choose the right format for your access pattern
2. Use CSR for row-wise operations, CSC for column-wise
3. Consider hybrid formats for mixed access patterns
4. Use batched operations when possible
5. Profile different sparse formats for your use case (see compare_format_performance above)

License

Licensed under the Apache License, Version 2.0. See LICENSE for details.