lowess implementation in Rust

High-performance LOWESS (Locally Weighted Scatterplot Smoothing) for Rust — Production-ready implementation with robust statistics, confidence intervals, and comprehensive features.

Why This Crate?

⚡ Blazingly Fast: 4-29× faster performance than Python's statsmodels
🎯 Production-Ready: Comprehensive error handling, numerical stability, extensive testing
📊 Feature-Rich: Confidence/prediction intervals, multiple kernels, cross-validation
🚀 Scalable: Streaming mode and delta optimization
🔬 Scientific: Validated against R and Python implementations
🛠️ Flexible: no_std support, multiple robustness methods

NOTE

From release 0.4.0 onwards, the parallelization feature and ndarray support are dropped, in order to make this crate a dependency-free, core LOWESS implementation, ensuring maximum compatibility with other crates or high-level APIs on top of this crate (e.g. fastLowess, Python bindings, R bindings, polars plugins, etc.).

👉👉👉 If you need parallelization or ndarray support, please use the fastLowess crate at GitHub or Crates.io. 👈👈👈

Quick Start

use lowess::prelude::*;

let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];

// Basic smoothing
let result = Lowess::new()
    .fraction(0.5)
    .adapter(Batch)
    .build()?
    .fit(&x, &y)?;

println!("Smoothed: {:?}", result.y);
# Ok::<(), LowessError>(())

Installation

[dependencies]
lowess = "0.4"

# For no_std environments (requires alloc)
lowess = { version = "0.4", default-features = false }

Features at a Glance

Feature	Description	Use Case
Robust Smoothing	IRLS with Bisquare/Huber/Talwar weights	Outlier-contaminated data
Confidence Intervals	Point-wise standard errors & bounds	Uncertainty quantification
Cross-Validation	Auto-select optimal fraction	Unknown smoothing parameter
Multiple Kernels	Tricube, Epanechnikov, Gaussian, etc.	Different smoothness profiles
Streaming Mode	Constant memory usage	Very large datasets
Delta Optimization	Skip dense regions	10× speedup on dense data

Common Use Cases

1. Robust Smoothing (Handle Outliers)

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
    .fraction(0.3)
    .iterations(5)                // Robust iterations
    .return_robustness_weights()  // Return outlier weights
    .adapter(Batch)
    .build()?
    .fit(&x, &y)?;

// Check which points were downweighted
if let Some(weights) = &result.robustness_weights {
    for (i, &w) in weights.iter().enumerate() {
        if w < 0.1 {
            println!("Point {} is likely an outlier", i);
        }
    }
}
# Ok::<(), LowessError>(())

2. Uncertainty Quantification

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
    .fraction(0.5)
    .weight_function(WeightFunction::Tricube)
    .confidence_intervals(0.95)
    .prediction_intervals(0.95)
    .adapter(Batch)
    .build()?
    .fit(&x, &y)?;

// Plot confidence bands
for i in 0..x.len() {
    println!("x={:.1}: y={:.2} CI=[{:.2}, {:.2}]",
        result.x[i],
        result.y[i],
        result.confidence_lower.as_ref().unwrap()[i],
        result.confidence_upper.as_ref().unwrap()[i]
    );
}
# Ok::<(), LowessError>(())

3. Automatic Parameter Selection

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8, 12.0, 14.1, 16.0];
// Let cross-validation find the optimal smoothing fraction
let result = Lowess::new()
    .cross_validate(&[0.2, 0.3, 0.5, 0.7], CrossValidationStrategy::KFold, Some(5))
    .adapter(Batch)
    .build()?
    .fit(&x, &y)?;

println!("Optimal fraction: {}", result.fraction_used);
println!("CV RMSE scores: {:?}", result.cv_scores);
# Ok::<(), LowessError>(())

4. Large Dataset Optimization

use lowess::prelude::*;

# let large_x: Vec<f64> = (0..5000).map(|i| i as f64).collect();
# let large_y: Vec<f64> = large_x.iter().map(|&x| x.sin()).collect();
// Enable all performance optimizations
let result = Lowess::new()
    .fraction(0.3)
    .delta(0.01)        // Skip dense regions
    .adapter(Batch)
    .build()?
    .fit(&large_x, &large_y)?;
# Ok::<(), LowessError>(())

5. Production Monitoring

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
    .fraction(0.5)
    .iterations(3)
    .return_diagnostics()
    .adapter(Batch)
    .build()?
    .fit(&x, &y)?;

if let Some(diag) = &result.diagnostics {
    println!("RMSE: {:.4}", diag.rmse);
    println!("R²: {:.4}", diag.r_squared);
    println!("Effective DF: {:.2}", diag.effective_df);

    // Quality checks
    if diag.effective_df < 2.0 {
        eprintln!("Warning: Very low degrees of freedom");
    }
}
# Ok::<(), LowessError>(())

6. Convenience Constructors

Pre-configured builders for common scenarios:

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
// For noisy data with outliers
let result = Lowess::<f64>::robust().adapter(Batch).build()?.fit(&x, &y)?;

// For speed on clean data
let result = Lowess::<f64>::quick().adapter(Batch).build()?.fit(&x, &y)?;
# Ok::<(), LowessError>(())

API Overview

Builder Methods

use lowess::prelude::*;

Lowess::new()
    // Core parameters
    .fraction(0.5)                  // Smoothing span (0, 1], default: 0.67
    .iterations(3)                  // Robustness iterations, default: 3
    .delta(0.01)                    // Interpolation threshold
    
    // Kernel selection
    .weight_function(WeightFunction::Tricube)  // Default
    
    // Robustness method
    .robustness_method(RobustnessMethod::Bisquare)  // Default
    
    // Intervals & diagnostics
    .confidence_intervals(0.95)
    .prediction_intervals(0.95)
    .return_diagnostics()
    .return_residuals()
    .return_robustness_weights()
    
    // Parameter selection
    .cross_validate(&[0.3, 0.5, 0.7], CrossValidationStrategy::KFold, Some(5))
    
    // Convergence
    .auto_converge(1e-4)
    .max_iterations(20)
    
    // Execution mode
    .adapter(Batch)  // or Streaming, Online
    .build()?;       // Build the model

Result Structure

pub struct LowessResult<T> {
    pub x: Vec<T>,                          // Sorted x values
    pub y: Vec<T>,                          // Smoothed y values
    pub standard_errors: Option<Vec<T>>,    // Point-wise SE
    pub confidence_lower: Option<Vec<T>>,   // CI lower bound
    pub confidence_upper: Option<Vec<T>>,   // CI upper bound
    pub prediction_lower: Option<Vec<T>>,   // PI lower bound
    pub prediction_upper: Option<Vec<T>>,   // PI upper bound
    pub residuals: Option<Vec<T>>,          // y - fitted
    pub robustness_weights: Option<Vec<T>>, // Final IRLS weights
    pub diagnostics: Option<Diagnostics<T>>,
    pub iterations_used: Option<usize>,     // Actual iterations
    pub fraction_used: T,                   // Selected fraction
    pub cv_scores: Option<Vec<T>>,          // CV RMSE per fraction
}

Execution Modes

Choose the right execution mode based on your use case:

Batch Processing (Standard)

For complete datasets in memory with full feature support:

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let model = Lowess::new()
    .fraction(0.5)
    .confidence_intervals(0.95)
    .return_diagnostics()
    .adapter(Batch)  // All features supported
    .build()?;

let result = model.fit(&x, &y)?;
# Ok::<(), LowessError>(())

Streaming Processing

For large datasets (>100K points) that don't fit in memory:

use lowess::prelude::*;

let mut processor = Lowess::new()
    .fraction(0.3)
    .iterations(2)
    .adapter(Streaming)
    .chunk_size(1000)   // Process 1000 points at a time
    .overlap(100)       // 100 points overlap between chunks
    .build()?;

// Process data in chunks
for chunk in data_chunks {
    let result = processor.process_chunk(&chunk.x, &chunk.y)?;
    // Handle results incrementally
}

let final_result = processor.finalize()?;
# Ok::<(), LowessError>(())

Online/Incremental Processing

For real-time data streams with sliding window:

use lowess::prelude::*;

let mut processor = Lowess::new()
    .fraction(0.2)
    .iterations(1)
    .adapter(Online)
    .window_capacity(100)  // Keep last 100 points
    .build()?;

// Process points as they arrive
for (x, y) in data_stream {
    if let Some(output) = processor.add_point(x, y)? {
        println!("Smoothed: {}", output.smoothed);
    }
}
# Ok::<(), LowessError>(())

Parameter Selection Guide

Fraction (Smoothing Span)

0.1-0.3: Local, captures rapid changes (wiggly)
0.4-0.6: Balanced, general-purpose
0.7-1.0: Global, smooth trends only
Default: 0.67 (2/3, Cleveland's choice)
Use CV when uncertain

Robustness Iterations

0: Clean data, speed critical
1-2: Light contamination
3: Default, good balance (recommended)
4-5: Heavy outliers
>5: Diminishing returns

Kernel Function

Tricube (default): Best all-around, smooth, efficient
Epanechnikov: Theoretically optimal MSE
Gaussian: Very smooth, no compact support
Uniform: Fastest, least smooth (moving average)

Delta Optimization

None: Small datasets (n < 1000)
0.01 × range(x): Good starting point for dense data
Manual tuning: Adjust based on data density

Error Handling

use lowess::prelude::*;

# let x = vec![1.0, 2.0, 3.0];
# let y = vec![2.0, 4.0, 6.0];
match Lowess::new().adapter(Batch).build()?.fit(&x, &y) {
    Ok(result) => {
        println!("Success: {:?}", result.y);
    },
    Err(LowessError::EmptyInput) => {
        eprintln!("Empty input arrays");
    },
    Err(LowessError::MismatchedInputs { x_len, y_len }) => {
        eprintln!("Length mismatch: x={}, y={}", x_len, y_len);
    },
    Err(LowessError::InvalidFraction(f)) => {
        eprintln!("Invalid fraction: {} (must be in (0, 1])", f);
    },
    Err(e) => {
        eprintln!("Error: {}", e);
    }
}
# Ok::<(), LowessError>(())

Examples

Comprehensive examples are available in the examples/ directory:

batch_smoothing.rs - 8 scenarios covering batch processing
- Basic smoothing, robust outlier handling, uncertainty quantification
- Cross-validation, diagnostics, kernel comparisons, robustness methods
- Quick and robust presets
online_smoothing.rs - 6 scenarios for real-time processing
- Basic streaming, sensor data simulation, outlier handling
- Window size effects, memory-bounded processing, sliding window behavior
streaming_smoothing.rs - 6 scenarios for large datasets
- Basic chunking, chunk size comparison, overlap strategies
- Large dataset processing, outlier handling, file-based simulation

Run examples with:

cargo run --example batch_smoothing
cargo run --example online_smoothing
cargo run --example streaming_smoothing

Feature Flags

std (default): Standard library support
dev: Exposes internal modules for testing (automatically enabled in development)

Standard configuration

[dependencies]
lowess = "0.4"

No-std configuration (requires alloc)

[dependencies]
lowess = { version = "0.4", default-features = false }

Validation

This implementation has been extensively validated against:

R's stats::lowess: Numerical agreement to machine precision
Python's statsmodels: Validated on 44 test scenarios
Cleveland's original paper: Reproduces published examples

MSRV (Minimum Supported Rust Version)

Rust 1.85.0 or later (requires Rust Edition 2024).

Contributing

Contributions welcome! See CONTRIBUTING.md for:

Bug reports and feature requests
Pull request guidelines
Development workflow
Testing requirements

License

MIT License - see LICENSE file.

References

Original papers:

Cleveland, W.S. (1979). "Robust Locally Weighted Regression and Smoothing Scatterplots". Journal of the American Statistical Association, 74(368): 829-836. DOI:10.2307/2286407
Cleveland, W.S. (1981). "LOWESS: A Program for Smoothing Scatterplots by Robust Locally Weighted Regression". The American Statistician, 35(1): 54.

Related implementations:

Citation

@software{lowess_rust_2025,
  author = {Valizadeh, Amir},
  title = {lowess: High-performance LOWESS for Rust},
  year = {2025},
  url = {https://github.com/thisisamirv/lowess},
  version = {0.4.0}
}

Author

Amir Valizadeh
📧 thisisamirv@gmail.com
🔗 GitHub

Keywords: LOWESS, LOESS, local regression, nonparametric regression, smoothing, robust statistics, time series, bioinformatics, genomics, signal processing

lowess 0.4.0