# lowess implementation in Rust
[](https://crates.io/crates/lowess)
[](https://docs.rs/lowess)
[](LICENSE)
[](https://www.rust-lang.org)
**High-performance LOWESS (Locally Weighted Scatterplot Smoothing) for Rust** — Production-ready implementation with robust statistics, confidence intervals, and comprehensive features.
## Why This Crate?
- ⚡ **Blazingly Fast**: 4-29× faster performance than Python's statsmodels
- 🎯 **Production-Ready**: Comprehensive error handling, numerical stability, extensive testing
- 📊 **Feature-Rich**: Confidence/prediction intervals, multiple kernels, cross-validation
- 🚀 **Scalable**: Streaming mode and delta optimization
- 🔬 **Scientific**: Validated against R and Python implementations
- 🛠️ **Flexible**: `no_std` support, multiple robustness methods
## NOTE
From release 0.4.0 onwards, the parallelization feature and ndarray support are dropped, in order to make this crate a dependency-free, core LOWESS implementation, ensuring maximum compatibility with other crates or high-level APIs on top of this crate (e.g. `fastLowess`, Python bindings, R bindings, polars plugins, etc.).
👉👉👉 If you need parallelization or ndarray support, please use the `fastLowess` crate at [GitHub](https://github.com/av746/fastLowess) or [Crates.io](https://crates.io/crates/fastLowess). 👈👈👈
## Quick Start
use lowess::prelude::*;
let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
// Basic smoothing
let result = Lowess::new()
.fraction(0.5)
.adapter(Batch)
.build()?
.fit(&x, &y)?;
println!("Smoothed: {:?}", result.y);
# Ok::<(), LowessError>(())
## Installation
[dependencies]
lowess = "0.4"
# For no_std environments (requires alloc)
lowess = { version = "0.4", default-features = false }
## Features at a Glance
| **Robust Smoothing** | IRLS with Bisquare/Huber/Talwar weights | Outlier-contaminated data |
| **Confidence Intervals** | Point-wise standard errors & bounds | Uncertainty quantification |
| **Cross-Validation** | Auto-select optimal fraction | Unknown smoothing parameter |
| **Multiple Kernels** | Tricube, Epanechnikov, Gaussian, etc. | Different smoothness profiles |
| **Streaming Mode** | Constant memory usage | Very large datasets |
| **Delta Optimization** | Skip dense regions | 10× speedup on dense data |
## Common Use Cases
### 1. Robust Smoothing (Handle Outliers)
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
.fraction(0.3)
.iterations(5) // Robust iterations
.return_robustness_weights() // Return outlier weights
.adapter(Batch)
.build()?
.fit(&x, &y)?;
// Check which points were downweighted
if let Some(weights) = &result.robustness_weights {
for (i, &w) in weights.iter().enumerate() {
if w < 0.1 {
println!("Point {} is likely an outlier", i);
}
}
}
# Ok::<(), LowessError>(())
### 2. Uncertainty Quantification
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
.fraction(0.5)
.weight_function(WeightFunction::Tricube)
.confidence_intervals(0.95)
.prediction_intervals(0.95)
.adapter(Batch)
.build()?
.fit(&x, &y)?;
// Plot confidence bands
for i in 0..x.len() {
println!("x={:.1}: y={:.2} CI=[{:.2}, {:.2}]",
result.x[i],
result.y[i],
result.confidence_lower.as_ref().unwrap()[i],
result.confidence_upper.as_ref().unwrap()[i]
);
}
# Ok::<(), LowessError>(())
### 3. Automatic Parameter Selection
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8, 12.0, 14.1, 16.0];
// Let cross-validation find the optimal smoothing fraction
let result = Lowess::new()
.cross_validate(&[0.2, 0.3, 0.5, 0.7], CrossValidationStrategy::KFold, Some(5))
.adapter(Batch)
.build()?
.fit(&x, &y)?;
println!("Optimal fraction: {}", result.fraction_used);
println!("CV RMSE scores: {:?}", result.cv_scores);
# Ok::<(), LowessError>(())
### 4. Large Dataset Optimization
use lowess::prelude::*;
# let large_x: Vec<f64> = (0..5000).map(|i| i as f64).collect();
# let large_y: Vec<f64> = large_x.iter().map(|&x| x.sin()).collect();
// Enable all performance optimizations
let result = Lowess::new()
.fraction(0.3)
.delta(0.01) // Skip dense regions
.adapter(Batch)
.build()?
.fit(&large_x, &large_y)?;
# Ok::<(), LowessError>(())
### 5. Production Monitoring
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
.fraction(0.5)
.iterations(3)
.return_diagnostics()
.adapter(Batch)
.build()?
.fit(&x, &y)?;
if let Some(diag) = &result.diagnostics {
println!("RMSE: {:.4}", diag.rmse);
println!("R²: {:.4}", diag.r_squared);
println!("Effective DF: {:.2}", diag.effective_df);
// Quality checks
if diag.effective_df < 2.0 {
eprintln!("Warning: Very low degrees of freedom");
}
}
# Ok::<(), LowessError>(())
### 6. Convenience Constructors
Pre-configured builders for common scenarios:
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
// For noisy data with outliers
let result = Lowess::<f64>::robust().adapter(Batch).build()?.fit(&x, &y)?;
// For speed on clean data
let result = Lowess::<f64>::quick().adapter(Batch).build()?.fit(&x, &y)?;
# Ok::<(), LowessError>(())
## API Overview
### Builder Methods
use lowess::prelude::*;
Lowess::new()
// Core parameters
.fraction(0.5) // Smoothing span (0, 1], default: 0.67
.iterations(3) // Robustness iterations, default: 3
.delta(0.01) // Interpolation threshold
// Kernel selection
.weight_function(WeightFunction::Tricube) // Default
// Robustness method
.robustness_method(RobustnessMethod::Bisquare) // Default
// Intervals & diagnostics
.confidence_intervals(0.95)
.prediction_intervals(0.95)
.return_diagnostics()
.return_residuals()
.return_robustness_weights()
// Parameter selection
.cross_validate(&[0.3, 0.5, 0.7], CrossValidationStrategy::KFold, Some(5))
// Convergence
.auto_converge(1e-4)
.max_iterations(20)
// Execution mode
.adapter(Batch) // or Streaming, Online
.build()?; // Build the model
### Result Structure
pub struct LowessResult<T> {
pub x: Vec<T>, // Sorted x values
pub y: Vec<T>, // Smoothed y values
pub standard_errors: Option<Vec<T>>, // Point-wise SE
pub confidence_lower: Option<Vec<T>>, // CI lower bound
pub confidence_upper: Option<Vec<T>>, // CI upper bound
pub prediction_lower: Option<Vec<T>>, // PI lower bound
pub prediction_upper: Option<Vec<T>>, // PI upper bound
pub residuals: Option<Vec<T>>, // y - fitted
pub robustness_weights: Option<Vec<T>>, // Final IRLS weights
pub diagnostics: Option<Diagnostics<T>>,
pub iterations_used: Option<usize>, // Actual iterations
pub fraction_used: T, // Selected fraction
pub cv_scores: Option<Vec<T>>, // CV RMSE per fraction
}
## Execution Modes
Choose the right execution mode based on your use case:
### Batch Processing (Standard)
For complete datasets in memory with full feature support:
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let model = Lowess::new()
.fraction(0.5)
.confidence_intervals(0.95)
.return_diagnostics()
.adapter(Batch) // All features supported
.build()?;
let result = model.fit(&x, &y)?;
# Ok::<(), LowessError>(())
### Streaming Processing
For large datasets (>100K points) that don't fit in memory:
use lowess::prelude::*;
let mut processor = Lowess::new()
.fraction(0.3)
.iterations(2)
.adapter(Streaming)
.chunk_size(1000) // Process 1000 points at a time
.overlap(100) // 100 points overlap between chunks
.build()?;
// Process data in chunks
for chunk in data_chunks {
let result = processor.process_chunk(&chunk.x, &chunk.y)?;
// Handle results incrementally
}
let final_result = processor.finalize()?;
# Ok::<(), LowessError>(())
### Online/Incremental Processing
For real-time data streams with sliding window:
use lowess::prelude::*;
let mut processor = Lowess::new()
.fraction(0.2)
.iterations(1)
.adapter(Online)
.window_capacity(100) // Keep last 100 points
.build()?;
// Process points as they arrive
for (x, y) in data_stream {
if let Some(output) = processor.add_point(x, y)? {
println!("Smoothed: {}", output.smoothed);
}
}
# Ok::<(), LowessError>(())
## Parameter Selection Guide
### Fraction (Smoothing Span)
- **0.1-0.3**: Local, captures rapid changes (wiggly)
- **0.4-0.6**: Balanced, general-purpose
- **0.7-1.0**: Global, smooth trends only
- **Default: 0.67** (2/3, Cleveland's choice)
- **Use CV** when uncertain
### Robustness Iterations
- **0**: Clean data, speed critical
- **1-2**: Light contamination
- **3**: Default, good balance (recommended)
- **4-5**: Heavy outliers
- **>5**: Diminishing returns
### Kernel Function
- **Tricube** (default): Best all-around, smooth, efficient
- **Epanechnikov**: Theoretically optimal MSE
- **Gaussian**: Very smooth, no compact support
- **Uniform**: Fastest, least smooth (moving average)
### Delta Optimization
- **None**: Small datasets (n < 1000)
- **0.01 × range(x)**: Good starting point for dense data
- **Manual tuning**: Adjust based on data density
## Error Handling
use lowess::prelude::*;
# let x = vec![1.0, 2.0, 3.0];
# let y = vec![2.0, 4.0, 6.0];
match Lowess::new().adapter(Batch).build()?.fit(&x, &y) {
Ok(result) => {
println!("Success: {:?}", result.y);
},
Err(LowessError::EmptyInput) => {
eprintln!("Empty input arrays");
},
Err(LowessError::MismatchedInputs { x_len, y_len }) => {
eprintln!("Length mismatch: x={}, y={}", x_len, y_len);
},
Err(LowessError::InvalidFraction(f)) => {
eprintln!("Invalid fraction: {} (must be in (0, 1])", f);
},
Err(e) => {
eprintln!("Error: {}", e);
}
}
# Ok::<(), LowessError>(())
## Examples
Comprehensive examples are available in the `examples/` directory:
- **`batch_smoothing.rs`** - 8 scenarios covering batch processing
- Basic smoothing, robust outlier handling, uncertainty quantification
- Cross-validation, diagnostics, kernel comparisons, robustness methods
- Quick and robust presets
- **`online_smoothing.rs`** - 6 scenarios for real-time processing
- Basic streaming, sensor data simulation, outlier handling
- Window size effects, memory-bounded processing, sliding window behavior
- **`streaming_smoothing.rs`** - 6 scenarios for large datasets
- Basic chunking, chunk size comparison, overlap strategies
- Large dataset processing, outlier handling, file-based simulation
Run examples with:
cargo run --example batch_smoothing
cargo run --example online_smoothing
cargo run --example streaming_smoothing
## Feature Flags
- **`std`** (default): Standard library support
- **`dev`**: Exposes internal modules for testing (automatically enabled in development)
### Standard configuration
[dependencies]
lowess = "0.4"
### No-std configuration (requires alloc)
[dependencies]
lowess = { version = "0.4", default-features = false }
## Validation
This implementation has been extensively validated against:
1. **R's stats::lowess**: Numerical agreement to machine precision
2. **Python's statsmodels**: Validated on 44 test scenarios
3. **Cleveland's original paper**: Reproduces published examples
## MSRV (Minimum Supported Rust Version)
Rust **1.85.0** or later (requires Rust Edition 2024).
## Contributing
Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for:
- Bug reports and feature requests
- Pull request guidelines
- Development workflow
- Testing requirements
## License
MIT License - see [LICENSE](LICENSE) file.
## References
**Original papers:**
- Cleveland, W.S. (1979). "Robust Locally Weighted Regression and Smoothing Scatterplots". _Journal of the American Statistical Association_, 74(368): 829-836. [DOI:10.2307/2286407](https://doi.org/10.2307/2286407)
- Cleveland, W.S. (1981). "LOWESS: A Program for Smoothing Scatterplots by Robust Locally Weighted Regression". _The American Statistician_, 35(1): 54.
**Related implementations:**
- [R stats::lowess](https://stat.ethz.ch/R-manual/R-devel/library/stats/html/lowess.html)
- [Python statsmodels](https://www.statsmodels.org/stable/generated/statsmodels.nonparametric.smoothers_lowess.lowess.html)
## Citation
@software{lowess_rust_2025,
author = {Valizadeh, Amir},
title = {lowess: High-performance LOWESS for Rust},
year = {2025},
url = {https://github.com/thisisamirv/lowess},
version = {0.4.0}
}
## Author
**Amir Valizadeh**
📧 <thisisamirv@gmail.com>
🔗 [GitHub](https://github.com/thisisamirv/lowess)
---
**Keywords**: LOWESS, LOESS, local regression, nonparametric regression, smoothing, robust statistics, time series, bioinformatics, genomics, signal processing