# lowess
[](https://crates.io/crates/lowess)
[](https://docs.rs/lowess)
[](LICENSE)
[](https://www.rust-lang.org)
**High-performance LOWESS (Locally Weighted Scatterplot Smoothing) for Rust** — 40-500× faster than Python's statsmodels with robust statistics, confidence intervals, and parallel execution.
## Why This Crate?
- ⚡ **Blazingly Fast**: 40-500× faster than statsmodels, sub-millisecond smoothing for 1000 points
- 🎯 **Production-Ready**: Comprehensive error handling, numerical stability, extensive testing
- 📊 **Feature-Rich**: Confidence/prediction intervals, multiple kernels, cross-validation
- 🚀 **Scalable**: Parallel execution, streaming mode, delta optimization
- 🔬 **Scientific**: Validated against R and Python implementations
- 🛠️ **Flexible**: `no_std` support, multiple robustness methods
## Quick Start
```rust
use lowess::Lowess;
let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
// Basic smoothing
let result = Lowess::new()
.fraction(0.5)
.fit(&x, &y)
.unwrap();
println!("Smoothed: {:?}", result.y);
```
## Installation
```toml
[dependencies]
lowess = "0.3"
# For no_std environments (requires alloc)
lowess = { version = "0.3", default-features = false }
```
## Features at a Glance
| **Robust Smoothing** | IRLS with Bisquare/Huber/Talwar weights | Outlier-contaminated data |
| **Confidence Intervals** | Point-wise standard errors & bounds | Uncertainty quantification |
| **Cross-Validation** | Auto-select optimal fraction | Unknown smoothing parameter |
| **Multiple Kernels** | Tricube, Epanechnikov, Gaussian, etc. | Different smoothness profiles |
| **Parallel Execution** | Multi-threaded via Rayon (std feature) | Large datasets (n > 1000) |
| **Streaming Mode** | Constant memory usage | Very large datasets |
| **Delta Optimization** | Skip dense regions | 10× speedup on dense data |
## Common Use Cases
### 1. Robust Smoothing (Handle Outliers)
```rust
use lowess::Lowess;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
.fraction(0.3)
.iterations(5) // Robust iterations
.with_robustness_weights() // Return outlier weights
.fit(&x, &y)?;
// Check which points were downweighted
if let Some(weights) = result.robustness_weights {
for (i, &w) in weights.iter().enumerate() {
if w < 0.1 {
println!("Point {} is likely an outlier", i);
}
}
}
# Ok::<(), lowess::LowessError>(())
```
### 2. Uncertainty Quantification
```rust
use lowess::Lowess;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
.fraction(0.5)
.with_confidence_intervals(0.95)
.with_prediction_intervals(0.95)
.fit(&x, &y)?;
// Plot confidence bands
for i in 0..x.len() {
println!("x={:.1}: y={:.2} CI=[{:.2}, {:.2}]",
result.x[i],
result.y[i],
result.confidence_lower.unwrap()[i],
result.confidence_upper.unwrap()[i]
);
}
# Ok::<(), lowess::LowessError>(())
```
### 3. Automatic Parameter Selection
```rust
use lowess::Lowess;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8, 12.0, 14.1, 16.0];
// Let cross-validation find the optimal smoothing fraction
let result = Lowess::new()
.cross_validate(&[0.2, 0.3, 0.5, 0.7])
.fit(&x, &y)?;
println!("Optimal fraction: {}", result.fraction_used);
println!("CV RMSE scores: {:?}", result.cv_scores);
# Ok::<(), lowess::LowessError>(())
```
### 4. Large Dataset Optimization
```rust
use lowess::Lowess;
# let large_x: Vec<f64> = (0..5000).map(|i| i as f64).collect();
# let large_y: Vec<f64> = large_x.iter().map(|&x| x.sin()).collect();
// Enable all performance optimizations
let result = Lowess::new()
.fraction(0.3)
.delta(0.01) // Skip dense regions
.parallel(true) // Multi-threaded (requires std feature)
.fit(&large_x, &large_y)?;
# Ok::<(), lowess::LowessError>(())
```
### 5. Production Monitoring
```rust
use lowess::Lowess;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
let result = Lowess::new()
.fraction(0.5)
.iterations(3)
.with_diagnostics()
.fit(&x, &y)?;
if let Some(diag) = result.diagnostics {
println!("RMSE: {:.4}", diag.rmse);
println!("R²: {:.4}", diag.r_squared);
println!("Effective DF: {:.2}", diag.effective_df.unwrap());
// Quality checks
if diag.effective_df.unwrap() < 2.0 {
eprintln!("Warning: Very low degrees of freedom");
}
}
# Ok::<(), lowess::LowessError>(())
```
### 6. Convenience Constructors
Pre-configured builders for common scenarios:
```rust
use lowess::Lowess;
# let x = vec![1.0, 2.0, 3.0, 4.0, 5.0];
# let y = vec![2.0, 4.1, 5.9, 8.2, 9.8];
// For noisy data with outliers
let result = Lowess::robust().fit(&x, &y)?;
// For speed on clean data
let result = Lowess::quick().fit(&x, &y)?;
// For comprehensive analysis
let result = Lowess::detailed().fit(&x, &y)?;
# Ok::<(), lowess::LowessError>(())
```
## Performance Benchmarks
Comparison against Python's statsmodels on typical workloads:
| 100 points | 2.71 ms | 0.17 ms | 0.15 ms | **16×** | **32×** |
| 1,000 points | 36.32 ms | 8.65 ms | 1.47 ms | **4×** | **39×** |
| 5,000 points | 373.15 ms | 211.87 ms | 6.97 ms | **2×** | **63×** |
| 10,000 points | 1,245.80 ms | 897.44 ms | 12.68 ms | **1.4×** | **110×** |
_Benchmarks conducted on ntel Core Ultra 7 268V (8 cores @ up to 5.0 GHz) running Arch Linux (6.17.9-arch1-1). See `validation/` directory for detailed methodology and reproducible test scripts._
## API Overview
### Builder Methods
```rust
use lowess::{Lowess, WeightFunction, builder::RobustnessMethod};
Lowess::new()
// Core parameters
.fraction(0.5) // Smoothing span (0, 1], default: 0.67
.iterations(3) // Robustness iterations, default: 3
.delta(0.01) // Interpolation threshold
// Kernel selection
.weight_function(WeightFunction::Tricube) // Default
// Robustness method
.robustness_method(RobustnessMethod::Bisquare) // Default
// Intervals & diagnostics
.with_confidence_intervals(0.95)
.with_prediction_intervals(0.95)
.with_both_intervals(0.95)
.with_diagnostics()
.with_all_diagnostics()
.with_residuals()
.with_robustness_weights()
// Parameter selection
.cross_validate(&[0.3, 0.5, 0.7])
.cross_validate_kfold(&[0.3, 0.5, 0.7], 5)
.cross_validate_loocv(&[0.3, 0.5, 0.7])
// Convergence
.auto_converge(1e-4)
.max_iterations(20)
// Performance (requires std feature, enabled by default)
.parallel(true)
// Convenience constructors
// Lowess::robust() // Pre-configured for outliers
// Lowess::quick() // Pre-configured for speed
// Lowess::detailed() // Pre-configured for analysis
;
```
### Result Structure
```rust
pub struct LowessResult<T> {
pub x: Vec<T>, // Sorted x values
pub y: Vec<T>, // Smoothed y values
pub standard_errors: Option<Vec<T>>, // Point-wise SE
pub confidence_lower: Option<Vec<T>>, // CI lower bound
pub confidence_upper: Option<Vec<T>>, // CI upper bound
pub prediction_lower: Option<Vec<T>>, // PI lower bound
pub prediction_upper: Option<Vec<T>>, // PI upper bound
pub residuals: Option<Vec<T>>, // y - fitted
pub robustness_weights: Option<Vec<T>>, // Final IRLS weights
pub diagnostics: Option<Diagnostics<T>>,
pub iterations_used: Option<usize>, // Actual iterations
pub fraction_used: T, // Selected fraction
pub cv_scores: Option<Vec<T>>, // CV RMSE per fraction
}
```
## Advanced Features
### Streaming Processing
For datasets too large to fit in memory:
```rust
use lowess::{Lowess, ProcessingMode, ProcessingVariant};
let variant = Lowess::new()
.fraction(0.3)
.for_mode(ProcessingMode::Streaming)
.chunk_size(1000)
.build()?;
match variant {
ProcessingVariant::Streaming(builder) => {
// Use streaming builder
},
_ => {}
}
# Ok::<(), lowess::LowessError>(())
```
### Online/Incremental Updates
Real-time smoothing with sliding window:
```rust
use lowess::{Lowess, ProcessingMode, ProcessingVariant};
let variant = Lowess::new()
.fraction(0.2)
.for_mode(ProcessingMode::Online)
.window_size(100)
.build()?;
match variant {
ProcessingVariant::Online(builder) => {
// Use online builder
},
_ => {}
}
# Ok::<(), lowess::LowessError>(())
```
### ndarray Integration
ndarray is always available (no feature flag needed):
```rust
use lowess::Lowess;
use ndarray::Array1;
let x: Array1<f64> = Array1::linspace(0.0, 10.0, 100);
let result = Lowess::new()
.fraction(0.3)
.fit(x.as_slice().unwrap(), y.as_slice().unwrap())?;
// Convert back to ndarray
let smoothed = Array1::from(result.y);
# Ok::<(), lowess::LowessError>(())
```
## Parameter Selection Guide
### Fraction (Smoothing Span)
- **0.1-0.3**: Local, captures rapid changes (wiggly)
- **0.4-0.6**: Balanced, general-purpose
- **0.7-1.0**: Global, smooth trends only
- **Default: 0.67** (2/3, Cleveland's choice)
- **Use CV** when uncertain
### Robustness Iterations
- **0**: Clean data, speed critical
- **1-2**: Light contamination
- **3**: Default, good balance (recommended)
- **4-5**: Heavy outliers
- **>5**: Diminishing returns
### Kernel Function
- **Tricube** (default): Best all-around, smooth, efficient
- **Epanechnikov**: Theoretically optimal MSE
- **Gaussian**: Very smooth, no compact support
- **Uniform**: Fastest, least smooth (moving average)
### Delta Optimization
- **None**: Small datasets (n < 1000)
- **0.01 × range(x)**: Good starting point for dense data
- **Manual tuning**: Adjust based on data density
## Error Handling
```rust
use lowess::{Lowess, LowessError};
match Lowess::new().fit(&x, &y) {
Ok(result) => {
println!("Success: {:?}", result.y);
},
Err(LowessError::EmptyInput) => {
eprintln!("Empty input arrays");
},
Err(LowessError::MismatchedInputs { x_len, y_len }) => {
eprintln!("Length mismatch: x={}, y={}", x_len, y_len);
},
Err(LowessError::InvalidFraction(f)) => {
eprintln!("Invalid fraction: {} (must be in (0, 1])", f);
},
Err(e) => {
eprintln!("Error: {}", e);
}
}
```
## Feature Flags
The crate has only **two features**:
- **`default`**: Enables `std` feature
- **`std`**: Standard library support (includes Rayon for parallelism)
```toml
# Standard configuration (includes parallel execution)
[dependencies]
lowess = "0.3"
# No-std configuration (requires alloc, no parallelism)
[dependencies]
lowess = { version = "0.3", default-features = false }
```
**Note**: There are no separate `parallel` or `ndarray` feature flags in v0.3. When `std` is enabled (default), parallel execution via Rayon is automatically available.
## Validation
This implementation has been extensively validated against:
1. **R's stats::lowess**: Numerical agreement to machine precision
2. **Python's statsmodels**: Validated on 44 test scenarios
3. **Cleveland's original paper**: Reproduces published examples
See `validation/` directory for cross-language comparison scripts.
## MSRV (Minimum Supported Rust Version)
Rust **1.86.0** or later (requires Rust Edition 2024).
## Contributing
Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for:
- Bug reports and feature requests
- Pull request guidelines
- Development workflow
- Testing requirements
## License
MIT License - see [LICENSE](LICENSE) file.
## References
**Original papers:**
- Cleveland, W.S. (1979). "Robust Locally Weighted Regression and Smoothing Scatterplots". _Journal of the American Statistical Association_, 74(368): 829-836. [DOI:10.2307/2286407](https://doi.org/10.2307/2286407)
- Cleveland, W.S. (1981). "LOWESS: A Program for Smoothing Scatterplots by Robust Locally Weighted Regression". _The American Statistician_, 35(1): 54.
**Related implementations:**
- [R stats::lowess](https://stat.ethz.ch/R-manual/R-devel/library/stats/html/lowess.html)
- [Python statsmodels](https://www.statsmodels.org/stable/generated/statsmodels.nonparametric.smoothers_lowess.lowess.html)
## Citation
```bibtex
@software{lowess_rust_2025,
author = {Valizadeh, Amir},
title = {lowess: High-performance LOWESS for Rust},
year = {2025},
url = {https://github.com/thisisamirv/lowess},
version = {0.3.0}
}
```
## Author
**Amir Valizadeh**
📧 thisisamirv@gmail.com
🔗 [GitHub](https://github.com/thisisamirv/lowess)
---
**Keywords**: LOWESS, LOESS, local regression, nonparametric regression, smoothing, robust statistics, time series, bioinformatics, genomics, signal processing