oxits 0.1.0

Time series classification and transformation library for Rust
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129

# Cross-Language Validation

This directory documents the validation methodology used to verify that
oxits (Rust) produces results consistent with pyts (Python), the reference
implementation.

## Methodology

### Golden Data Generation

Python scripts in `test_harness/generators/` use pyts to generate reference
output for known inputs. The master script `test_harness/generate_golden_data.py`
runs all 9 generators and writes JSON fixtures to `tests/golden_data/`.

```
test_harness/
  generate_golden_data.py          # master script
  generators/
    gen_preprocessing.py           # 8 fixtures
    gen_approximation.py           # 7 fixtures
    gen_metrics.py                 # 5 fixtures
    gen_bag_of_words.py            # 2 fixtures
    gen_image.py                   # 6 fixtures
    gen_decomposition.py           # 3 fixtures
    gen_transformation.py          # 3 fixtures
    gen_classification.py          # 4 fixtures
    gen_multivariate.py            # 3 fixtures
```

Total: **41 golden fixtures** across 9 modules.

### Rust Golden Tests

Integration tests in `tests/golden_*.rs` load the JSON fixtures and compare
oxits output against pyts reference data. The common test infrastructure is
in `tests/common/mod.rs`.

| Test file | Module | Tests | Comparison |
|-----------|--------|-------|------------|
| `golden_preprocessing.rs` | Preprocessing | 8 | Exact (eps < 1e-10) |
| `golden_approximation.rs` | Approximation | 5 | Exact for PAA/DFT; +-1 bin for SAX |
| `golden_bag_of_words.rs` | Bag of Words | 2 | Sorted word set equality |
| `golden_decomposition.rs` | Decomposition | 3 | Sign-aware for SSA; reconstruction check |
| `golden_image.rs` | Image | 3 | Exact for GAF (eps < 1e-7); structural for reduced |
| `golden_metrics.rs` | Metrics | 5 | Exact (eps < 1e-10) |
| `golden_transformation.rs` | Transformation | 3 | Structural (shape, non-negative, non-empty) |
| `golden_classification.rs` | Classification | 4 | Exact for KNN; structural for BOSSVS |
| `golden_multivariate.rs` | Multivariate | 3 | Exact for JRP+threshold; structural for percentage |

Total: **36 golden integration tests** (+ 3 existing = 39 new tests across sessions).

### Comparison Types

**Exact comparison** (epsilon < 1e-10 to 1e-6): Used when both implementations
perform the same arithmetic operations in the same order. Works for:
- Preprocessing scalers (element-wise operations)
- PAA (segment averaging)
- DTW (identical DP recurrence)
- KNN (deterministic distance + sorting)

**Cross-language tolerance** (epsilon < 1e-7): Used when operation ordering
differs between NumPy and Rust (e.g., SIMD reduction order affects floating-point
accumulation). Works for:
- DFT (FFT library differences: numpy.fft vs realfft)
- GAF (trigonometric operations)

**Sign-aware comparison**: SVD implementations (numpy vs nalgebra) may produce
singular vectors with flipped signs. SSA components are compared both directly
and with sign flip. Works for:
- SSA no-grouping (individual eigenvectors)

**Structural validation**: Used when the algorithm pipeline amplifies small
floating-point differences into categorically different outputs:
- SAX: Bin assignments on boundaries (allow +-1 bin)
- BOSS/BagOfPatterns: Vocabulary and TF-IDF settings differ
- BOSSVS: Predictions depend on entire SFA→BOSS→TF-IDF pipeline
- ROCKET: Random kernels use different PRNGs (numpy MT19937 vs Rust)
- SSA auto-grouping: Grouping heuristic is implementation-dependent
- JRP/RP percentage: Percentile rounding differs

### Known Differences

| Algorithm | Difference | Impact |
|-----------|-----------|--------|
| SAX | Bin boundary rounding after StandardScaler | Up to 1 bin shift |
| ROCKET | Different PRNG (numpy vs rand crate) | Different random kernels |
| SSA auto | Different grouping heuristic | Different component groupings |
| BagOfPatterns | L2 normalization in TF-IDF | Different weight magnitudes |
| JRP/RP | Percentile computation rounding | Different binary thresholds |

## Running Validation

### Prerequisites

```bash
cd test_harness
python -m venv .venv
.venv/bin/pip install pyts numpy scikit-learn
```

### Generate Golden Data

```bash
cd test_harness
.venv/bin/python generate_golden_data.py
```

### Run Golden Tests

```bash
cargo test --all-features  # runs all 229 tests including 41 golden tests
```

### Coverage

```bash
cargo llvm-cov --all-features --summary-only
```

Current coverage: **96.9%** line coverage (229 tests, --fail-under-lines 90 in CI).

## Results Summary

- **41 golden fixtures** generated from pyts across 9 modules
- **229 total tests** passing (178 unit + 41 golden + 10 other integration)
- **0 clippy warnings**
- **96.9% line coverage**
- All exact-comparison tests pass at epsilon < 1e-6 or tighter
- Structural tests verify shape, symmetry, value ranges, and reconstruction