arcweight 0.3.0

A high-performance, modular library for weighted finite state transducers with comprehensive examples and benchmarks
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108

# Test Organization

This directory contains Rust integration tests for ArcWeight.

## Directory Structure

```
tests/
├── *_tests.rs            # Integration test files (Rust convention)
└── README.md             # This file
```

**Note**: Python binding tests follow industry standard and are located in `pyo3-bindings/tests/`. This follows the pattern used by popular PyO3 projects like rustfst, Polars, and Hugging Face Tokenizers.

## Test File Organization

Rust's test discovery only works for files directly in `tests/`, not in subdirectories. Therefore, all integration test files are kept in the root `tests/` directory with descriptive names.

Integration test files follow a consistent naming convention:
- All files end with `_tests.rs` for consistency
- Files are organized by functionality using descriptive names
- Examples: `algorithms_tests.rs`, `fst_tests.rs`, `io_tests.rs`, `compose_lookahead_tests.rs`

## Test Types

### Unit Tests
Unit tests are located in source files (in `#[cfg(test)]` modules) and test individual functions and methods in isolation.

### Integration Tests
Integration tests are in `tests/*.rs` files and test how components work together. They verify complete workflows and cross-component interactions.

### Doc Tests
Documentation examples in `///` comments serve as tests. All doc examples must compile and run correctly (or be marked with `no_run` for examples requiring external resources).

### Python Tests
Python tests are located in `pyo3-bindings/tests/` (following industry standard for PyO3 projects). They test the Python API exposed by the PyO3 bindings.

## Running Tests

```bash
# Run all Rust tests (unit + integration + doc tests)
cargo test --all-features

# Run only unit tests (in source files)
cargo test --lib

# Run only integration tests
cargo test --test '*'

# Run only doc tests
cargo test --doc --all-features

# Run Python tests
python -m pytest pyo3-bindings/tests/ -v

# Run specific test file
cargo test --test path_iter_tests

# Run with verbose output
cargo test --all-features -- --nocapture
```

## Feature-Specific Tests

Some tests require specific features to be enabled:

```bash
# Test parallel algorithms
cargo test --features parallel

# Test serialization
cargo test --features serde

# Test with custom allocator
cargo test --features fast-alloc

# Test all features (recommended for CI)
cargo test --all-features
```

## Test Coverage

The test suite covers:

- **Unit Tests**: Individual function and method correctness
- **Integration Tests**: Component interactions and workflows
- **Doc Tests**: Documentation example correctness (528+ doc tests)
- **Python Tests**: Python API correctness (187+ tests in `pyo3-bindings/tests/`)
- **Edge Cases**: Error conditions, boundary cases, empty inputs

### Test Counts (v0.3.0)

| Category | Count |
|----------|-------|
| Rust unit tests | 874+ |
| Rust doc tests | 528+ |
| Python tests | 187+ |

## CI Integration

The full test suite runs in CI via `./scripts/ci-check.sh`. This includes:
- Tests with all feature combinations
- Clippy linting
- Documentation build
- Python binding tests
- MSRV compatibility check (Rust 1.85)

See `scripts/README.md` for the complete CI check workflow.