# Changelog
## 0.2.0-beta.1 (2026-04-05)
### Architecture
- **Split `pipeline.rs` (2447 lines) into `pipeline/` module** with 7 focused submodules:
`chunked`, `cli`, `mod` (orchestration), `retry`, `single`, `sink`, `validate`.
- **Split `config.rs` (1708 lines) into `config/` module** with 4 submodules:
`models`, `resolve`, `tests`, `mod` (validation & loading).
- **Split `preflight.rs` (1425 lines) into `preflight/` module** with 5 submodules:
`analysis`, `doctor`, `mod` (orchestration), `mysql`, `postgres`.
- All public API paths unchanged — external callers unaffected.
### Reliability
- **Export failures now propagate to CLI exit code**: `run_export_job` returns
`Result<()>` and failures are collected; `rivet run` exits with non-zero when
any export fails (critical for CI, cron, and orchestrators).
- **SQLite migration errors are fatal**: `migrate()` returns `Result` and
`StateStore::open()` fails if any migration step errors, preventing silent
partial schema states.
- **Typed error classification**: `classify_error` now checks Postgres `SqlState`
codes and MySQL numeric error codes before falling back to string matching,
giving more precise transient-vs-permanent classification for retries.
- **Replaced production `unwrap()` calls** with `expect()` and descriptive messages
across `pipeline/`, `config/`, `state.rs`, `format/csv.rs`, and `source/`.
- **Versioned SQLite schema migrations**: `schema_version` table tracks applied
migrations; new databases start at v3, legacy databases are detected and upgraded
automatically. Future schema changes only require adding a new migration entry.
- **Graceful mutex poison handling**: parallel chunked workers use
`unwrap_or_else(|e| e.into_inner())` instead of `expect()`, preventing
cascading panics if a worker thread panics.
### Code quality
- **Zero clippy warnings**: resolved all `collapsible_if`, `too_many_arguments`,
`derivable_impl`, `manual_clamp`, `if_let_some_result`, `manual_range_contains`,
`write_literal`, `is_multiple_of`, unused-import, and needless-borrow lints.
- Added `// SAFETY:` documentation on the sole `unsafe` block (`resource.rs` macOS RSS).
- Added crate-level `//!` documentation to `lib.rs`.
- **Tuning `profile_name()` now returns the configured profile** rather than
inferring from numeric fields, ensuring metrics and logs match the YAML config.
### Memory optimization
- **Streaming cloud uploads**: S3, GCS, and stdout destinations now use
`std::io::copy` instead of loading entire temporary files into RAM. Memory
footprint during upload is O(buffer) instead of O(file_size).
- **Early `drop(rows)` in Postgres source**: raw `Vec<Row>` is freed
immediately after conversion to Arrow `RecordBatch`, reducing transient
memory overlap.
- **jemalloc** (`tikv-jemallocator`) added as an optional default-on allocator.
jemalloc aggressively returns freed pages to the OS, reducing peak RSS by
~30–40% at smaller batch sizes compared to the system allocator.
### Config validation
- **Misplaced tuning field detection**: if `batch_size`, `profile`,
`throttle_ms`, or other tuning fields are placed directly under `source:`
or in an `exports[]` entry instead of inside `tuning:`, Rivet now rejects
the config with a clear error and a fix suggestion. Previously, these
fields were silently ignored by serde, causing unexpected defaults.
### Testing
- **617 tests** (537 unit + 80 integration), up from 274.
- New test coverage for: cursor extraction (all Arrow types), strip internal
column, quality tracking, validate_output (corrupt/empty/missing files),
CSV golden tests (Binary, Float32, Int16, Boolean+nulls, multi-batch),
Parquet nullable + multi-batch roundtrip, resolve_vars edge cases,
parse_file_size regressions, notify trigger matching, quality multi-batch
aggregation, parse_params, format_bytes boundary values.
### Dependencies
- **Replaced deprecated `serde_yaml`** with `serde_yml` 0.0.12.
- Updated 52 transitive dependencies (tokio 1.51, hyper 1.9, postgres 0.19.13,
libc 0.2.184, and others).
### Documentation
- **USER_GUIDE.md**: added jemalloc section, memory optimization tips, streaming
upload notes, misplaced tuning field detection, troubleshooting section,
documented `--export` and `--last` flags for `metrics` and `state files`.
- **README.md**: added stdout destination to config reference.
### Packaging
- Added `license = "MIT"`, `repository`, `rust-version = "1.94"` to `Cargo.toml`.
- Added `LICENSE` (MIT) file.
- Added `rust-toolchain.toml` pinning toolchain to 1.94 with rustfmt + clippy.
- Added `exclude` list to `Cargo.toml` for clean `cargo publish` (excludes `dev/`,
`tests/`, `.github/`, `USER_TEST_PLAN.md`).
### CI
- `.github/workflows/ci.yml` with five jobs: `rustfmt`, `clippy -D warnings`,
`cargo test`, `cargo build --release`, and **`cargo audit`** (security).
- All jobs pinned to Rust **1.94** (matches `rust-version` and `rust-toolchain.toml`).
## 0.1.0
Initial release.