proofframe 0.4.0-alpha.4

Rust-native Arrow data contracts, canonical fingerprints, and proof receipts
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124

<div align="center">
  <img src="https://raw.githubusercontent.com/emirhuseynrmx/proofframe/main/assets/banner.png" alt="ProofFrame" width="100%" />
</div>

# ProofFrame

[![Crates.io](https://img.shields.io/crates/v/proofframe.svg)](https://crates.io/crates/proofframe)
[![docs.rs](https://img.shields.io/docsrs/proofframe)](https://docs.rs/proofframe)
[![CI](https://github.com/emirhuseynrmx/proofframe/actions/workflows/ci.yml/badge.svg)](https://github.com/emirhuseynrmx/proofframe/actions/workflows/ci.yml)
[![License](https://img.shields.io/badge/license-Apache--2.0-blue)](https://github.com/emirhuseynrmx/proofframe/blob/main/LICENSE)
[![MSRV](https://img.shields.io/badge/MSRV-1.85-orange)]()
[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-db61a2)](https://github.com/sponsors/emirhuseynrmx)

**Arrow-native data contracts, canonical fingerprints, and proof receipts for Rust.**

ProofFrame is a Rust crate for checking Arrow `RecordBatchReader` streams and producing deterministic evidence:

- versioned canonical BLAKE3 dataset fingerprints (`pf-fp-v1`);
- typed validation reports with `valid`, `violation_count`, `truncated`, and bounded row findings;
- exact keyed diffs with added, removed, and changed-column evidence;
- high-signal PII scanning and train/test leakage checks;
- Ed25519 signed proof receipts;
- `#![forbid(unsafe_code)]`;
- optional Python bindings behind the `python` feature, not enabled by default.

The default crates.io build is a normal Rust library. It does not require PyO3, does not enable
`pyo3/extension-module`, and does not package Python source into the crate artifact.

## Install

```bash
cargo add proofframe@0.4.0-alpha.4
```

## Quick use

```rust
use proofframe::{profile_reader, validate_reader, ColumnContract, Contract};
use std::collections::HashMap;

let profile = profile_reader(reader)?;

let contract = Contract {
    columns: HashMap::from([
        ("id".to_string(), ColumnContract {
            required: true,
            unique: true,
            not_null: true,
            ..ColumnContract::default()
        }),
    ]),
    max_findings: 100,
};

let report = validate_reader(reader_again, &contract)?;

assert!(profile.fingerprint.starts_with("pf-fp-v1:"));
assert_eq!(report.valid, report.violation_count == 0);
```

## Public Rust API

The core functions accept Arrow readers and return typed Rust structs:

- `profile_reader(reader) -> Result<Profile, ProofFrameError>`
- `profile_reader_with_distinct(reader, DistinctMode::None|Exact) -> Result<Profile, ProofFrameError>`
- `fingerprint_reader(reader) -> Result<String, ProofFrameError>`
- `validate_reader(reader, &contract) -> Result<ValidationReport, ProofFrameError>`
- `validate_fast_reader(reader, &contract) -> Result<FastValidationReport, ProofFrameError>`
- `diff_readers(before, after, &keys) -> Result<DiffReport, ProofFrameError>`
- `scan_pii_reader(reader, max_findings) -> Result<PiiReport, ProofFrameError>`
- `detect_leakage_readers(train, test, &keys, max_samples) -> Result<LeakageReport, ProofFrameError>`
- `receipt::generate_keypair_json()`
- `receipt::sign_json(report_json, private_key)`
- `receipt::verify_json(receipt_json)`

## Features

```toml
[dependencies]
proofframe = "0.4.0-alpha.4"
```

The default feature set is intentionally empty. Enable `python` only when building the Python
extension path:

```toml
proofframe = { version = "0.4.0-alpha.4", features = ["python"] }
```

The PyPI package enables `python` plus `pyo3/extension-module` through maturin. Plain Rust tests and
`cargo package` should not need Python linker symbols, including on macOS.

## Validation semantics

- Null values are ignored by `unique`; combine `unique` with `not_null` when nulls must be rejected.
- Float uniqueness uses IEEE bit semantics via `to_bits()`: `-0.0` and `0.0` are distinct, and NaN
  payload differences are distinct.
- Timestamp uniqueness and min/max use the native integer value in the declared Arrow time unit;
  different timestamp units or timezones are different schema types and are not normalized together.
- Integer min/max and uniqueness use native Arrow integer values, not display strings.
- Decimal values are fingerprinted canonically, but validation min/max does not coerce decimals
  through display text; add an explicit decimal rule in a future contract schema if needed.
- Duplicate findings render only the offending duplicate value when user-facing evidence is emitted;
  clean rows are not stringified for numeric hot paths.

## Why ProofFrame

Most data checks answer one narrow question: did this table pass? ProofFrame is built for the
production question: **what exactly was checked, why did it fail, and can we prove that later?**

It keeps validation evidence bounded while still counting every violation, fingerprints the ordered
Arrow stream with canonical bytes rather than display formatting, and emits deterministic reports
that can be stored in CI, data contracts, and release artifacts.

## Python

Python users should install the wheel from PyPI:

```bash
pip install proofframe==0.4.0a4
```

The Python README and CLI documentation live in the repository `README.md`.