datarust-profile 0.2.0

One-call data profiling and quality reports for the datarust ecosystem
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121

# datarust-profile

**One-call data profiling and data-quality reports** for the [datarust] ecosystem.

`datarust-profile` takes a numeric [`Matrix`], a categorical [`StrMatrix`], or a
mixed table of both, and produces a [`DatasetProfile`] describing shape, memory
footprint, duplicate rows, per-column descriptive statistics (mean, std,
five-number summary, cardinality, top value) and a list of data-quality
findings. Profiles can be rendered to JSON (via the `serde` feature) or to a
self-contained HTML report with no extra dependencies.

It is the Rust-native, dependency-free counterpart of tools like
[ydata-profiling] / [fg-data-profiling]: instead of a Python package wrapping
pandas, it is a small crate built on [datarust]'s statistics primitives and
designed to be embedded in CLIs, services, notebooks, and CI pipelines.

[datarust]: https://crates.io/crates/datarust
[ydata-profiling]: https://github.com/ydataai/ydata-profiling
[fg-data-profiling]: https://github.com/Data-Centric-AI-Community/fg-data-profiling

## Install

```toml
[dependencies]
datarust-profile = "0.1"
```

For JSON output, enable the `serde` feature (this also pulls in
`datarust/serde`):

```toml
[dependencies]
datarust-profile = { version = "0.1", features = ["serde"] }
```

The default build has **zero external dependencies** beyond `datarust` itself.

## Quick start

```rust
use datarust::Matrix;
use datarust_profile::{profile_matrix, report};

let m = Matrix::from_rows(vec![
    vec![1.0, 10.0],
    vec![2.0, 12.0],
    vec![3.0, f64::NAN],
]).unwrap();

let profile = profile_matrix(&m, Some(&["x".into(), "y".into()])).unwrap();

println!("{}x{}, {} duplicates",
    profile.n_rows, profile.n_columns, profile.duplicate_rows);

for col in &profile.columns {
    println!("{:<6} {:<11} missing={:.0}%",
        col.name, col.column_type, col.missing_fraction * 100.0);
}

// Self-contained HTML report (no extra deps):
std::fs::write("profile.html", report::to_html(&profile)).unwrap();
```

## What it computes

For each column, depending on its inferred type:

| Numeric                           | Categorical                          |
|-----------------------------------|--------------------------------------|
| `count`, `missing_count`, `missing_fraction` | `count`, `missing_count`, `missing_fraction` |
| `mean`, `std` (sample, ddof = 1)  | `unique` (cardinality)               |
| five-number summary: min/Q1/median/Q3/max | `top` (most frequent value)   |
|                                   | `freq` (count of `top`)              |

Dataset-wide: `n_rows`, `n_columns`, estimated `memory_bytes`, exact
`duplicate_rows` and `duplicate_fraction`.

### Data-quality findings

`quality::run_checks` scans the profile against configurable
[`Thresholds`] and emits [`QualityIssue`]s:

- **HighMissing** — missing fraction at/above threshold.
- **ConstantColumn** — numeric column with near-zero variance.
- **NearUnique** — categorical column whose cardinality ≈ row count (likely an
  identifier, not a feature).
- **DuplicateRows** — exact-duplicate rows present.

Each finding carries a [`Severity`] (`Info` / `Warning` / `Critical`) and an
optional column name. The HTML and JSON renderers include findings by default.

## Output formats

| Format | Feature  | Notes                                              |
|--------|----------|----------------------------------------------------|
| HTML   | —        | Single self-contained `.html`, inline CSS, no JS.  |
| JSON   | `serde`  | Pretty-printed; schema mirrors the in-memory types. |

```rust
#[cfg(feature = "serde")]
{
    use datarust_profile::report::{to_json, JsonReport};
    let json = to_json(&JsonReport::from_profile(&profile)).unwrap();
    std::fs::write("profile.json", json).unwrap();
}
```

## Relationship to `datarust`

`datarust-profile` is a sibling crate in the [datarust workspace][repo]. It
reuses `datarust`'s `Matrix` / `StrMatrix` containers and its `stats` module
(`mean`, `std`, `quantile`, `median_sorted`) rather than reimplementing them, so
profiling stays consistent with the rest of the ecosystem. Column-type
inference, missing-value handling, cardinality counting, and the report
renderers live in this crate.

[repo]: https://github.com/genc-murat/datarust

## License

MIT.