Crate datarust_profile

Expand description

§datarust-profile

One-call data profiling and data-quality reports for the datarust ecosystem.

datarust-profile takes a numeric datarust::Matrix, a categorical datarust::StrMatrix, a mixed table of both — or, behind the csv feature, a CSV file path — and produces a DatasetProfile describing shape, memory footprint, duplicate rows, per-column descriptive statistics (mean, std, five-number summary, cardinality, top value) and a list of data-quality findings. Profiles can be rendered to JSON (via the serde feature), to a self-contained HTML report, or to a terminal table (via the pretty feature) — with no extra dependencies.

§Features

csv — pure-Rust RFC-4180 reader (csv) plus one-call entry points (profile_csv, DatasetProfile::from_csv, DatasetProfile::from_csv_with_types).
streaming — single-pass bounded-memory CSV profiling (profile_csv_streaming). Implies csv.
pretty — terminal-table renderer (report::to_terminal).
cli — the datarust-profile binary (CSV → HTML/JSON/terminal).
serde — JSON serialization of profiles and findings.

§Quick start

use datarust::Matrix;
use datarust_profile::{profile_matrix, report};

let m = Matrix::from_rows(vec![
    vec![1.0, 10.0],
    vec![2.0, 12.0],
    vec![3.0, f64::NAN],
]).unwrap();

let profile = profile_matrix(&m, None).unwrap();
println!("{}x{}, {} duplicates",
    profile.n_rows, profile.n_columns, profile.duplicate_rows);

// HTML report (always available, no extra deps):
let html = report::to_html(&profile);
std::fs::write("profile.html", html).unwrap();

§JSON output (feature `serde`)

Requires the serde feature. The example below is marked ignore so that it is not compiled by cargo test --doc when serde is disabled.

use datarust::Matrix;
use datarust_profile::{profile_matrix, report};

let m = Matrix::from_rows(vec![vec![1.0]]).unwrap();
let profile = profile_matrix(&m, None).unwrap();
let json = report::to_json(
    &report::JsonReport::from_profile(&profile),
).unwrap();
std::fs::write("profile.json", json).unwrap();

Re-exports§

pub use error::ProfileError;
pub use error::Result;
pub use profile::CategoricalStats;
pub use profile::ColumnProfile;
pub use profile::CorrelationMatrix;
pub use profile::DatasetProfile;
pub use profile::FiveNumber;
pub use profile::Histogram;
pub use profile::NumericStats;
pub use profile::PointBiserialEntry;
pub use profile::Relationships;
pub use quality::run_checks;
pub use quality::QualityIssue;
pub use quality::QualityKind;
pub use quality::Thresholds;
pub use types::ColumnType;
pub use types::Severity;
pub use csv::read_csv;
pub use csv::read_csv_reader;
pub use csv::read_csv_with;
pub use csv::CsvReadOptions;
pub use csv::CsvTable;
pub use streaming::profile_csv_streaming;
pub use streaming::profile_csv_streaming_with;
pub use streaming::StreamingOptions;

Modules§

csv: Pure-Rust CSV reading for the csv feature.
error: Error types returned by datarust-profile.
infer: Column-type inference for string-typed data.
profile: Dataset and column profiling.
quality: Data-quality checks and findings.
report: Report renderers for a crate::DatasetProfile.
streaming: Streaming single-pass CSV profiling (the streaming feature).
types: Shared value types used across the profiling modules.

Functions§

profile_csv: Profiles a CSV file, inferring each column’s type.
profile_csv_with: Profiles a CSV file with explicit csv::CsvReadOptions. Requires the csv feature.
profile_csv_with_types: Profiles a CSV file with per-column forced types (see DatasetProfile::from_csv_with_types). Requires the csv feature.
profile_matrix: Profiles a numeric Matrix.
profile_matrix_with_target: Profiles a numeric Matrix and sets a designated target column for leakage detection.
profile_str_matrix: Profiles a string StrMatrix, inferring each column’s type.
profile_table: Profiles a mixed table: a numeric Matrix block followed by a categorical StrMatrix block, both with the same row count.
profile_table_with_target: Profiles a mixed table and sets a designated target column for leakage detection.