Struct Profiler 

pub struct Profiler { /* private fields */ }

Unified profiler with builder pattern

This is the primary entry point for data profiling. It dispatches to the appropriate internal engine based on the configured EngineType.

Examples

use dataprof::{Profiler, EngineType};

// Simple usage with auto engine selection (recommended)
let report = Profiler::new().analyze_file("data.csv").unwrap();

// Explicit engine selection
let report = Profiler::new()
    .engine(EngineType::Columnar)
    .analyze_file("data.csv")
    .unwrap();

Implementations

impl Profiler

pub fn new() -> Self

Create a new profiler with default settings (Auto engine selection)

pub fn with_config(config: ProfilerConfig) -> Self

Create a profiler from an existing configuration

pub fn engine(self, engine: EngineType) -> Self

Set the engine type

pub fn chunk_size(self, chunk_size: ChunkSize) -> Self

Set the chunk size for streaming engines

pub fn sampling(self, strategy: SamplingStrategy) -> Self

Set the sampling strategy

pub fn memory_limit_mb(self, mb: usize) -> Self

Set the memory limit in megabytes (applies to Incremental and Columnar engines)

pub fn stop_when(self, condition: StopCondition) -> Self

Set a stop condition for early termination.

Fully evaluated for CSV sources by EngineType::Auto (which routes to the incremental engine) and EngineType::Incremental. It is also fully evaluated by Profiler::profile_stream.

The columnar engine and the JSON/Parquet parsers cannot evaluate a condition per chunk, so they honour a plain row limit (StopCondition::MaxRows) and reject richer conditions — byte caps, schema stability, confidence thresholds — with DataProfilerError::InvalidConfiguration rather than silently scanning the whole source.

pub fn format(self, format: FileFormat) -> Self

Override automatic format detection.

By default the format is inferred from the file extension. Use this method when the extension is missing or misleading (e.g. a CSV file named .dat).

pub fn csv_delimiter(self, delimiter: u8) -> Self

Set a custom CSV delimiter (single byte). None = auto-detect.

pub fn csv_flexible(self, flexible: bool) -> Self

Set whether to allow ragged CSV rows. None = use parser default.

pub fn json_error_policy(self, policy: JsonErrorPolicy) -> Self

Set how malformed JSON/JSONL records are handled.

Default is JsonErrorPolicy::Skip: malformed records are skipped and counted in execution.error_count. JsonErrorPolicy::Strict aborts on the first malformed record.

pub fn database(self, config: DatabaseConfig) -> Self

Set database connection configuration for analyze_query().

pub fn connection_string(self, conn: &str) -> Self

Convenience: set a database connection string with default config.

pub fn quality_dimensions(self, dims: Vec<QualityDimension>) -> Self

Select which ISO 25012 quality dimensions to compute.

By default all dimensions are evaluated. Call this method with a subset to skip the rest — dimensions that are not requested will appear as None in the report.

Passing an empty vector requests no dimension, which means quality is not analyzed at all: the report carries no quality metrics, exactly as if metric_packs had omitted MetricPack::Quality. That is distinct from requesting dimensions and finding none assessable — that still reports quality, with nothing assessed.

pub fn metric_packs(self, packs: Vec<MetricPack>) -> Self

Select which metric packs to compute.

Controls high-level categories: Schema (always included), Statistics, Patterns, Quality. None = all (default).

pub fn columns(self, columns: Vec<String>) -> Self

Select top-level columns to profile by name.

Parquet applies the selection while decoding; CSV and JSON still read the whole record but only analyze and report the selected columns. Reports preserve source-schema order regardless of the order supplied here. A nested Parquet field is selected as one top-level column, including all of its leaves; dotted leaf paths are not report columns today. Any explicit selection withholds row-level completeness and uniqueness.

pub fn locale(self, locale: Locale) -> Self

Set the locale for pattern detection.

Locale-matching patterns get a confidence boost; non-matching locale patterns are suppressed unless they have a very high match rate.

Parse a tag that came from a user or another language with Locale::parse_optional, which normalises the common spellings and rejects the rest:

use dataprof::{Locale, Profiler};

let profiler = Profiler::new().locale(Locale::It);
let from_user = Locale::parse_optional(Some("it-IT")).unwrap();
assert_eq!(from_user, Some(Locale::It));

pub fn positive_columns(self, columns: Vec<String>) -> Self

Mark columns whose numeric values are expected to be non-negative.

pub fn identifier_columns(self, columns: Vec<String>) -> Self

Mark numeric-looking columns that should be profiled as identifiers.

pub fn temporal_columns(self, columns: Vec<String>) -> Self

Mark columns whose values should contribute to timeliness metrics.

pub fn temporal_pairs(self, pairs: Vec<(String, String)>) -> Self

Declare start/end column pairs for timeliness to compare, such as valid_from/valid_until. Declared pairs are compared as given; the other temporal columns are still paired by name.

pub fn progress_interval(self, interval: Duration) -> Self

Set the progress update interval (default: 500ms)

pub fn progress_sink(self, sink: ProgressSink) -> Self

Set a progress sink for receiving structured progress events.

Every run reports at least Started and Finished. Chunk events come from the incremental engine, which a CSV source reaches with EngineType::Incremental or, when a sink is set, EngineType::Auto. Other routes (the columnar engine, JSON, Parquet) read without reporting, so they are bracketed by Started and Finished alone.

pub fn on_progress<F>(self, callback: F) -> Self

where F: Fn(ProgressEvent) + Send + Sync + 'static,

Set a synchronous callback for progress events (convenience method).

See progress_sink for which events each route reports.

pub fn progress_channel(self, tx: Sender<ProgressEvent>) -> Self

Set an async channel for progress events (requires async-streaming feature).

Intermediate events (such as ProgressEvent::ChunkProcessed) are best-effort and dropped if the channel is full rather than blocking profiling. The terminal ProgressEvent::Finished event uses a slot reserved up front via try_reserve_owned(), guaranteeing delivery without blocking even when the consumer drains the channel only after profiling completes.

pub fn analyze_file<P: AsRef<Path>>( &self, file_path: P, ) -> Result<ProfileReport, DataProfilerError>

Analyze a file and return a quality report

pub fn analyze_source( &self, source: &DataSource, ) -> Result<ProfileReport, DataProfilerError>

Analyze a DataSource and return a quality report.

Supports DataSource::File synchronously. For DataSource::Query, use analyze_source_async() or analyze_query() (requires database feature).

pub async fn analyze_source_async( &self, source: &DataSource, ) -> Result<ProfileReport, DataProfilerError>

Analyze a DataSource asynchronously.

Supports DataSource::File and DataSource::Query. For queries, a database connection must be configured via .database() or .connection_string() (requires database feature).

pub fn infer_schema<P: AsRef<Path>>( &self, path: P, ) -> Result<SchemaResult, DataProfilerError>

Infer the schema (column names + data types) of a file.

Respects the builder’s format override. This is much faster than a full analyze_file — it reads only a small sample of rows. A Parquet file is answered from its metadata except for text columns, whose type only their values decide (#693).

pub fn quick_row_count<P: AsRef<Path>>( &self, path: P, ) -> Result<RowCountEstimate, DataProfilerError>

Quick row count (exact or estimated) for a file.

Respects the builder’s format override. Returns an exact count for small files and Parquet; an estimate for large CSV/JSON files.

pub fn analyze_structure<P: AsRef<Path>>( &self, path: P, max_rows: Option<usize>, ) -> Result<StructureReport, DataProfilerError>

Analyze file structure with a bounded, lightweight pass.

Respects the builder’s format override. This returns schema, row count, and sampled per-column counters without running full profiling, quality metrics, pattern detection, or recommendations.

pub fn detect_format(file_path: &Path) -> FileFormat

Detect file format from extension

impl Profiler

pub async fn analyze_query( &self, query: &str, ) -> Result<ProfileReport, DataProfilerError>

Profile a database query or table asynchronously.

Requires a database connection configured via .database() or .connection_string().

Quality metrics are computed by default. Use .quality_dimensions() to select a subset.

Examples

use dataprof::Profiler;

let report = Profiler::new()
    .connection_string("sqlite:///tmp/test.db")
    .analyze_query("SELECT * FROM users")
    .await?;
println!("Rows: {}", report.execution.rows_processed);

pub async fn analyze_query_no_quality( &self, query: &str, ) -> Result<ProfileReport, DataProfilerError>

Profile a database query without computing quality metrics.

Faster than analyze_query() since it skips ISO 25012 quality metric computation.

impl Profiler

pub async fn profile_stream( &self, source: impl AsyncDataSource, ) -> Result<ProfileReport, DataProfilerError>

Profile data from any async byte stream.

This is the primary async entry point for embedding dataprof in async services. The stream is consumed incrementally — memory usage is bounded regardless of total data size.

Supports CSV, JSON, and JSONL formats. For Parquet, use profile_file or profile_url (requires parquet-async feature) since Parquet requires seeking to the file footer.

Note: EngineType is ignored — async profiling always uses the streaming pipeline internally.

Examples

use dataprof::{Profiler, AsyncSourceInfo, BytesSource, FileFormat};

let csv_data = b"name,age\nAlice,30\nBob,25\n";
let source = BytesSource::new(
    bytes::Bytes::from_static(csv_data),
    AsyncSourceInfo::new("request-body", FileFormat::Csv)
        .size_hint(Some(csv_data.len() as u64)),
);

let report = Profiler::new().profile_stream(source).await?;
println!("Columns: {}", report.execution.columns_detected);

pub async fn profile_file<P: AsRef<Path>>( &self, file_path: P, ) -> Result<ProfileReport, DataProfilerError>

Profile a local file asynchronously.

Detects the format from the file extension (override with .format()). All formats are supported, including Parquet (handled via spawn_blocking).

Examples

use dataprof::Profiler;

let report = Profiler::new().profile_file("data.csv").await?;
println!("Rows: {}", report.execution.rows_processed);

pub async fn infer_schema_stream( &self, source: impl AsyncDataSource, ) -> Result<SchemaResult, DataProfilerError>

Infer schema from any async byte stream.

True async — no file path needed. Reads up to 1000 rows for CSV/JSON/JSONL. Parquet is not supported (requires seeking).

pub async fn quick_row_count_stream( &self, source: impl AsyncDataSource, ) -> Result<RowCountEstimate, DataProfilerError>

Quick row count from any async byte stream.

True async — always a full scan (no sampling, since stream size is unknown). Parquet is not supported (requires seeking).

impl Profiler

pub async fn profile_url( &self, url: &str, ) -> Result<ProfileReport, DataProfilerError>

Profile data from a remote URL.

Supports CSV, JSON, and JSONL when async-streaming is enabled. Remote Parquet additionally requires the parquet-async feature so the footer can be fetched via HTTP Range requests without downloading the entire file.

Format is detected from the URL path extension. Use .format() to override when the URL has no extension (e.g., API endpoints).

Examples

use dataprof::{Profiler, FileFormat};

let report = Profiler::new()
    .format(FileFormat::Csv)
    .profile_url("https://example.com/api/data")
    .await?;

Trait Implementations

impl Default for Profiler

fn default() -> Self

Returns the “default value” for a type.