Skip to main content

Crate pomelo_audit

Crate pomelo_audit 

Source
Expand description

Read-only data-quality audit of a pomelo data-layout tree (#133).

Given a synced prices/ / fundamentals/ / panels/ / tracked/ tree, run_data_audit answers “is this clean enough to trust a backtest?” — turning “high-quality data” from a claim into a measurement. It also doubles as the verification tool for #131 (filing-date lag) and #132 (snapshot-factor coverage).

Storage-agnostic (#149): run_data_audit takes any ObjectSource + ObjectLister, so it audits a local tree or an S3/R2 tree identically — yuzu-cli data-audit --data s3://… builds a pomelo_s3::OutStore and passes it straight in. Discovery (which symbols / fundamentals files / membership panels exist) goes through ObjectLister::list; reads go through ObjectSource::get. Over S3 this means shallow checks (coverage, membership) cost a handful of ListObjectsV2 calls, while deep checks (gaps / jumps / NaN / lookahead) GET every object — auditing a full R2 tree deeply is therefore comparable in cost to downloading it (see docs/fmp-data-source.md).

No network beyond the source’s own reads, no engine run. It reuses the pomelo_data loaders and returns a serializable DataAuditReport of per-check OK / WARN / FAIL verdicts, so any front end — yuzu-cli data-audit, a nightly job, or a backend service — can call the same logic. The CLI is a thin shim that renders/emits the report and maps a FAIL to a non-zero exit.

§Module layout

  • [report] — the Status / Check / DataAuditReport shapes, render_table, and the shared formatting/date helpers.
  • [checks] — the individual check_* data-quality checks.
  • [scan] — tree discovery and the single-pass fundamentals scan.
  • this module — run_data_audit orchestration + the public re-exports.

Structs§

Check
One named check: a status, a one-line human summary, and structured details.
DataAuditReport
The full audit report — serialized directly for --json.

Enums§

Status
A single check’s verdict. Ordered Ok < Warn < Fail so the report’s overall status is the max across checks.

Functions§

render_table
Render the report as a compact human-readable table (the CLI’s default output).
run_data_audit
Run every check over the data-layout tree served by src, windowed to [from, to]. data_dir is a display label only (a local path or an s3://… URL) — it never drives I/O; all discovery goes through src. Fail-soft: a missing directory or file downgrades a check, never panics.