Expand description
Read-only data-quality audit of a pomelo data-layout tree (#133).
Given a synced prices/ / fundamentals/ / panels/ / tracked/ tree,
run_data_audit answers “is this clean enough to trust a backtest?” —
turning “high-quality data” from a claim into a measurement. It also
doubles as the verification tool for #131 (filing-date lag) and #132
(snapshot-factor coverage).
Storage-agnostic (#149): run_data_audit takes any ObjectSource +
ObjectLister, so it audits a local tree or an S3/R2 tree identically —
yuzu-cli data-audit --data s3://… builds a pomelo_s3::OutStore and
passes it straight in. Discovery (which symbols / fundamentals files /
membership panels exist) goes through ObjectLister::list; reads go
through ObjectSource::get. Over S3 this means shallow checks (coverage,
membership) cost a handful of ListObjectsV2 calls, while deep checks
(gaps / jumps / NaN / lookahead) GET every object — auditing a full R2 tree
deeply is therefore comparable in cost to downloading it (see
docs/fmp-data-source.md).
No network beyond the source’s own reads, no engine run. It reuses the
pomelo_data loaders and returns a serializable DataAuditReport of
per-check OK / WARN / FAIL verdicts, so any front end —
yuzu-cli data-audit, a nightly job, or a backend service — can call the
same logic. The CLI is a thin shim that renders/emits the report and maps a
FAIL to a non-zero exit.
§Module layout
- [
report] — theStatus/Check/DataAuditReportshapes,render_table, and the shared formatting/date helpers. - [
checks] — the individualcheck_*data-quality checks. - [
scan] — tree discovery and the single-pass fundamentals scan. - this module —
run_data_auditorchestration + the public re-exports.
Structs§
- Check
- One named check: a status, a one-line human summary, and structured details.
- Data
Audit Report - The full audit report — serialized directly for
--json.
Enums§
- Status
- A single check’s verdict. Ordered
Ok < Warn < Failso the report’s overall status is the max across checks.
Functions§
- render_
table - Render the report as a compact human-readable table (the CLI’s default output).
- run_
data_ audit - Run every check over the data-layout tree served by
src, windowed to[from, to].data_diris a display label only (a local path or ans3://…URL) — it never drives I/O; all discovery goes throughsrc. Fail-soft: a missing directory or file downgrades a check, never panics.