Skip to main content

Crate geoparquet_validator

Crate geoparquet_validator 

Source
Expand description

Validate GeoParquet files: the abstract tests of the OGC GeoParquet 2.0 draft, and the 1.0 and 1.1 community rules, on top of the Apache Arrow Rust parquet crate. No DuckDB, GDAL or PROJ.

The simplest entry points are validate for a path or URL and validate_bytes for a file already in memory; both return a checks::Report, which serialises to the JSON described by schemas/report.schema.json. checks::run takes any source::Source for finer control.

let report = geoparquet_validator::validate("example.parquet", None)?;
for o in &report.outcomes {
    println!("{:?} {} {}", o.status, o.id, o.message);
}
println!("{}", serde_json::to_string_pretty(&report)?);

The same code is the geoparquet-validator command (cli, the default feature), the Python package (python), a C library (capi) and the browser build (wasm).

Modules§

checks
The abstract tests of the OGC GeoParquet 2.0 draft (ogc/abstract_tests), one function per conformance class, run on a single file with nothing but the parquet crate.
cli
The command line. main.rs is a shim over run, so the Python package can offer the same command without a second binary.
corpus
Runs the suite over the geoparquet-testing corpus: every file under data/ must pass Core (and Covering when claimed); every file in bad_data/manifest.json must fail the test that corresponds to its expected_failure.
crs
The two places a GeoParquet 2.0 file states its CRS, reduced to something comparable without a CRS library: an authority:code pair, “undefined”, or “cannot tell”.
source
Where a file comes from: a local path, an in-memory buffer, or an object read with range requests behind a small cache (object stores with the remote feature; a JavaScript fetch callback in the browser build).
spatial
Spatial-order metric of /conf/distribution/spatial-order: skip rate over sample windows, relative to an ideal tiling of the extent into the same number of row groups. Parameters follow gpio check spatial (geoparquet-io PR #774): 20 windows spanning 10 % of each dimension, seed 42, pass at 0.70 of the ideal, verdict withheld below five row groups. The window sequence itself differs from gpio’s (different random generator), so verdicts near the threshold can differ between the two tools.
verify
Regression suite: run every fixture under a directory and compare the failing test ids of each file with a committed manifest ({"relative/path.parquet": {"fail": [...], "skip": [...]}}); every test not listed must pass.
wkb
Minimal ISO WKB decoder: enough to know a value’s type and dimension, its coordinate ranges, its XY vertices and its polygon rings. Rejects EWKB, members of the wrong type or dimension, and counts that the remaining bytes cannot hold. Never allocates from a count.

Functions§

validate
Check a local path or, with the remote feature, an s3://, gs://, az:// or https:// URL. max_rows reads only the first row groups holding that many rows; the report then says the data tests were sampled.
validate_bytes
Check a file already in memory; name only labels the report.