grib-rust

Rust GRIB reader, writer, and shared core primitives for weather and climate data. Default grib-core, grib-reader, and grib-writer builds have no C libraries or build scripts and no unsafe crate code beyond memmap2; optional GRIB2 image and CCSDS packing codecs are behind feature flags. The jpeg2000 feature compiles OpenJPEG behind isolated safe reader/writer interfaces, and the ccsds feature compiles the pinned libaec source through the isolated safe grib-aec interface.

The workspace minimum supported Rust version is 1.87. CI compiles every target with that exact toolchain and tests the supported feature combinations on Linux, macOS, and Windows.

Crates

Reader Usage

use grib_reader::GribFile;

let file = GribFile::open("gfs.grib2")?;
println!("messages: {}", file.message_count());

for msg in file.messages() {
    println!(
        "  {} {:?} {:04}-{:02}-{:02}T{:02}:{:02}:{:02}Z",
        msg.parameter_name(),
        msg.grid_shape(),
        msg.reference_time().year,
        msg.reference_time().month,
        msg.reference_time().day,
        msg.reference_time().hour,
        msg.reference_time().minute,
        msg.reference_time().second,
    );

    if let Some(valid) = msg.valid_time() {
        println!(
            "  valid {:04}-{:02}-{:02}T{:02}:{:02}:{:02}Z",
            valid.year,
            valid.month,
            valid.day,
            valid.hour,
            valid.minute,
            valid.second,
        );
    }
}

let field = file.message(0)?;
let flat = field.read_flat_data_as_f32()?;
println!("decoded values: {}", flat.len());

let mut reused = vec![0.0_f64; field.grid_shape().0 * field.grid_shape().1];
field.decode_into(&mut reused)?;

let data = field.read_data_as_f64()?;
println!("ndarray shape: {:?}", data.shape());

let north_up = field.read_data_north_up_as_f64()?;
println!("north-up shape: {:?}", north_up.shape());

let tolerant = GribFile::builder()
    .strict(false)
    .from_bytes(std::fs::read("mixed.bin")?)?;
println!("recoverable messages: {}", tolerant.message_count());

The builder bounds decoded-field and coordinate-axis allocations by default. Tune max_decoded_points and max_axis_points, or use the without_*_limit methods, when intentionally reading unusually large grids.

Custom GRIB2 local parameter tables can be authored as CSV and supplied as an overlay. The reader checks WMO Code Table 4.2 first for standard parameters, then checks caller entries and the built-in local registry for local-use category or parameter numbers.

use grib_reader::{GribFile, LocalParameterTable};

let table = LocalParameterTable::from_csv_str(
    "center_id,subcenter_id,local_table_version,discipline,category,number,short_name,description\n\
     42,0,1,0,16,196,LREFC,Local composite reflectivity\n",
)?;
let local_parameters = table.entries();

let file = GribFile::builder()
    .local_parameters(&local_parameters)
    .from_bytes(std::fs::read("local-product.grib2")?)?;

Writer Usage

use grib_core::{
    AnalysisOrForecastTemplate, FixedSurface, GridDefinition, Identification, LatLonGrid,
    ProductDefinition, ProductDefinitionTemplate,
};
use grib_writer::{Grib2FieldBuilder, GribWriter, PackingStrategy};

let grid = GridDefinition::LatLon(LatLonGrid {
    ni: 2,
    nj: 2,
    lat_first: 50_000_000,
    lon_first: -120_000_000,
    lat_last: 49_000_000,
    lon_last: -119_000_000,
    di: 1_000_000,
    dj: 1_000_000,
    scanning_mode: 0,
});
let id = Identification {
    center_id: 7,
    subcenter_id: 0,
    master_table_version: 35,
    local_table_version: 1,
    significance_of_reference_time: 1,
    reference_year: 2026,
    reference_month: 3,
    reference_day: 20,
    reference_hour: 12,
    reference_minute: 0,
    reference_second: 0,
    production_status: 0,
    processed_data_type: 1,
};
let product = ProductDefinition {
    parameter_category: 0,
    parameter_number: 0,
    template: ProductDefinitionTemplate::AnalysisOrForecast(AnalysisOrForecastTemplate {
        type_of_generating_process: 2,
        background_generating_process_identifier: 0,
        generating_process_identifier: 0,
        hours_after_data_cutoff: Some(0),
        minutes_after_data_cutoff: Some(0),
        forecast_time_unit: 1,
        forecast_time: 6,
        first_surface: Some(FixedSurface::with_value(103, 0, 850)),
        second_surface: None,
    }),
};

let values = [1.0, 2.0, f64::NAN, 4.0];
let field = Grib2FieldBuilder::new()
    .identification(id)
    .grid(grid)
    .product(product)
    .packing(PackingStrategy::SimpleAuto { decimal_scale: 0 })
    .values(&values)
    .build()?;

let mut bytes = Vec::new();
GribWriter::new(&mut bytes).write_grib2_message([field])?;

Supported Now

• GRIB1 and GRIB2 message scanning with "GRIB"/"7777" boundary detection
• Logical field indexing for multi-field GRIB2 messages
• GRIB2 Section 2 access and same-message bitmap reuse through indicator 254
• Regular latitude/longitude grids for GRIB1 and GRIB2
• Reader GRIB2 rotated latitude/longitude template 3.1 and regular Gaussian template 3.40, with explicit typed rejection of quasi-regular grids
• Reader GRIB2 Mercator grid template 3.10, polar stereographic grid template 3.20, Lambert conformal grid template 3.30, and Albers equal-area grid template 3.31 metadata, projected coordinate offsets, and flat data decode
• Reader simple packing for GRIB1 and GRIB2
• Reader framing for ECMWF-era GRIB1 messages larger than the 24-bit standard length limit
• Reader GRIB1 row-by-row, constant-width, and variable-width second-order grid-point packing
• Reader GRIB1 predefined bitmaps via caller-supplied center-defined bitmap tables
• GRIB2 complex packing with general or row-by-row group splitting, including spatial differencing
• Reader and writer GRIB2 IEEE template 5.4 with binary32, binary64, and correctly rounded binary128 conversion
• Reader GRIB2 run-length template 5.200 with scaled level values and embedded missing-value runs
• Feature-gated reader and writer GRIB2 JPEG2000 template 5.40 and PNG template 5.41 packing
• Feature-gated reader and writer GRIB2 CCSDS/AEC template 5.42 packing through the bundled libaec interface
• WMO parameter table lookups (Code Table 4.2) plus center/subcenter/local-table-aware GRIB2 local parameter entries and CSV authoring helpers
• Typed metadata access for reference time, parameter identity, product metadata, grid geometry, and lat/lon coordinates
• Explicit north-up decode methods in addition to reader order that preserves the encoded i/j scan directions
• Reader GRIB2 product definition templates 4.0, 4.1, 4.2, 4.5, 4.6, 4.8, 4.9, 4.10, 4.11, 4.12, and 4.15
• Writer GRIB2 product definition templates 4.0, 4.1, 4.2, 4.5, 4.6, 4.8, 4.9, 4.10, 4.11, 4.12, and 4.15
• Forecast valid-time helpers for supported fixed-width GRIB1/GRIB2 time units
• GribFile::builder() for strict or tolerant scanning and allocation limits
• Bitmap and embedded complex-packing missing values surfaced as NaN, with primary and secondary substitute metadata available to callers
• Parallel field decoding via Rayon
• Output: caller-owned &mut [f32]/&mut [f64], flat Vec<f32>/Vec<f64>, or ndarray::ArrayD<f32>/ArrayD<f64>
• Memory-mapped files, owned byte buffers, or arbitrary Read streams
• Feature-gated reader GRIB2 CCSDS/AEC template 5.42 decode using the bundled, security-fixed libaec 1.1.7 codec
• Writer GRIB2 regular lat/lon fields with simple packing template 5.0, general or row-by-row complex packing template 5.2, and spatial differencing template 5.3
• Writer GRIB2 rotated latitude/longitude grid template 3.1 and regular Gaussian grid template 3.40 fields
• Writer GRIB2 Mercator grid template 3.10, polar stereographic grid template 3.20, Lambert conformal grid template 3.30, and Albers equal-area grid template 3.31 fields
• Feature-gated writer GRIB2 JPEG2000 template 5.40 and PNG template 5.41 packed data encode
• Feature-gated writer GRIB2 CCSDS/AEC template 5.42 encode with automatic bit-width selection, bitmap support, and configurable codec parameters
• Writer GRIB2 bitmap section generation from explicit masks or NaN values
• Writer single-message multi-field GRIB2 output with reused grid sections and bitmap indicator 254 for repeated bitmaps
• Writer GRIB1 regular lat/lon fields with simple packing and optional explicit or predefined bitmap section

Not Yet Supported

• Remaining GRIB2 grid templates beyond 3.0, 3.1, 3.10, 3.20, 3.30, 3.31, and 3.40
• Remaining reader GRIB2 product definition templates beyond 4.0, 4.1, 4.2, 4.5, 4.6, 4.8, 4.9, 4.10, 4.11, 4.12, and 4.15
• Remaining writer GRIB2 product definition templates beyond 4.0, 4.1, 4.2, 4.5, 4.6, 4.8, 4.9, 4.10, 4.11, 4.12, and 4.15 Unsupported decode and encode operations fail explicitly with typed errors; unknown GRIB2 product templates remain indexable with their raw bytes preserved. Calendar-dependent forecast units such as months and years are exposed through raw metadata but currently return None from valid_time().

API Compatibility

The workspace is pre-1.0. GridDefinition and DataRepresentation are intentionally #[non_exhaustive] because WMO can add templates and producers can use center-specific local templates. Downstream code should prefer query helpers for common behavior or include a wildcard arm when matching specific template families.

Feature flags

Testing

cargo fmt --all --check
cargo clippy --all-targets --all-features --locked -- -D warnings
cargo run -p grib-reader --example sync_corpus --locked
git diff --exit-code
cargo test --all-features --locked
cargo test --no-default-features --locked
cargo +1.87.0 check --workspace --all-targets --all-features --locked
cargo doc --workspace --all-features --no-deps --locked
cargo deny --locked check
cargo deny --manifest-path grib-reader/fuzz/Cargo.toml --locked check -A unmatched-skip
cargo +1.91.0 llvm-cov --workspace --all-features --locked --fail-under-lines 85
./scripts/run-reference-parity.sh
cargo check --manifest-path grib-reader/fuzz/Cargo.toml --bins --locked
cargo clippy --manifest-path grib-reader/fuzz/Cargo.toml --bins --locked -- -D warnings
./scripts/run-fuzz-smoke.sh 60
./scripts/verify-packages.sh

The coverage command requires cargo-llvm-cov 0.8.6 and the dependency-policy commands require cargo-deny 0.19.4. Fuzzing requires a nightly toolchain and cargo-fuzz 0.13.1. CI pins the fuzzer toolchain to nightly-2026-07-11.

verify-packages.sh assembles each normalized crate archive and compiles all of those archives together through local registry patches. This verifies unpublished workspace dependencies without relying on stable Cargo's broken temporary-registry workspace verifier.

The Reference Compat workflow runs the Dockerized ecCodes parity suite for pull requests, main/master pushes, release tags, and a weekly scheduled check. Its image compiles SHA-256-verified ecCodes 2.47.0 and libaec 1.1.7 source archives. The reader and writer parity tests stay #[ignore] for normal local cargo test because they require the ecCodes helper; CI invokes them through:

./scripts/run-reference-parity.sh

The parity and benchmark containers are uniquely named, removed after each run, and default to two CPUs, four GiB of memory, and two Cargo build jobs. Set ECCODES_DOCKER_CPUS, ECCODES_DOCKER_MEMORY, or CARGO_BUILD_JOBS to adjust those limits explicitly. On non-Linux hosts, container artifacts use container-local storage so native Linux objects cannot collide with the host's target/; set ECCODES_CARGO_TARGET_DIR to override that location.

Release Checklist

git switch main
git pull --ff-only
git merge <release-branch>

./scripts/verify-packages.sh
cargo publish -p grib-aec --dry-run --locked
cargo publish -p grib-aec --locked
cargo publish -p grib-core --dry-run --locked
cargo publish -p grib-core --locked
cargo publish -p grib-reader --dry-run --locked
cargo publish -p grib-reader --locked
cargo publish -p grib-writer --dry-run --locked
cargo publish -p grib-writer --locked

git tag -a v<version> -m "v<version>"
git push origin main
git push origin v<version>

Corpus And Fuzzing

• Bootstrap corpus samples live in grib-reader/tests/corpus/bootstrap/
• Real interoperability samples belong in grib-reader/tests/corpus/interop/samples/
• Regenerate the bootstrap and fuzz seed corpora with cargo run -p grib-reader --example sync_corpus
• Fuzzer entry points and usage notes live in grib-reader/fuzz/README.md

Reference Checks

• ./scripts/run-reference-parity.sh runs the Dockerized ecCodes parity suite; it is mandatory in the Reference Compat workflow for pull requests, main/master pushes, release tags, and weekly scheduled verification.
• grib-writer has a versioned dev-dependency on grib-reader for local validation tests and benchmarks, so dry-run and publish it after grib-reader v0.8.0 is visible in the crates.io index.
• For reference comparisons and current benchmark results against ecCodes, see docs/benchmark-report.md. Re-run the benchmark scripts after corpus changes before using those numbers as current throughput claims.

License

MIT OR Apache-2.0