Crate dmap 

A library for SuperDARN DMAP file I/O.

  

This library also has a Python API using pyo3.

For more information about DMAP files, see RST or pyDARNio.

The main feature of this crate is the Record trait, which defines a valid DMAP record and functions for reading from and writing to byte streams. The SuperDARN file formats IQDAT, RAWACF, FITACF, GRID, MAP, and SND are all supported with structs that implement Record, namely:

• IqdatRecord
• RawacfRecord
• FitacfRecord
• GridRecord
• MapRecord
• SndRecord

Each struct has a list of required and optional fields that it uses to verify the integrity of the record. Only fields listed in the required and optional lists are allowed, no required field can be missing, and each field has an expected primitive type. Additionally, each format has groupings of vector fields which must all share the same shape; e.g. acfd and xcfd in a RAWACF file.

There is also a generic DmapRecord struct which has no knowledge of required or optional fields. When reading from a byte stream, the parsed data will be identical when using both a specific format like RawacfRecord and the generic DmapRecord; however, when writing to a byte stream, the output may differ. Since DmapRecord has no knowledge of the expected primitive type for each field, it defaults to a type that fits the data. For example, the stid field may be saved as an i8 when using DmapRecord instead of an i16 which RawacfRecord specifies it must be.

Each type of record has a specific field ordering hard-coded by this library. This is the order in which fields are written to file, and may not match the ordering of fields generated by RST. This also means that round-trip I/O (i.e. reading a file and writing back out to a new file) is not guaranteed to generate an identical file; however, it is guaranteed that all the information is the same, just not necessarily in the same order.

Examples

Convenience functions for reading from and writing to a file exist to simplify the most common use cases. This is defined by Record::read_file

use dmap::*;
use std::path::PathBuf;

let path = PathBuf::from("tests/test_files/test.rawacf");
let rawacf_data = RawacfRecord::read_file(&path)?;
let unchecked_data = DmapRecord::read_file(&path)?;

assert_eq!(rawacf_data.len(), unchecked_data.len());
assert_eq!(rawacf_data[0].get(&"stid".to_string()), unchecked_data[0].get(&"stid".to_string()));

// Write the records to a file
let out_path = PathBuf::from("tests/test_files/output.rawacf");
RawacfRecord::write_to_file(&rawacf_data, &out_path)?;

You can read from anything that implements the Read trait using the functions exposed by the Record trait. Detection and decompression of bz2 is also conducted automatically.

use dmap::*;
use std::fs::File;
use itertools::izip;

let file = File::open("tests/test_files/test.rawacf.bz2")?;  // `File` implements the `Read` trait
let rawacf_data = RawacfRecord::read_records(file)?;

let uncompressed_data = RawacfRecord::read_file("tests/test_files/test.rawacf")?;
assert_eq!(rawacf_data.len(), uncompressed_data.len());
for (left, right) in izip!(rawacf_data, uncompressed_data) {
    assert_eq!(left, right)
}

Re-exports

pub use crate::error::DmapError;

pub use crate::formats::dmap::DmapRecord;

pub use crate::formats::fitacf::FitacfRecord;

pub use crate::formats::grid::GridRecord;

pub use crate::formats::iqdat::IqdatRecord;

pub use crate::formats::map::MapRecord;

pub use crate::formats::rawacf::RawacfRecord;

pub use crate::formats::snd::SndRecord;

pub use crate::record::Record;

Modules

error

Error type for dmap.

formats

The supported DMAP file formats.

record

Defines the Record trait, which contains the shared behaviour that all DMAP records must have.

types

Low-level data types within DMAP records.

Functions

try_write_dmap

Attempts to convert recs to DmapRecord then append to outfile.

try_write_fitacf

Attempts to convert recs to FitacfRecord then append to outfile.

try_write_grid

Attempts to convert recs to GridRecord then append to outfile.

try_write_iqdat

Attempts to convert recs to IqdatRecord then append to outfile.

try_write_map

Attempts to convert recs to MapRecord then append to outfile.

try_write_rawacf

Attempts to convert recs to RawacfRecord then append to outfile.

try_write_snd

Attempts to convert recs to SndRecord then append to outfile.