dta

A pure Rust library for reading and writing Stata data formats.

This crate covers two related-but-distinct file formats:

• DTA — the binary format Stata uses to persist datasets. Every released version is supported (102 through 119), including XML-framed releases (117+), tagged missing values, value-label sets, and long-string (strL) storage.
• DCT — Stata's plain-text dictionary format, used to describe fixed-width or free-format .dat/.raw data files. Common when importing into Stata from external systems.

Both formats sit under Stata, still the dominant interchange format in academic economics, public health, and social science research — often the only format that downstream collaborators can actually open.

The DTA API is built around a typestate chain, so you can't accidentally write a schema before a header, or read data bytes before the schema has been parsed. Each phase borrows the underlying I/O handle and hands it to the next phase, keeping the pipeline zero-copy where possible.

The DCT API is a two-step builder — parse the dictionary into a Schema, then pair the schema with a data source to read records.

Usage — DTA

Reading a file

The reader walks through the sections of a DTA file in order. Each phase returns the next phase when you're done with it.

use dta::stata::dta::dta_reader::DtaReader;
use dta::stata::dta::dta_error::Result;

fn read_stata(path: &str) -> Result<()> {
    // HeaderReader: parse the file header (release, byte order, dataset
    // label, timestamp, K, N).
    let header_reader = DtaReader::new().from_path(path)?;

    // SchemaReader: parse variable types, names, formats, labels, and
    // value-label associations.
    let schema_reader = header_reader.read_header()?;
    let header = schema_reader.header().clone();
    println!("Release {}, {} variables, {} observations",
        header.release(), header.variable_count(), header.observation_count());

    // CharacteristicReader: iterate (or skip) the characteristics /
    // expansion-fields section.
    let mut characteristic_reader = schema_reader.read_schema()?;
    characteristic_reader.skip_to_end()?;

    // RecordReader: iterate observation rows.
    let mut record_reader = characteristic_reader.into_record_reader()?;
    let schema = record_reader.schema().clone();
    while let Some(record) = record_reader.read_record()? {
        for (variable, value) in schema.variables().iter().zip(record.values()) {
            println!("{}: {:?}", variable.name(), value);
        }
    }

    // LongStringReader: read `strL` entries referenced from the data
    // section (117+ only — transitions silently on earlier releases).
    let mut long_string_reader = record_reader.into_long_string_reader()?;
    long_string_reader.skip_to_end()?;

    // ValueLabelReader: iterate value-label sets.
    let mut value_label_reader = long_string_reader.into_value_label_reader()?;
    while let Some(set) = value_label_reader.read_value_label_set()? {
        println!("label set: {}", set.name());
    }

    Ok(())
}

Each reader exposes header() and schema() accessors, so you can inspect metadata without threading them through yourself.

Lazy records

RecordReader::read_record() eagerly decodes every cell in the row. If you only need a subset of columns — or you want to defer parsing until you know whether you care — use read_lazy_record() instead. LazyRecord hands back raw byte slices that you decode one cell at a time.

while let Some(lazy) = record_reader.read_lazy_record()? {
    // Decode only the columns you need.
    let id = lazy.value(0)?;
    let name = lazy.value(5)?;
}

Writing a file

The writer is the mirror image of the reader: a typestate chain that locks in the ordering of the header, schema, characteristics, data, long strings, and value labels. The <map> offsets and the header's K/N placeholders are patched in place as each section closes, which is why the writer requires Write + Seek.

use dta::stata::dta::byte_order::ByteOrder;
use dta::stata::dta::dta_writer::DtaWriter;
use dta::stata::dta::header::Header;
use dta::stata::dta::release::Release;
use dta::stata::dta::schema::Schema;
use dta::stata::dta::value::Value;
use dta::stata::dta::variable::Variable;
use dta::stata::dta::variable_type::VariableType;
use dta::stata::stata_double::StataDouble;
use dta::stata::stata_long::StataLong;
use dta::stata::dta::dta_error::Result;

fn write_stata(path: &str) -> Result<()> {
    let header = Header::builder(Release::V118, ByteOrder::LittleEndian)
        .dataset_label("example dataset")
        .build();

    let schema = Schema::builder()
        .add_variable(Variable::builder(VariableType::Long, "id").format("%12.0g"))
        .add_variable(Variable::builder(VariableType::Double, "price").format("%10.2f"))
        .add_variable(Variable::builder(VariableType::FixedString(32), "name").format("%32s"))
        .build()?;

    let characteristic_writer = DtaWriter::new()
        .from_path(path)?
        .write_header(header)?
        .write_schema(schema)?;

    // Skip characteristics (they're optional) and begin writing rows.
    let mut record_writer = characteristic_writer.into_record_writer()?;
    record_writer.write_record(&[
        Value::Long(StataLong::Present(1)),
        Value::Double(StataDouble::Present(19.99)),
        Value::String("widget"),
    ])?;
    record_writer.write_record(&[
        Value::Long(StataLong::Present(2)),
        Value::Double(StataDouble::Missing),
        Value::String("gadget"),
    ])?;

    // Transition through the remaining sections (strLs and value labels
    // are optional) and close the file.
    record_writer
        .into_long_string_writer()?
        .into_value_label_writer()?
        .finish()?;

    Ok(())
}

RecordWriter::write_record validates arity and per-column types against the schema, so a mismatch is caught at the call site rather than producing a malformed file.

Usage — DCT

A .dct dictionary describes the schema of a plain-text data file: where each variable starts, how wide it is, what type it is, what its name is, and (optionally) a label. The associated data may live in a separate .dat/.raw file (the dictionary's using clause) or inline after the closing }.

use dta::stata::dct::dct_reader::DctReader;
use dta::stata::dct::dct_source::DctSource;
use dta::stata::dct::dct_error::Result;

fn read_dct(dict_path: &str, data_path: &str) -> Result<()> {
    // Step 1: parse the dictionary. The result tells you whether the
    // data file is embedded after the closing `}` or external.
    let source = DctSource::options().from_path(dict_path)?;

    let mut reader = match source {
        DctSource::External(schema) => {
            // Step 2a: open the data file separately.
            DctReader::options(schema).from_path(data_path)?
        }
        DctSource::Embedded { schema, reader } => {
            // Step 2b: data was inlined; `reader` is positioned at it.
            // Pair it with the schema through the same options
            // builder so reader-side knobs apply to either path.
            DctReader::options(schema).from_reader(reader)
        }
    };

    // Capture column names up front: the lending pattern means
    // `record` borrows `reader` exclusively, so `reader.schema()`
    // can't be called inside the loop body.
    let column_names: Vec<String> = reader
        .schema()
        .columns()
        .iter()
        .map(|c| c.name().to_string())
        .collect();

    // Step 3: iterate records.
    while let Some(record) = reader.read_record()? {
        for (name, value) in column_names.iter().zip(record.values()) {
            println!("{}: {:?}", name, value);
        }
    }

    // Per-record warnings (blank short-line fields treated as
    // missing, integers auto-promoted to a wider type) accumulate
    // on the reader and are cleared at the start of each call.
    for warning in reader.warnings() {
        eprintln!("warning: {warning}");
    }

    Ok(())
}

Lazy DCT records

If you only need a subset of columns per row, read_lazy_record() defers value decoding until you ask for it. Useful for skipping the parse work on the columns you don't care about.

while let Some(lazy) = reader.read_lazy_record()? {
    let id = lazy.value(0)?;
    let name = lazy.value(5)?;
    // ...other columns are never decoded
}

What's supported

The DCT parser supports the directives you'll see in real-world dictionary files:

• _column(#) — absolute byte position (1-based; stored 0-based internally)
• _skip(#) and bare _skip — per-variable column-pointer modifier (relative to the running cursor; combines with _column(#) when both are present)
• _newline — multi-line observations; _column(#) references restart at 1 after each
• lrecl(#) and firstlineoffile(#) directives
• All Stata storage types (byte, int, long, float, double, str, str#)
• All read formats (%w.df, %w.dg, %w.de, %wf, %f, %ws, %s)
• Both single-line and multi-line records
• Both fixed-width and free-format reads
• The using FILE clause and the optional infile prefix
• Comments (* lines)

Per-record diagnostics surface as DctWarning values on the reader: blank short-line fields treated as system missing, integer values auto-promoted to a wider storage type when they fall in the missing-marker range, declared using paths the library doesn't act on, and unrecognized directives.

A note on real-world coverage: the rarer directives (lrecl(#), firstlineoffile(#), _newline, free-format reads) are implemented against Stata's documentation but I haven't found a .dct file in the wild that uses them — every public dictionary I sampled had been cleaned up to the common subset. If you hit a bug with one of these, please file an issue with a small representative .dct file and the matching data file so I can reproduce.

If you don't intend to inspect warnings() — for example, in a tight read loop where you only care about values — disable warning collection on the options builder so the reader can skip building the diagnostic structs entirely:

let mut reader = DctReader::options(schema)
    .record_warnings(false)
    .from_path(data_path)?;

Feature Flags

chrono — date/time helpers

Disabled by default. Stata stores dates and timestamps as plain numeric values whose meaning is encoded in the variable's display format (%td, %tc, %tm, …). Enabling chrono adds typed conversions to chrono types so you don't have to track the 1960 epoch, the milliseconds-vs-days distinction, or the legacy %d alias yourself.

[dependencies]

dta = { version = "0.6", features = ["chrono"] }

chrono = "0.4"

The module is layered. The lower layers ship without any time-crate dependency and are always available — only the typed adapters live behind the feature flag:

• dta::stata::temporal::conversion — pure numeric helpers (td_days_to_unix_days, tc_millis_to_unix_millis, plus (year, sub-period) decomposers for %tw/%tm/%tq/%th). Useful as-is for consumers using jiff, time, or raw Unix timestamps.
• dta::stata::temporal::TemporalKind::from_format — classify a Stata format string into Date, DateTime, Year, Week, etc. Recognizes the eight %t* prefixes plus the legacy %d alias, and ignores display suffixes (%tdCCYY-NN-DD classifies the same as bare %td).
• dta::stata::temporal::chrono_adapter (feature-gated) — NaiveDate / NaiveDateTime adapters, Value-aware helpers that handle storage-type widening and Stata missing-value sentinels, and a temporal_from_value(&Value, &str) -> Option<StataTemporal> dispatcher that picks the right path from a column's format string.

use dta::stata::dta::dta_reader::DtaReader;
use dta::stata::dta::dta_error::Result;
use dta::stata::temporal::chrono_adapter::{StataTemporal, temporal_from_value};

fn dump_temporal_columns(path: &str) -> Result<()> {
    let mut reader = DtaReader::new()
        .from_path(path)?
        .read_header()?
        .read_schema()?;
    reader.skip_to_end()?;
    let mut record_reader = reader.into_record_reader()?;
    let schema = record_reader.schema().clone();

    while let Some(record) = record_reader.read_record()? {
        for (variable, value) in schema.variables().iter().zip(record.values()) {
            match temporal_from_value(value, variable.format()) {
                Some(StataTemporal::Date(d)) => println!("{}: {}", variable.name(), d),
                Some(StataTemporal::DateTime(dt)) => println!("{}: {}", variable.name(), dt),
                Some(StataTemporal::Year(y)) => println!("{}: {}", variable.name(), y),
                Some(StataTemporal::YearMonth { year, month }) => {
                    println!("{}: {}-{:02}", variable.name(), year, month);
                }
                Some(_) | None => {} // quarter / half / week / non-temporal
            }
        }
    }
    Ok(())
}

temporal_from_value returns None for non-temporal columns, Stata missing-value sentinels (. and .a–.z), storage/format mismatches (e.g., a %tc cell stored as Long), and the leap-second-adjusted %tC format (chrono can't model leap seconds; drop to the Layer 1 tc_millis_to_unix_millis helper if you need to make a policy decision yourself).

The feature also enables StataTimestamp::to_naive_date_time, which converts the file-header timestamp directly to a NaiveDateTime.

tokio — async I/O

Disabled by default. Enable it for async reading and writing with tokio.

[dependencies]

dta = { version = "0.6", features = ["tokio"] }

tokio = { version = "1", features = ["fs", "io-util", "rt", "macros"] }

This unlocks async terminals on every options builder:

• DTA: DtaReader::from_tokio_* returns an AsyncHeaderReader that mirrors the sync chain with .await at each step; DtaWriter::from_tokio_* returns an AsyncHeaderWriter. The async writer requires AsyncWrite + AsyncSeek + Unpin so that the XML <map> and header placeholders can still be patched in place.
• DCT: DctSource::options().from_tokio_* returns a DctSource<R>; DctReader::options(schema).from_tokio_* returns an AsyncDctReader<R> whose read_record / read_lazy_record methods are .awaited.

let mut reader = DctReader::options(schema).from_tokio_path(path).await?;
while let Some(record) = reader.read_record().await? {
    // ...
}

The sync and async DCT paths share the same pure parsing state — only the I/O loop differs.

use dta::stata::dta::dta_reader::DtaReader;
use dta::stata::dta::dta_error::Result;

async fn read_async(path: &str) -> Result<()> {
    let mut characteristic_reader = DtaReader::new()
        .from_tokio_path(path)
        .await?
        .read_header()
        .await?
        .read_schema()
        .await?;
    characteristic_reader.skip_to_end().await?;

    let mut record_reader = characteristic_reader.into_record_reader().await?;
    while let Some(_record) = record_reader.read_record().await? {
        // ...
    }

    Ok(())
}

Character encoding

encoding_rs is a hard dependency, not a feature flag. The reader and writer pick an encoding from the release number by default: Windows-1252 for pre-118 files and UTF-8 for 118+. If you need to override that — for example, reading a pre-118 file produced by a locale that wasn't Windows-1252 — pass an explicit encoding:

use dta::stata::dta::dta_reader::DtaReader;

let reader = DtaReader::new()
    .encoding(encoding_rs::SHIFT_JIS)
    .from_path("legacy.dta")?;

Benchmarks

The project ships Criterion benchmarks that exercise sync and async read/write throughput against a synthetic panel-style file:

# Sync benchmarks only (no optional features)

cargo bench


# Include the async (tokio) benchmarks

cargo bench --all-features

Results are saved to target/criterion/ with HTML reports. To run a single benchmark by name:

cargo bench --all-features -- read_from_cursor

The first run generates benches/data/bench_large.dta (~100K rows) and caches it for subsequent runs.

Profiling

The repository includes a profiling workflow built on samply. One-time setup:

# Install samply (Linux / macOS / Windows)

cargo install --locked samply


# On Linux, samply needs perf_event_open access:

echo 1 | sudo tee /proc/sys/kernel/perf_event_paranoid

Then run the profiling script:

# Default: 1M records, top 10 functions

./profile.sh


# Customize record count and report depth

./profile.sh --records 500000 --top 20

On Windows, use .\profile.ps1 with the same options (-Records, -Top). samply uses ETW on Windows, so the script must run from an elevated PowerShell window.

The script builds optimized binaries with debug symbols, records separate samply profiles for the sync write, sync read, async write, and async read phases, and prints a table of the hottest functions from this crate by inclusive and self time.

About

This is a passion project that I maintain on my own time. I care deeply about its quality and want it to be genuinely useful, but I also want to keep it fun and sustainable. To that end:

• Bug reports are always welcome. Please file issues for anything that isn't working correctly.
• Feature requests are best expressed as pull requests. I'm much more likely to engage with a well-crafted PR than a request for new work.
• Timelines are my own. I'll get to things when I can, and I may close issues or PRs that don't align with the project's direction – nothing personal.

If you find this library valuable, the best way to support it is to contribute or share it with others.