dbn-cli
This crate provides a CLI tool dbn for converting Databento
Binary Encoding (DBN) files to text formats, as well as updating legacy DBZ files to
DBN.
For more information about DBN, read our introduction to DBN.
Installation
Pre-built dbn binaries are attached to every release
of this repository.
Builds are available for the following targets:
- Linux:
x86_64andaarch64, bothgnuandmusl - macOS:
x86_64andaarch64 - Windows:
x86_64andaarch64
Each release ships a dbn-<version>-<target>.tar.gz archive (or .zip on Windows)
together with a .sha256 checksum file.
Download the archive for your platform, verify the checksum, extract the archive,
and move the dbn binary somewhere on your PATH.
Alternatively, if you have a Rust toolchain installed, you can build and install from source with Cargo:
Usage
dbn currently supports CSV and JSON lines as output formats.
By default, dbn outputs the result to standard output for ease of use with
text-based command-line utilities.
Running
will print the header row and the first 5 data rows in some.dbn in CSV format to the console.
Similarly, running
|
Will extract only the high prices from ohlcv-1d.dbn.zst.
dbn works with both uncompressed and Zstandard-compressed DBN files.
You can also save the results directly to another file by running
dbn will create a new file some.csv with the data from some.dbn.zst
formatted as JSON.
When the file name passed --output or -o ends in .json or .csv, you
can omit the --json and --csv flags.
This writes the contents of another.dbn.zst to data.json in CSV format.
By default, dbn will not overwrite an existing file.
To replace the contents of an existing file and allow overwriting files, pass
the -f or --force flag.
Merging DBN files
You can use dbn to merge several DBN files into one, including files of different schemas.
# Merge files split by symbols
# Or by date
The only limitation is they must be from the same dataset.
Splitting DBN files
You can also split one DBN file into several by passing --split-by/-S with a split method and
--output-pattern/-O with a pattern for the output file names.
# By schema, such as from a live data capture
# By time
# By symbol, with the short argument forms
Compressing the output
In addition to reading Zstandard-compressed files, dbn can also write compressed JSON and CSV.
or explicitly
dbn ohlcv-1d.dbn --json --zstd -o ohlcv-1d.json.zst
Converting DBZ files to DBN
DBN is an evolution of DBZ, which required Zstandard. To update an old DBZ file to Zstandard-compressed DBN, run
or pass --dbn to set the output encoding explicitly.
Reading and writing fragments
dbn can also read and write DBN files without the metadata header, these are called "DBN fragments".
Pass the --input-fragment or --input-zstd-fragment flag to read a DBN file without a metadata header.
dbn 120000_121000.mbo.dbn.frag --input-fragment --json
You can also write DBN files without the metadata header with the --fragment or -F flag.
License
Distributed under the Apache 2.0 License.