Crate dbn 

The official crate for working with Databento Binary Encoding (DBN), an extremely fast message encoding and storage format for normalized market data. The DBN specification includes a simple, self-describing metadata header and a fixed set of struct definitions, which enforce a standardized way to normalize market data.

All official Databento client libraries use DBN under the hood, both as a data interchange format and for in-memory representation of data. DBN is also the default encoding for all Databento APIs, including live data streaming, historical data streaming, and batch flat files. For more information about the encoding, read our introduction to DBN.

The crate supports reading and writing DBN files and streams, as well as converting them to other Encodings. It can also be used to update legacy DBZ files to DBN.

This crate provides:

• Decoders for DBN and DBZ (the precursor to DBN), both sync and async, with the async feature flag
• Encoders for CSV, DBN, and JSON, both sync and async, with the async feature flag
• Normalized market data struct definitions corresponding to the different market data schemas offered by Databento
• A wrapper type for holding a reference to a record struct of a dynamic type
• Helper functions and macros for common tasks

Feature flags

• async: enables async decoding and encoding
• python: enables pyo3 bindings
• serde: enables deriving serde traits for types
• trivial_copy: enables deriving the Copy trait for records

Modules

compat

Compatibility shims for different DBN versions.

decode

Decoding DBN and Zstd-compressed DBN files and streams. Sync decoders implement

encode

Encoding DBN and Zstd-compressed DBN files and streams. Encoders implement the EncodeDbn trait.

enums

Enums used in Databento APIs.

error

Types for errors that can occur while working with DBN.

flags

Bit set flags used in Databento market data.

macros

Helper macros for working with multiple RTypes, Schemas, and types of records.

metadata

Contains Metadata struct which comes at the beginning of any DBN file or stream and MetadataBuilder for creating a Metadata with defaults.

pretty

Contains new types for pretty-printing timestamp and prices found in DBN records.

publishers

Enumerations for different data sources, venues, and publishers.

pythonpython

Python wrappers around dbn functions. These are implemented here instead of in python/ to be able to implement pyo3 traits for DBN types.

record

Market data types for encoding different Databento Schemas in the most recent DBN version, as well as conversion functions.

record_ref

The RecordRef struct for non-owning dynamically-typed references to DBN records.

rtype

Record types, possible values for RecordHeader::rtype.

symbol_map

Maps for mapping instrument IDs to human-readable symbols.

v1

Record data types for encoding different Databento Schemas in DBN version 1.

v2

Record data types for encoding different Databento Schemas in DBN version 2.

v3

Record data types for encoding different Databento Schemas in the upcoming DBN version 3.

Macros

rtype_async_dispatchDeprecated

Specializes a generic async function to all record types and dispatches based rtype.

rtype_dispatch

Dispatches to a generic function or method based on $rtype and optionally $ts_out.

rtype_dispatch_with_ts_outDeprecated

Specializes a generic function to all record types wrapped in WithTsOut and dispatches based on the rtype.

rtype_method_dispatchDeprecated

Specializes a generic method to all record types and dispatches based rtype.

rtype_ts_out_async_dispatchDeprecated

Specializes a generic async function to all record types and dispatches based rtype and ts_out.

rtype_ts_out_async_method_dispatchDeprecated

Specializes a generic async method to all record types and dispatches based rtype and ts_out.

rtype_ts_out_dispatchDeprecated

Specializes a generic function to all record types and dispatches based on the rtype and ts_out.

rtype_ts_out_method_dispatchDeprecated

Specializes a generic method to all record types and dispatches based on the rtype and ts_out.

schema_async_method_dispatchDeprecated

Specializes a generic async method to all record types with an associated schema.

schema_dispatch

Dispatches to a generic function or method based on $schema and optionally $ts_out.

schema_method_dispatchDeprecated

Specializes a generic method to all record types with an associated schema.

schema_ts_out_method_dispatchDeprecated

Specializes a generic method to all record types based on the associated type for schema and ts_out.

Structs

BboMsg

Subsampled market by price with a known book depth of 1. The record of the Bbo1S and Bbo1M schemas.

BidAskPair

A price level.

CbboMsg

Subsampled consolidated market by price with a known book depth of 1. The record of the Cbbo1S and Cbbo1M schemas.

Cmbp1Msg

Consolidated market-by-price implementation with a known book depth of 1. The record of the Cmbp1 schema.

ConsolidatedBidAskPair

A price level consolidated from multiple venues.

DbnVersion

New type for validating DBN versions.

ErrorMsg

An error message from the Databento Live Subscription Gateway (LSG).

FlagSet

A transparent wrapper around the bit field used in several DBN record types, namely MboMsg and record types derived from it.

ImbalanceMsg

An auction imbalance message.

InstrumentDefMsg

A definition of an instrument. The record of the Definition schema.

MappingInterval

The resolved symbol for a date range.

MboMsg

A market-by-order (MBO) tick message. The record of the Mbo schema.

Mbp1Msg

Market-by-price implementation with a known book depth of 1. The record of the Mbp1 schema.

Mbp10Msg

Market-by-price implementation with a known book depth of 10. The record of the Mbp10 schema.

Metadata

Information about the data contained in a DBN file or stream. DBN requires the Metadata to be included at the start of the encoded data.

MetadataBuilder

Helper for constructing Metadata structs with defaults.

OhlcvMsg

Open, high, low, close, and volume. The record of the following schemas:

PitSymbolMap

A point-in-time symbol map. Useful for working with live symbology or a historical request over a single day or other situations where the symbol mappings are known not to change.

RecordHeader

Common data for all Databento records. Always found at the beginning of a record struct.

RecordRef

A wrapper around a non-owning immutable reference to a DBN record. This wrapper allows for mixing of record types and schemas and runtime record polymorphism.

StatMsg

A statistics message. A catchall for various data disseminated by publishers. The stat_type indicates the statistic contained in the message.

StatusMsg

A trading status update message. The record of the Status schema.

SymbolMapping

A raw symbol and its symbol mappings for different time ranges within the query range.

SymbolMappingMsg

A symbol mapping message from the live API which maps a symbol from one SType to another.

SystemMsg

A non-error message from the Databento Live Subscription Gateway (LSG). Also used for heartbeating.

TradeMsg

Market-by-price implementation with a book depth of 0. Equivalent to MBP-0. The record of the Trades schema.

TsSymbolMap

A timeseries symbol map. Generally useful for working with historical data and is commonly built from a Metadata object via Self::from_metadata().

WithTsOut

Wrapper object for records that include the live gateway send timestamp (ts_out).

Enums

Action

An order event or order book operation.

Compression

A compression format or none if uncompressed.

Dataset

A source of data.

Encoding

A data encoding format.

Error

An error that can occur while processing DBN data.

ErrorCode

An error code from the live subscription gateway.

InstrumentClass

The class of instrument.

MatchAlgorithm

The type of matching algorithm used for the instrument at the exchange.

Publisher

A specific Venue from a specific data source.

RType

A record type, i.e. a sentinel for different types implementing HasRType.

RecordEnum

An owned DBN record type of flexible type. Unlike RecordRef, this type allows matching.

RecordRefEnum

An immutable reference to a DBN record of flexible type. Unlike RecordRef, this type allows matching.

SType

A symbology type. Refer to the symbology documentation for more information.

Schema

A data record schema.

SecurityUpdateAction

The type of InstrumentDefMsg update.

Side

A side of the market. The side of the market for resting orders, or the side of the aggressor for trades.

StatType

The type of statistic contained in a StatMsg.

StatUpdateAction

The type of StatMsg update.

StatusAction

The primary enum for the type of StatusMsg update.

StatusReason

The secondary enum for a StatusMsg update, explains the cause of a halt or other change in action.

SystemCode

A SystemMsg code indicating the type of message from the live subscription gateway.

TradingEvent

Further information about a status update.

TriState

An enum for representing unknown, true, or false values. Equivalent to Option<bool> but with a human-readable repr.

UserDefinedInstrument

Whether the instrument is user-defined.

Venue

A trading execution venue.

VersionUpgradePolicy

How to handle decoding DBN data from other versions.

Constants

ASSET_CSTR_LEN

The length of the fixed-length asset string.

DBN_VERSION

The current version of the DBN encoding, which is different from the crate version.

FIXED_PRICE_SCALE

The denominator of fixed prices in DBN.

MAX_RECORD_LEN

The length in bytes of the largest record type.

SYMBOL_CSTR_LEN

The length of fixed-length symbol strings.

UNDEF_ORDER_SIZE

The sentinel value for an unset or null order quantity.

UNDEF_PRICE

The sentinel value for an unset or null price.

UNDEF_STAT_QUANTITY

The sentinel value for an unset or null stat quantity.

UNDEF_TIMESTAMP

The sentinel value for an unset or null timestamp.

Traits

HasRType

An extension of the Record trait for types with a static RType. Used for determining if a rtype matches a type.

Record

Used for polymorphism around types all beginning with a RecordHeader where rtype is the discriminant used for indicating the type of record.

RecordMut

Used for polymorphism around mutable types beginning with a RecordHeader.

SymbolIndex

Used for retrieving a symbol mapping for a DBN record.

Type Aliases

Bbo1MMsg

The record of the Bbo1M schema.

Bbo1SMsg

The record of the Bbo1S schema.

Cbbo1MMsg

The record of the Cbbo1M schema.

Cbbo1SMsg

The record of the Cbbo1S schema.

Result

An alias for a Result with dbn::Error as the error type.

TbboMsg

The record of the Tbbo schema.

TcbboMsg

The record of the Tcbbo schema.