Crate dbn

The official crate for working with Databento Binary Encoding (DBN), an extremely fast message encoding and storage format for normalized market data. The DBN specification includes a simple, self-describing metadata header and a fixed set of struct definitions, which enforce a standardized way to normalize market data.

All official Databento client libraries use DBN under the hood, both as a data interchange format and for in-memory representation of data. DBN is also the default encoding for all Databento APIs, including live data streaming, historical data streaming, and batch flat files. For more information about the encoding, read our introduction to DBN.

The crate supports reading and writing DBN files and streams, as well as converting them to other Encodings. It can also be used to update legacy DBZ files to DBN.

This crate provides:

• Decoders for DBN and DBZ (the precursor to DBN), both sync and async, with the async feature flag
• Encoders for CSV, DBN, and JSON, both sync and async, with the async feature flag
• Normalized market data struct definitions corresponding to the different market data schemas offered by Databento
• A wrapper type for holding a reference to a record struct of a dynamic type
• Helper functions and macros for common tasks

Feature flags

• async: enables async decoding and encoding
• python: enables pyo3 bindings
• serde: enables deriving serde traits for types
• trivial_copy: enables deriving the Copy trait for records

Re-exports

• pub use crate::enums::flags;
• pub use crate::enums::rtype;
• pub use crate::enums::Action;
• pub use crate::enums::Compression;
• pub use crate::enums::Encoding;
• pub use crate::enums::InstrumentClass;
• pub use crate::enums::MatchAlgorithm;
• pub use crate::enums::RType;
• pub use crate::enums::SType;
• pub use crate::enums::Schema;
• pub use crate::enums::SecurityUpdateAction;
• pub use crate::enums::Side;
• pub use crate::enums::StatType;
• pub use crate::enums::StatUpdateAction;
• pub use crate::enums::StatusAction;
• pub use crate::enums::StatusReason;
• pub use crate::enums::TradingEvent;
• pub use crate::enums::TriState;
• pub use crate::enums::UserDefinedInstrument;
• pub use crate::enums::VersionUpgradePolicy;
• pub use crate::error::Error;
• pub use crate::error::Result;
• pub use crate::metadata::MappingInterval;
• pub use crate::metadata::Metadata;
• pub use crate::metadata::MetadataBuilder;
• pub use crate::metadata::SymbolMapping;
• pub use crate::publishers::Dataset;
• pub use crate::publishers::Publisher;
• pub use crate::publishers::Venue;
• pub use crate::record::Bbo1MMsg;
• pub use crate::record::Bbo1SMsg;
• pub use crate::record::BidAskPair;
• pub use crate::record::Cbbo1MMsg;
• pub use crate::record::Cbbo1SMsg;
• pub use crate::record::CbboMsg;
• pub use crate::record::ConsolidatedBidAskPair;
• pub use crate::record::ErrorMsg;
• pub use crate::record::HasRType;
• pub use crate::record::ImbalanceMsg;
• pub use crate::record::InstrumentDefMsg;
• pub use crate::record::MboMsg;
• pub use crate::record::Mbp10Msg;
• pub use crate::record::Mbp1Msg;
• pub use crate::record::OhlcvMsg;
• pub use crate::record::Record;
• pub use crate::record::RecordHeader;
• pub use crate::record::RecordMut;
• pub use crate::record::StatMsg;
• pub use crate::record::StatusMsg;
• pub use crate::record::SymbolMappingMsg;
• pub use crate::record::SystemMsg;
• pub use crate::record::TbboMsg;
• pub use crate::record::TradeMsg;
• pub use crate::record::WithTsOut;
• pub use crate::record_ref::RecordRef;
• pub use crate::symbol_map::PitSymbolMap;
• pub use crate::symbol_map::SymbolIndex;
• pub use crate::symbol_map::TsSymbolMap;

Modules

• compat
  Compatibility shims for different DBN versions.
• datasets
  Contains dataset code constants.
• decode
  Decoding DBN and Zstd-compressed DBN files and streams. Decoders implement the DecodeDbn trait.
• encode
  Encoding DBN and Zstd-compressed DBN files and streams. Encoders implement the EncodeDbn trait.
• enums
  Enums used in Databento APIs.
• error
  Types for errors that can occur while working with DBN.
• macros
  Helper macros for working with multiple RTypes, Schemas, and types of records.
• metadata
  Contains Metadata struct which comes at the beginning of any DBN file or stream and MetadataBuilder for creating a Metadata with defaults.
• pretty
  Contains new types for pretty-printing values found in DBN records.
• publishers
  Enumerations for different data sources, venues, and publishers.
• pythonpython
  Python wrappers around dbn functions. These are implemented here instead of in python/ to be able to implement pyo3 traits for DBN types.
• record
  Market data types for encoding different Databento Schemas and conversion functions.
• record_ref
  The RecordRef struct for non-owning references to DBN records.
• symbol_map
  Maps for mapping instrument IDs to human-readable symbols.

Macros

• rtype_async_dispatch
  Specializes a generic async function to all record types and dispatches based rtype.
• rtype_dispatch
  Specializes a generic function to all record types and dispatches based rtype.
• rtype_dispatch_with_ts_out
  Specializes a generic function to all record types wrapped in WithTsOut and dispatches based on the rtype.
• rtype_method_dispatch
  Specializes a generic method to all record types and dispatches based rtype.
• rtype_ts_out_async_dispatch
  Specializes a generic async function to all record types and dispatches based rtype and ts_out.
• rtype_ts_out_async_method_dispatch
  Specializes a generic async method to all record types and dispatches based rtype and ts_out.
• rtype_ts_out_dispatch
  Specializes a generic function to all record types and dispatches based on the rtype and ts_out.
• rtype_ts_out_method_dispatch
  Specializes a generic method to all record types and dispatches based on the rtype and ts_out.
• schema_async_method_dispatch
  Specializes a generic async method to all record types with an associated schema.
• schema_dispatch
  Specializes a generic function to all record types with an associated schema.
• schema_method_dispatch
  Specializes a generic method to all record types with an associated schema.
• schema_ts_out_method_dispatch
  Specializes a generic method to all record types based on the associated type for schema and ts_out.

Enums

• RecordEnum
  An owned DBN record type of flexible type.
• RecordRefEnum
  An immutable reference to a DBN record of flexible type. Unlike RecordRef, this type allows matching.

Constants

• DBN_VERSION
  The current version of the DBN encoding, which is different from the crate version.
• FIXED_PRICE_SCALE
  The denominator of fixed prices in DBN.
• MAX_RECORD_LEN
  The length in bytes of the largest record type.
• SYMBOL_CSTR_LEN
  The length of fixed-length symbol strings.
• UNDEF_ORDER_SIZE
  The sentinel value for an unset or null order quantity.
• UNDEF_PRICE
  The sentinel value for an unset or null price.
• UNDEF_STAT_QUANTITY
  The sentinel value for an unset or null stat quantity.
• UNDEF_TIMESTAMP
  The sentinel value for an unset or null timestamp.