Module status 

Status

The page documents the supported types both from an Arrow and a Rust perspective.

• Arrow data types
• Rust types
  • Native / standard types
  • chrono types
  • jiff types
  • rust_decimal::Decimal
  • bigdecimal::BigDecimal

Arrow data types

• Null
• Boolean
• Int8, Int16, Int32, Int64
• UInt8, UInt16, UInt32, UInt64
• Float16: can be serialized / deserialized from Rust f32
• Float32, Float64
• Timestamp
• Date32
• Date64
• Time32
• Time64
• Duration
• Interval
• Timestamp(Second | Millisecond | Microsecond | Nanosecond, None | Some("UTC")): at the moment only timestamps without timezone or UTC timezone are supported
• Binary
• FixedSizeBinary
• LargeBinary
• Utf8
• LargeUtf8
• List
• LargeList
• FixedSizeList
• Struct
• Union: at the moment only dense unions are supported
• Map: at the moment only unsorted maps are supported
• Dictionary: at the moment only Utf8 and LargeUtf8 as values are supported
• Decimal128(precision, scale): decimals that are serialized to string or float are supported. Decimal128 arrays are always deserialized as string. Values are truncated to the given (precision, scale) range. Values too large for this range will result in a serialization error.
• Decimal256(precision, scale)

Rust types

Native / standard types

• bool
• i8, i16, i32, i64
• u8, u16, u32, u64
• f32, f64
• char: serialized as u32
• Option<T>: if T is supported
• (): serialized as a missing value, Option<()> is always deserialized as None
• struct S{ .. }: if the fields are supported
• Vec<T>: if T is supported. Any type that serializes into a Serde sequence is supported
• HashMap<K, V>, BTreeMap<K, V> and similar map types are supported if K and V are supported
• tuples: tuples or tuple structs are not yet supported. It is planned to map them to struct arrays with numeric field names
• enum ... { }: enums are mapped to union arrays. At the moment options of unions are not supported. Also unions with more than 127 variants are not supported. All types of union variants (unit, newtype, tuple, struct) are supported
• struct S(T): newtype structs are supported, if T is supported

chrono types

chrono::DateTime<Utc>

• is serialized / deserialized as strings
• can be mapped to Utf8, LargeUtf8, Timestamp(.., Some("UTC"))
• from_samples detects
  • LargeUtf8 without configuration
  • Timestamp(Millisecond, Some("UTC")) when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

With chrono::serde::ts_microseconds:

• is serialized / deserialized as i64
• can be mapped to Utf8, LargeUtf8, Timestamp(.., Some("UTC"))
• from_samples and from_type detect Int64

chrono::NaiveDateTime

• is serialized / deserialized as strings
• can be mapped to Utf8, LargeUtf8, Timestamp(.., None)
• from_samples detects
  • LargeUtf8 without configuration
  • Timestamp(Millisecond, None) when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

chrono::NaiveTime

• serialized / deserialized as strings
• can be mapped to Utf8, LargeUtf8, Time32(..) and Time64 arrays
• from_samples detects
  • LargeUtf8 without configuration
  • Time64(Nanosecond) when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

chrono::NaiveDate

• is serialized as Serde strings
• can be mapped to Utf8, LargeUtf8, Date32 arrays
• from_samples detects
  • LargeUtf8 without configuration
  • Date32 when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

chrono::Duration does not support Serde and is therefore not supported

jiff types

jiff::Date

• is serialized as Serde strings
• can me mapped to Utf8, LargeUtf8, Date32
• from_samples detects
  • LargeUtf8 without configuration
  • Date32 when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

jiff::Time

• is serialized as Serde strings
• can me mapped to Utf8, LargeUtf8, Time32(..), Time64(..)
• from_samples detects
  • LargeUtf8 without configuration
  • Time64(Nanosecond) when setitng guess_dates = true
• from_type is not supported, as the type is not self-describing

jiff::DateTime

• is serialized as Serde strings
• can me mapped to Utf8, LargeUtf8, Timestmap(.., None)
• from_samples detects
  • LargeUtf8 without configuration
  • Timestamp(Millisecond, None) when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

jiff::Timestamp

• is serialized as Serde strings
• can me mapped to Utf8, LargeUtf8, Timestamp(.., Some("UTC"))
• from_samples detects
  • LargeUtf8 without configuration
  • Timestamp(Millisecond, Some("UTC")) when setting guess_dates = true
• from_type is not supported, as the type is not self-describing

jiff::Span

• is serialized as Serde strings
• can me mapped to Utf8, LargeUtf8, Duration(..)
• from_samples detects LargeUtf8
• from_type is not supported, as the type is not self-describing

jiff::SignedDuration

Same as jiff::Span

jiff::Zoned

is not supported as there is no clear way of implementation

rust_decimal::Decimal

• for the float and str (de)serialization options when using the Decimal128(..) data type

bigdecimal::BigDecimal

• when using the Decimal128(..) data type