Parquet Derive
A crate for deriving RecordWriter
for arbitrary, simple structs. This does not generate writers for arbitrarily nested
structures. It only works for primitives and a few generic structures and
various levels of reference. Please see features checklist for what is currently
supported.
Derive also has some support for the chrono time library. You must must enable the chrono
feature to get this support.
Usage
Add this to your Cargo.toml:
[]
= "23.0.0"
= "23.0.0"
and this to your crate root:
extern crate parquet;
extern crate parquet_derive;
Example usage of deriving a RecordWriter
for your struct:
use parquet;
use RecordWriter;
// Initialize your parquet file
let mut writer = new.unwrap;
let mut row_group = writer.next_row_group.unwrap;
// Build up your records
let chunks = vec!;
// The derived `RecordWriter` takes over here
.write_to_row_group;
writer.close_row_group.unwrap;
writer.close.unwrap;
Features
- Support writing
String
,&str
,bool
,i32
,f32
,f64
,Vec<u8>
- Support writing dictionaries
- Support writing logical types like timestamp
- Derive definition_levels for
Option
- Derive definition levels for nested structures
- Derive writing tuple struct
- Derive writing
tuple
container types
Requirements
- Same as
parquet-rs
Test
Testing a *_derive
crate requires an intermediate crate. Go to parquet_derive_test
and run cargo test
for
unit tests.
Docs
To build documentation, run cargo doc --no-deps
.
To compile and view in the browser, run cargo doc --no-deps --open
.
License
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0.