TPC-DS Data Generator in Arrow format

Generate TPC-DS data directly into Apache Arrow format using the tpcdsgen and arrow crates.

Supports arrow 60 (default) and arrow 59:

# arrow 60
tpcdsgen-arrow = "..."
# arrow 59
tpcdsgen-arrow = { version = "...", default-features = false, features = ["arrow_59"] }

Example usage

See docs.rs page

Testing

This crate ensures correct results using two methods.

1. Basic functional tests are in Rust doc tests in the source code (cargo test --locked -p tpcdsgen-arrow --doc)
2. The reparse integration test ensures that the Arrow generators produce the same results as parsing the original DAT format (cargo test --locked -p tpcdsgen-arrow --test reparse)

Contributing

Please see CONTRIBUTING.md for more information on how to contribute to this project.