TPC Tools
Command-line tools for invoking TPC-H and TPC-DS data generators in parallel and re-organizing the output files into directory structures that can be consumed by tools such as Apache Spark or Apache Arrow DataFusion/Ballista.
Also supports converting the output to Parquet.
TPC-DS
Install dependencies.
Download data generator from https://www.tpc.org/tpc_documents_current_versions/current_specifications5.asp
Generate data.
Example output.
Generated TPC-DS data at scale factor 1000 with 48 partitions in: 6247.155671938s
Convert to Parquet
TPC-H
Install dependencies.
Generate data.
Convert data to Parquet
Legal Stuff
TPC-H is Copyright © 1993-2022 Transaction Processing Performance Council. The full TPC-H specification in PDF format can be found here
TPC-DS is Copyright © 2021 Transaction Processing Performance Council. The full TPC-DS specification in PDF format can be found here
TPC, TPC Benchmark, TPC-H, and TPC-DS are trademarks of the Transaction Processing Performance Council.