rust-data-processing 0.3.4

Schema-first ingestion (CSV, JSON, Parquet, Excel) into an in-memory DataSet, plus Polars-backed pipelines, SQL, profiling, validation, and map/reduce-style processing.
1
2
3
4
5
6
7

# MinIO S3A for integration handoff / Databricks warehouse paths.
spark.hadoop.fs.s3a.endpoint                    http://minio:9000
spark.hadoop.fs.s3a.access.key                  rdp_minio
spark.hadoop.fs.s3a.secret.key                  rdp_minio_secret
spark.hadoop.fs.s3a.path.style.access           true
spark.hadoop.fs.s3a.connection.ssl.enabled      false
spark.hadoop.fs.s3a.impl                        org.apache.hadoop.fs.s3a.S3AFileSystem