Crate pyspark_arrow_rs

Source

Traits§

HasArrowSparkSchema

Functions§

field_to_spark
Convert arrow field to spark SQL representation
get_arrow_schema
Depending on serde_arrow, get the fields and do some cleanup for unsupported types like largeutf8
get_spark_ddl
Get spark DDL as a string
to_record_batch
Generic function to convert vec of structs into a recordbatch

Derive Macros§

HasArrowSparkSchema