Module spark_connect_rs::dataframe

source ·
Expand description

The spark-connect-rs crate is currently just a meta-package shim for spark-connect-core DataFrame representation for Spark Connection

Structs§

  • DataFrame
    DataFrame is composed of a SparkSession referencing a Spark Connect enabled cluster, and a LogicalPlanBuilder which represents the unresolved spark::Plan to be submitted to the cluster when an action is called.
  • DataFrameReader
    DataFrameReader represents the entrypoint to create a DataFrame from a specific file format.
  • DataFrameWriter
    DataFrameWriter provides the ability to output a DataFrame to a specific file format supported by Spark
  • DataStreamReader
    DataStreamReader represents the entrypoint to create a streaming DataFrame
  • DataStreamWriter
    DataStreamWriter provides the ability to output a StreamingQuery which can then be used to monitor the active stream
  • StreamingQuery
    Represents the active streaming created from a start on the writer

Enums§