Enum DataType 

pub enum DataType {
    Null,
    Boolean,
    Byte,
    Short,
    Integer,
    Long,
    Float,
    Double,
    Decimal {
        precision: i32,
        scale: i32,
    },
    String {
        collation: String,
    },
    Char {
        length: i32,
    },
    Varchar {
        length: i32,
    },
    Binary,
    Date,
    Timestamp,
    TimestampNtz,
    Time {
        precision: i32,
    },
    CalendarInterval,
    YearMonthInterval {
        start_field: i32,
        end_field: i32,
    },
    DayTimeInterval {
        start_field: i32,
        end_field: i32,
    },
    Array {
        element_type: Box<DataType>,
        contains_null: bool,
    },
    Map {
        key_type: Box<DataType>,
        value_type: Box<DataType>,
        value_contains_null: bool,
    },
    Struct {
        fields: Vec<StructField>,
    },
    Variant,
    Geometry {
        srid: i32,
    },
    Geography {
        srid: i32,
    },
    Udt {
        type_str: String,
        jvm_class: Option<String>,
        python_class: Option<String>,
        serialized_python_class: Option<String>,
        sql_type: Option<Box<DataType>>,
    },
    Unparsed {
        data_type_string: String,
    },
}

The base DataType representation, mirroring pyspark.sql.types.DataType.

All concrete types are variants of this enum. Each variant carries the data needed to fully specify that type (e.g., DecimalType carries precision and scale).

Variants

Null

pyspark.sql.types.NullType

Boolean

pyspark.sql.types.BooleanType

Byte

pyspark.sql.types.ByteType (tinyint)

Short

pyspark.sql.types.ShortType (smallint)

Integer

pyspark.sql.types.IntegerType (int)

Long

pyspark.sql.types.LongType (bigint)

Float

pyspark.sql.types.FloatType

Double

pyspark.sql.types.DoubleType

Decimal

pyspark.sql.types.DecimalType

Fields

precision: i32

scale: i32

String

pyspark.sql.types.StringType

Fields

collation: String

Char

pyspark.sql.types.CharType

Fields

length: i32

Varchar

pyspark.sql.types.VarcharType

Fields

length: i32

Binary

pyspark.sql.types.BinaryType

Date

pyspark.sql.types.DateType

Timestamp

pyspark.sql.types.TimestampType

TimestampNtz

pyspark.sql.types.TimestampNTZType

Time

pyspark.sql.types.TimeType

Fields

precision: i32

CalendarInterval

pyspark.sql.types.CalendarIntervalType

YearMonthInterval

pyspark.sql.types.YearMonthIntervalType

Fields

start_field: i32

end_field: i32

DayTimeInterval

pyspark.sql.types.DayTimeIntervalType

Fields

start_field: i32

end_field: i32

Array

pyspark.sql.types.ArrayType

Fields

element_type: Box<DataType>

contains_null: bool

Map

pyspark.sql.types.MapType

Fields

key_type: Box<DataType>

value_type: Box<DataType>

value_contains_null: bool

Struct

pyspark.sql.types.StructType

Fields

fields: Vec<StructField>

Variant

pyspark.sql.types.VariantType

Geometry

pyspark.sql.types.GeometryType

Fields

srid: i32

Geography

pyspark.sql.types.GeographyType

Fields

srid: i32

Udt

pyspark.sql.types.UserDefinedType (stub)

Fields

type_str: String

jvm_class: Option<String>

python_class: Option<String>

serialized_python_class: Option<String>

sql_type: Option<Box<DataType>>

Unparsed

pyspark.sql.connect.types.UnparsedDataType - a DDL type string left for the server to parse (round-trips through the unparsed proto).

Fields

data_type_string: String

Implementations

impl DataType

pub fn from_ddl(ddl_str: &str) -> Result<DataType>

Parses a DDL-formatted string into a DataType, mirroring DataType.fromDDL().

This supports:

• Primitive types: int, bigint, string, double, boolean, date, timestamp, binary, tinyint, smallint, float, decimal(p,s), char(n), varchar(n), interval
• Complex types: array<…>, map<…,…>, structname:type,...
• Top-level struct can omit the “struct<>” wrapper for backward compatibility
• DDL like “a INT, b STRING” is parsed as a struct

Examples:

DataType::from_ddl("int") // IntegerType
DataType::from_ddl("array<string>") // ArrayType(StringType, true)
DataType::from_ddl("struct<name:string,age:int>") // StructType
DataType::from_ddl("a INT, b STRING") // Top-level struct

pub fn need_conversion(&self) -> bool

Returns whether this type needs conversion between Python objects and internal SQL objects. This is used to avoid unnecessary conversions for ArrayType/MapType/StructType.

Types that need conversion include:

• DateType: needs conversion to/from datetime.date
• TimestampType: needs conversion to/from datetime.datetime
• TimestampNTZType: needs conversion to/from datetime.datetime (no timezone)
• TimeType: needs conversion to/from datetime.time
• DayTimeIntervalType: needs conversion to/from datetime.timedelta
• CalendarIntervalType: needs conversion
• YearMonthIntervalType: needs conversion (complex)
• ArrayType: if element type needs conversion
• MapType: if key or value type needs conversion
• StructType: always needs conversion

pub fn type_name(&self) -> String

Returns the type name, mirroring DataType.typeName().

For most types, this is the class name with the “Type” suffix removed and lowercased. E.g., “ByteType” -> “byte”, but NullType -> “void”, and special handling for others.

pub fn simple_string(&self) -> String

Returns the simple string representation, mirroring DataType.simpleString().

For example:

• “int”, “string”, “boolean”
• “decimal(10,0)”, “char(50)”, “varchar(100)”
• “array”, “map<string,int>”, “structname:string,age:int”
• “interval day to second”

pub fn json_value(&self) -> Value

Returns the JSON value representation, mirroring DataType.jsonValue().

Most simple types return their type name as a string. Complex types (Array, Map, Struct) return a dictionary with type and component info.

pub fn json(&self) -> String

Converts to a JSON string, mirroring DataType.json().

pub fn from_json(value: &Value) -> Result<DataType>

Parses a JSON value into a DataType, mirroring the reverse of json() / jsonValue().

pub fn from_json_str(s: &str) -> Result<DataType>

Parses a JSON string into a DataType (convenience over [from_json]).

pub fn to_proto(&self) -> DataType

Converts to a protobuf DataType, mirroring pyspark.sql.connect.types.pyspark_types_to_proto_types.

pub fn from_proto(proto: &DataType) -> Result<DataType>

Converts from a protobuf DataType, mirroring pyspark.sql.connect.types.proto_schema_to_pyspark_data_type.

impl DataType

Helper methods for StructType operations, mirroring pyspark.sql.types.StructType. Since StructType is represented as DataType::Struct { fields }, these methods provide convenience operations for struct types.

pub fn field_names(&self) -> Result<Vec<String>>

Returns all field names in a StructType, mirroring StructType.fieldNames().

Returns an error if called on a non-Struct type.

pub fn names(&self) -> Result<Vec<String>>

Alias for field_names(), also mirroring pyspark’s names attribute.

pub fn to_ddl(&self) -> Result<String>

DDL string for a StructType, mirroring StructType.toDDL(): comma-separated name type[ NOT NULL][ COMMENT '...'] per field.

pub fn tree_string(&self) -> Result<String>

Tree-string for a StructType, mirroring StructType.treeString().

pub fn tree_string_with_depth(&self, max_depth: i32) -> Result<String>

Like DataType::tree_string, but stops recursing into nested structs once max_depth nesting levels have been printed (top-level fields are depth 1). Mirrors StructType.treeString(maxDepth).

pub fn to_nullable(&self) -> DataType

Return a copy with every field made nullable (recursively), mirroring StructType.toNullable().

pub fn add( &self, field_name: &str, field_type: DataType, nullable: bool, metadata: Option<BTreeMap<String, Value>>, ) -> Result<DataType>

Adds a field to a StructType, mirroring StructType.add().

This is a builder method that returns a new StructType with the field added. Returns an error if called on a non-Struct type.

Example:

let struct_type = DataType::Struct { fields: vec![] };
let with_field = struct_type.add(
    "name",
    DataType::String { collation: "UTF8_BINARY".to_string() },
    true,
    None,
)?;

Trait Implementations

impl Clone for DataType

fn clone(&self) -> DataType

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DataType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DataType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Hash for DataType

fn hash<H: Hasher>(&self, state: &mut H)

Feeds this value into the given Hasher.

fn hash_slice<H>(data: &[Self], state: &mut H)

where H: Hasher, Self: Sized,

Feeds a slice of this type into the given Hasher.

impl PartialEq for DataType

fn eq(&self, other: &DataType) -> bool

Equality operator ==.

fn ne(&self, other: &Rhs) -> bool

Inequality operator !=.

impl StructuralPartialEq for DataType