Struct datafusion_python::common::data_type::DataTypeMap

pub struct DataTypeMap {
    pub arrow_type: PyDataType,
    pub python_type: PythonType,
    pub sql_type: SqlType,
}

These bindings are tying together several disparate systems. You have SQL types for the SQL strings and RDBMS systems itself. Rust types for the DataFusion code Arrow types which represents the underlying arrow format Python types which represent the type in Python It is important to keep all of those types in a single and managable location. Therefore this structure exists to map those types and provide a simple place for developers to map types from one system to another.

Fields

arrow_type: PyDataType

python_type: PythonType

sql_type: SqlType

Implementations

impl DataTypeMap

pub fn map_from_arrow_type(arrow_type: &DataType) -> Result<DataTypeMap, PyErr>

pub fn map_from_scalar_value( scalar_val: &ScalarValue ) -> Result<DataTypeMap, PyErr>

Generate the DataTypeMap from a ScalarValue instance

pub fn map_from_scalar_to_arrow( scalar_val: &ScalarValue ) -> Result<DataType, PyErr>

Maps a ScalarValue to an Arrow DataType

impl DataTypeMap

pub fn py_new( arrow_type: PyDataType, python_type: PythonType, sql_type: SqlType ) -> Self

pub fn py_map_from_parquet_type_str( parquet_str_type: String ) -> PyResult<DataTypeMap>

When using pyarrow.parquet.read_metadata().schema.column(x).physical_type you are presented with a String type for schema rather than an object type. Here we make a best effort to convert that to a physical type.

pub fn py_map_from_arrow_type(arrow_type: &PyDataType) -> PyResult<DataTypeMap>

pub fn py_map_from_arrow_type_str( arrow_type_str: String ) -> PyResult<DataTypeMap>

pub fn py_map_from_sql_type(sql_type: &SqlType) -> PyResult<DataTypeMap>

pub fn friendly_arrow_type_name(&self) -> PyResult<&str>

Unfortunately PyO3 does not allow for us to expose the DataType as an enum since we cannot directly annotae the Enum instance of dependency code. Therefore, here we provide an enum to mimic it.

Trait Implementations

impl Clone for DataTypeMap

fn clone(&self) -> DataTypeMap

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DataTypeMap

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl IntoPy<Py<PyAny>> for DataTypeMap

fn into_py(self, py: Python<'_>) -> PyObject

Performs the conversion.

impl PyClass for DataTypeMap

type Frozen = False

Whether the pyclass is frozen.

impl PyClassImpl for DataTypeMap

const IS_BASETYPE: bool = true

#[pyclass(subclass)]

const IS_SUBCLASS: bool = false

#[pyclass(extends=…)]

const IS_MAPPING: bool = false

#[pyclass(mapping)]

const IS_SEQUENCE: bool = false

#[pyclass(sequence)]

type Layout = PyCell<DataTypeMap>

Layout

type BaseType = PyAny

Base class

type ThreadChecker = ThreadCheckerStub<DataTypeMap>

This handles following two situations:

type PyClassMutability = <<PyAny as PyClassBaseType>::PyClassMutability as PyClassMutability>::MutableChild

Immutable or mutable

type Dict = PyClassDummySlot

Specify this class has #[pyclass(dict)] or not.

type WeakRef = PyClassDummySlot

Specify this class has #[pyclass(weakref)] or not.

type BaseNativeType = PyAny

The closest native ancestor. This is PyAny by default, and when you declare #[pyclass(extends=PyDict)], it’s PyDict.

fn items_iter() -> PyClassItemsIter

fn doc(py: Python<'_>) -> PyResult<&'static CStr>

Rendered class doc

fn lazy_type_object() -> &'static LazyTypeObject<Self>

fn dict_offset() -> Option<isize>

fn weaklist_offset() -> Option<isize>

impl PyClassNewTextSignature<DataTypeMap> for PyClassImplCollector<DataTypeMap>

fn new_text_signature(self) -> Option<&'static str>

impl<'a, 'py> PyFunctionArgument<'a, 'py> for &'a DataTypeMap

type Holder = Option<PyRef<'py, DataTypeMap>>

fn extract(obj: &'py PyAny, holder: &'a mut Self::Holder) -> PyResult<Self>

impl<'a, 'py> PyFunctionArgument<'a, 'py> for &'a mut DataTypeMap

type Holder = Option<PyRefMut<'py, DataTypeMap>>

fn extract(obj: &'py PyAny, holder: &'a mut Self::Holder) -> PyResult<Self>

impl PyMethods<DataTypeMap> for PyClassImplCollector<DataTypeMap>

fn py_methods(self) -> &'static PyClassItems

impl PyTypeInfo for DataTypeMap

type AsRefTarget = PyCell<DataTypeMap>

Utility type to make Py::as_ref work.

const NAME: &'static str = _

Class name.

const MODULE: Option<&'static str> = _

Module name, if any.

fn type_object_raw(py: Python<'_>) -> *mut PyTypeObject

Returns the PyTypeObject instance for this type.

fn type_object(py: Python<'_>) -> &PyType

Returns the safe abstraction over the type object.

fn is_type_of(object: &PyAny) -> bool

Checks if object is an instance of this type or a subclass of this type.

fn is_exact_type_of(object: &PyAny) -> bool

Checks if object is an instance of this type.