Struct DataTypeMap

pub struct DataTypeMap {
    pub arrow_type: PyDataType,
    pub python_type: PythonType,
    pub sql_type: SqlType,
}

These bindings are tying together several disparate systems. You have SQL types for the SQL strings and RDBMS systems itself. Rust types for the DataFusion code Arrow types which represents the underlying arrow format Python types which represent the type in Python It is important to keep all of those types in a single and manageable location. Therefore this structure exists to map those types and provide a simple place for developers to map types from one system to another.

Fields

arrow_type: PyDataType

python_type: PythonType

sql_type: SqlType

Implementations

impl DataTypeMap

pub fn map_from_arrow_type(arrow_type: &DataType) -> Result<DataTypeMap, PyErr>

pub fn map_from_scalar_value( scalar_val: &ScalarValue, ) -> Result<DataTypeMap, PyErr>

Generate the DataTypeMap from a ScalarValue instance

pub fn map_from_scalar_to_arrow( scalar_val: &ScalarValue, ) -> Result<DataType, PyErr>

Maps a ScalarValue to an Arrow DataType

impl DataTypeMap

pub fn py_new( arrow_type: PyDataType, python_type: PythonType, sql_type: SqlType, ) -> Self

pub fn py_map_from_parquet_type_str( parquet_str_type: String, ) -> PyResult<DataTypeMap>

When using pyarrow.parquet.read_metadata().schema.column(x).physical_type you are presented with a String type for schema rather than an object type. Here we make a best effort to convert that to a physical type.

pub fn py_map_from_arrow_type(arrow_type: &PyDataType) -> PyResult<DataTypeMap>

pub fn py_map_from_arrow_type_str( arrow_type_str: String, ) -> PyResult<DataTypeMap>

pub fn py_map_from_sql_type(sql_type: &SqlType) -> PyResult<DataTypeMap>

pub fn friendly_arrow_type_name(&self) -> PyResult<&str>

Unfortunately PyO3 does not allow for us to expose the DataType as an enum since we cannot directly annotae the Enum instance of dependency code. Therefore, here we provide an enum to mimic it.

Trait Implementations

impl Clone for DataTypeMap

fn clone(&self) -> DataTypeMap

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for DataTypeMap

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl IntoPy<Py<PyAny>> for DataTypeMap

fn into_py(self, py: Python<'_>) -> PyObject

👎Deprecated since 0.23.0: IntoPy is going to be replaced by IntoPyObject. See the migration guide (https://pyo3.rs/v0.23.0/migration) for more information.

Performs the conversion.

impl<'py> IntoPyObject<'py> for DataTypeMap

type Target = DataTypeMap

The Python output type

type Output = Bound<'py, <DataTypeMap as IntoPyObject<'py>>::Target>

The smart pointer type to use.

type Error = PyErr

The type returned in the event of a conversion error.

fn into_pyobject( self, py: Python<'py>, ) -> Result<<Self as IntoPyObject<'_>>::Output, <Self as IntoPyObject<'_>>::Error>

Performs the conversion.

impl PyClass for DataTypeMap

type Frozen = False

Whether the pyclass is frozen.

impl PyClassBaseType for DataTypeMap

type LayoutAsBase = PyClassObject<DataTypeMap>

type BaseNativeType = <DataTypeMap as PyClassImpl>::BaseNativeType

type Initializer = PyClassInitializer<DataTypeMap>

type PyClassMutability = <DataTypeMap as PyClassImpl>::PyClassMutability

impl PyClassImpl for DataTypeMap

const IS_BASETYPE: bool = true

#[pyclass(subclass)]

const IS_SUBCLASS: bool = false

#[pyclass(extends=…)]

const IS_MAPPING: bool = false

#[pyclass(mapping)]

const IS_SEQUENCE: bool = false

#[pyclass(sequence)]

type BaseType = PyAny

Base class

type ThreadChecker = SendablePyClass<DataTypeMap>

This handles following two situations:

type PyClassMutability = <<PyAny as PyClassBaseType>::PyClassMutability as PyClassMutability>::MutableChild

Immutable or mutable

type Dict = PyClassDummySlot

Specify this class has #[pyclass(dict)] or not.

type WeakRef = PyClassDummySlot

Specify this class has #[pyclass(weakref)] or not.

type BaseNativeType = PyAny

The closest native ancestor. This is PyAny by default, and when you declare #[pyclass(extends=PyDict)], it’s PyDict.

fn items_iter() -> PyClassItemsIter

fn doc(py: Python<'_>) -> PyResult<&'static CStr>

Rendered class doc

fn lazy_type_object() -> &'static LazyTypeObject<Self>

fn dict_offset() -> Option<isize>

fn weaklist_offset() -> Option<isize>

impl PyClassNewTextSignature<DataTypeMap> for PyClassImplCollector<DataTypeMap>

fn new_text_signature(self) -> Option<&'static str>

impl<'a, 'py> PyFunctionArgument<'a, 'py, false> for &'a DataTypeMap

type Holder = Option<PyRef<'py, DataTypeMap>>

fn extract( obj: &'a Bound<'py, PyAny>, holder: &'a mut Self::Holder, ) -> PyResult<Self>

impl<'a, 'py> PyFunctionArgument<'a, 'py, false> for &'a mut DataTypeMap

type Holder = Option<PyRefMut<'py, DataTypeMap>>

fn extract( obj: &'a Bound<'py, PyAny>, holder: &'a mut Self::Holder, ) -> PyResult<Self>

impl PyMethods<DataTypeMap> for PyClassImplCollector<DataTypeMap>

fn py_methods(self) -> &'static PyClassItems

impl PyTypeInfo for DataTypeMap

const NAME: &'static str = "DataTypeMap"

Class name.

const MODULE: Option<&'static str>

Module name, if any.

fn type_object_raw(py: Python<'_>) -> *mut PyTypeObject

Returns the PyTypeObject instance for this type.

fn type_object(py: Python<'_>) -> Bound<'_, PyType>

Returns the safe abstraction over the type object.

fn type_object_bound(py: Python<'_>) -> Bound<'_, PyType>

👎Deprecated since 0.23.0: renamed to PyTypeInfo::type_object

Deprecated name for PyTypeInfo::type_object.

fn is_type_of(object: &Bound<'_, PyAny>) -> bool

Checks if object is an instance of this type or a subclass of this type.

fn is_type_of_bound(object: &Bound<'_, PyAny>) -> bool

👎Deprecated since 0.23.0: renamed to PyTypeInfo::is_type_of

Deprecated name for PyTypeInfo::is_type_of.

fn is_exact_type_of(object: &Bound<'_, PyAny>) -> bool

Checks if object is an instance of this type.

fn is_exact_type_of_bound(object: &Bound<'_, PyAny>) -> bool

👎Deprecated since 0.23.0: renamed to PyTypeInfo::is_exact_type_of

Deprecated name for PyTypeInfo::is_exact_type_of.

impl DerefToPyAny for DataTypeMap