Struct polars_core::datatypes::Logical

source ·
pub struct Logical<Logical: PolarsDataType, Physical: PolarsDataType>(pub ChunkedArray<Physical>, _, pub Option<DataType>);
Expand description

Maps a logical type to a chunked array implementation of the physical type. This saves a lot of compiler bloat and allows us to reuse functionality.

Tuple Fields§

§0: ChunkedArray<Physical>§2: Option<DataType>

Implementations§

source§

impl Logical<DecimalType, Int128Type>

source

pub fn precision(&self) -> Option<usize>

Available on crate feature dtype-decimal only.
source

pub fn scale(&self) -> usize

Available on crate feature dtype-decimal only.
source§

impl<K: PolarsDataType, T: PolarsDataType> Logical<K, T>
where Self: LogicalType,

source

pub fn field(&self) -> Field

Methods from Deref<Target = ChunkedArray<T>>§

source

pub fn sum(&self) -> Option<IdxSize>

source

pub fn min(&self) -> Option<bool>

source

pub fn max(&self) -> Option<bool>

source

pub fn mean(&self) -> Option<f64>

source

pub fn append(&mut self, other: &Self)

Append in place. This is done by adding the chunks of other to this ChunkedArray.

See also extend for appends to the underlying memory

source

pub fn apply_values_generic<'a, U, K, F>(&'a self, op: F) -> ChunkedArray<U>
where U: PolarsDataType, F: FnMut(T::Physical<'a>) -> K, U::Array: ArrayFromIter<K>,

source

pub fn apply_nonnull_values_generic<'a, U, K, F>( &'a self, dtype: DataType, op: F ) -> ChunkedArray<U>

Applies a function only to the non-null elements, propagating nulls.

source

pub fn try_apply_nonnull_values_generic<'a, U, K, F, E>( &'a self, op: F ) -> Result<ChunkedArray<U>, E>
where U: PolarsDataType, F: FnMut(T::Physical<'a>) -> Result<K, E>, U::Array: ArrayFromIter<K> + ArrayFromIter<Option<K>>,

Applies a function only to the non-null elements, propagating nulls.

source

pub fn apply_generic<'a, U, K, F>(&'a self, op: F) -> ChunkedArray<U>
where U: PolarsDataType, F: FnMut(Option<T::Physical<'a>>) -> Option<K>, U::Array: ArrayFromIter<Option<K>>,

source

pub fn try_apply_generic<'a, U, K, F, E>( &'a self, op: F ) -> Result<ChunkedArray<U>, E>
where U: PolarsDataType, F: FnMut(Option<T::Physical<'a>>) -> Result<Option<K>, E> + Copy, U::Array: ArrayFromIter<Option<K>>,

source

pub fn cast_and_apply_in_place<F, S>(&self, f: F) -> ChunkedArray<S>
where F: Fn(S::Native) -> S::Native + Copy, S: PolarsNumericType,

Cast a numeric array to another numeric data type and apply a function in place. This saves an allocation.

source

pub fn apply_mut<F>(&mut self, f: F)
where F: Fn(T::Native) -> T::Native + Copy,

source

pub fn apply_mut<'a, F>(&'a self, f: F) -> Self
where F: FnMut(&'a str) -> &'a str,

source

pub fn apply_to_buffer<'a, F>(&'a self, f: F) -> Self
where F: FnMut(&'a str, &mut String),

Utility that reuses an string buffer to amortize allocations. Prefer this over an apply that returns an owned String.

source

pub fn apply_mut<'a, F>(&'a self, f: F) -> Self
where F: FnMut(&'a [u8]) -> &'a [u8],

source

pub fn apply_as_ints<F>(&self, f: F) -> Series
where F: Fn(&Series) -> Series,

source

pub fn apply_as_ints<F>(&self, f: F) -> Series
where F: Fn(&Series) -> Series,

source

pub fn len(&self) -> usize

Get the length of the ChunkedArray

source

pub fn null_count(&self) -> usize

Return the number of null values in the ChunkedArray.

source

pub unsafe fn set_null_count(&mut self, null_count: IdxSize)

Set the null count directly.

This can be useful after mutably adjusting the validity of the underlying arrays.

§Safety

The new null count must match the total null count of the underlying arrays.

source

pub fn is_empty(&self) -> bool

Check if ChunkedArray is empty.

source

pub fn rechunk(&self) -> Self

source

pub fn slice(&self, offset: i64, length: usize) -> Self

Slice the array. The chunks are reallocated the underlying data slices are zero copy.

When offset is negative it will be counted from the end of the array. This method will never error, and will slice the best match when offset, or length is out of bounds

source

pub fn limit(&self, num_elements: usize) -> Self
where Self: Sized,

Take a view of top n elements

source

pub fn head(&self, length: Option<usize>) -> Self
where Self: Sized,

Get the head of the ChunkedArray

source

pub fn tail(&self, length: Option<usize>) -> Self
where Self: Sized,

Get the tail of the ChunkedArray

source

pub fn prune_empty_chunks(&mut self)

Remove empty chunks.

source

pub fn to_decimal(&self, infer_length: usize) -> PolarsResult<Series>

Available on crate feature dtype-decimal only.

Convert an StringChunked to a Series of DataType::Decimal. Scale needed for the decimal type are inferred. Parsing is not strict.
Scale inference assumes that all tested strings are well-formed numbers, and may produce unexpected results for scale if this is not the case.

If the decimal precision and scale are already known, consider using the cast method.

source

pub fn extend(&mut self, other: &Self)

Extend the memory backed by this array with the values from other.

Different from ChunkedArray::append which adds chunks to this ChunkedArray extend appends the data from other to the underlying PrimitiveArray and thus may cause a reallocation.

However if this does not cause a reallocation, the resulting data structure will not have any extra chunks and thus will yield faster queries.

Prefer extend over append when you want to do a query after a single append. For instance during online operations where you add n rows and rerun a query.

Prefer append over extend when you want to append many times before doing a query. For instance when you read in multiple files and when to store them in a single DataFrame. In the latter case finish the sequence of append operations with a rechunk.

source

pub fn for_each<'a, F>(&'a self, op: F)
where F: FnMut(Option<T::Physical<'a>>),

source

pub fn is_null(&self) -> BooleanChunked

Get a mask of the null values.

source

pub fn is_not_null(&self) -> BooleanChunked

Get a mask of the valid values.

source

pub fn rolling_map_float<F>( &self, window_size: usize, f: F ) -> PolarsResult<Self>
where F: FnMut(&mut ChunkedArray<T>) -> Option<T::Native>,

Available on crate feature rolling_window only.

Apply a rolling custom function. This is pretty slow because of dynamic dispatch.

source

pub unsafe fn to_string(&self) -> StringChunked

§Safety

String is not validated

source

pub fn as_binary(&self) -> BinaryChunked

source

pub fn any(&self) -> bool

Returns whether any of the values in the column are true.

Null values are ignored.

source

pub fn all(&self) -> bool

Returns whether all values in the array are true.

Null values are ignored.

source

pub fn any_kleene(&self) -> Option<bool>

Returns whether any of the values in the column are true.

The output is unknown (None) if the array contains any null values and no true values.

source

pub fn all_kleene(&self) -> Option<bool>

Returns whether all values in the column are true.

The output is unknown (None) if the array contains any null values and no false values.

source

pub fn is_nan(&self) -> BooleanChunked

source

pub fn is_not_nan(&self) -> BooleanChunked

source

pub fn is_finite(&self) -> BooleanChunked

source

pub fn is_infinite(&self) -> BooleanChunked

source

pub fn none_to_nan(&self) -> Self

Convert missing values to NaN values.

source

pub fn to_canonical(&self) -> Self

source

pub fn par_iter(&self) -> impl ParallelIterator<Item = Option<Series>> + '_

source

pub fn par_iter_indexed( &mut self ) -> impl IndexedParallelIterator<Item = Option<Series>> + '_

source

pub fn par_iter_indexed( &self ) -> impl IndexedParallelIterator<Item = Option<&str>>

source

pub fn par_iter(&self) -> impl ParallelIterator<Item = Option<&str>> + '_

source

pub fn iter(&self) -> impl PolarsIterator<Item = Option<T::Physical<'_>>>

source

pub fn to_ndarray(&self) -> PolarsResult<ArrayView1<'_, T::Native>>

Available on crate feature ndarray only.

If data is aligned in a single chunk and has no Null values a zero copy view is returned as an ndarray

source

pub fn to_ndarray<N>(&self) -> PolarsResult<Array2<N::Native>>

Available on crate feature ndarray only.

If all nested Series have the same length, a 2 dimensional ndarray::Array is returned.

source

pub fn to_bytes_hashes<'a>( &'a self, multithreaded: bool, hb: RandomState ) -> Vec<Vec<BytesHash<'a>>>

source

pub unsafe fn with_chunks(&self, chunks: Vec<ArrayRef>) -> Self

§Safety

The Arrow datatype of all chunks must match the PolarsDataType T.

source

pub unsafe fn amortized_iter( &self ) -> AmortizedListIter<'_, impl Iterator<Item = Option<ArrayBox>> + '_>

This is an iterator over a ListChunked that save allocations. A Series is: 1. Arc<ChunkedArray> ChunkedArray is: 2. Vec< 3. ArrayRef>

The ArrayRef we indicated with 3. will be updated during iteration. The Series will be pinned in memory, saving an allocation for

  1. Arc<..>
  2. Vec<…>
§Warning

Though memory safe in the sense that it will not read unowned memory, UB, or memory leaks this function still needs precautions. The returned should never be cloned or taken longer than a single iteration, as every call on next of the iterator will change the contents of that Series.

§Safety

The lifetime of UnstableSeries is bound to the iterator. Keeping it alive longer than the iterator is UB.

source

pub unsafe fn amortized_iter_with_name( &self, name: &str ) -> AmortizedListIter<'_, impl Iterator<Item = Option<ArrayBox>> + '_>

§Safety

The lifetime of UnstableSeries is bound to the iterator. Keeping it alive longer than the iterator is UB.

source

pub fn apply_amortized_generic<'a, F, K, V>(&'a self, f: F) -> ChunkedArray<V>

Apply a closure F elementwise.

source

pub fn try_apply_amortized_generic<'a, F, K, V>( &'a self, f: F ) -> PolarsResult<ChunkedArray<V>>

source

pub fn for_each_amortized<'a, F>(&'a self, f: F)
where F: FnMut(Option<UnstableSeries<'a>>),

source

pub fn zip_and_apply_amortized<'a, T, I, F>( &'a self, ca: &'a ChunkedArray<T>, f: F ) -> Self
where T: PolarsDataType, &'a ChunkedArray<T>: IntoIterator<IntoIter = I>, I: TrustedLen<Item = Option<T::Physical<'a>>>, F: FnMut(Option<UnstableSeries<'a>>, Option<T::Physical<'a>>) -> Option<Series>,

Zip with a ChunkedArray then apply a binary function F elementwise.

source

pub fn binary_zip_and_apply_amortized<'a, T, U, F>( &'a self, ca1: &'a ChunkedArray<T>, ca2: &'a ChunkedArray<U>, f: F ) -> Self

source

pub fn try_zip_and_apply_amortized<'a, T, I, F>( &'a self, ca: &'a ChunkedArray<T>, f: F ) -> PolarsResult<Self>
where T: PolarsDataType, &'a ChunkedArray<T>: IntoIterator<IntoIter = I>, I: TrustedLen<Item = Option<T::Physical<'a>>>, F: FnMut(Option<UnstableSeries<'a>>, Option<T::Physical<'a>>) -> PolarsResult<Option<Series>>,

source

pub fn apply_amortized<'a, F>(&'a self, f: F) -> Self
where F: FnMut(UnstableSeries<'a>) -> Series,

Apply a closure F elementwise.

source

pub fn try_apply_amortized<'a, F>(&'a self, f: F) -> PolarsResult<Self>

source

pub fn inner_dtype(&self) -> DataType

Get the inner data type of the list.

source

pub fn set_inner_dtype(&mut self, dtype: DataType)

source

pub fn set_fast_explode(&mut self)

source

pub fn _can_fast_explode(&self) -> bool

source

pub unsafe fn to_logical(&mut self, inner_dtype: DataType)

Set the logical type of the ListChunked.

§Safety

The caller must ensure that the logical type given fits the physical type of the array.

source

pub fn get_inner(&self) -> Series

Get the inner values as Series, ignoring the list offsets.

source

pub fn apply_to_inner( &self, func: &dyn Fn(Series) -> PolarsResult<Series> ) -> PolarsResult<ListChunked>

Ignore the list indices and apply func to the inner type as Series.

source

pub unsafe fn get_object_unchecked( &self, index: usize ) -> Option<&dyn PolarsObjectSafe>

Available on crate feature object only.

Get a hold to an object that can be formatted or downcasted via the Any trait.

§Safety

No bounds checks

source

pub fn get_object(&self, index: usize) -> Option<&dyn PolarsObjectSafe>

Available on crate feature object only.

Get a hold to an object that can be formatted or downcasted via the Any trait.

source

pub fn sample_n( &self, n: usize, with_replacement: bool, shuffle: bool, seed: Option<u64> ) -> PolarsResult<Self>

Available on crate feature random only.

Sample n datapoints from this ChunkedArray.

source

pub fn sample_frac( &self, frac: f64, with_replacement: bool, shuffle: bool, seed: Option<u64> ) -> PolarsResult<Self>

Available on crate feature random only.

Sample a fraction between 0.0-1.0 of this ChunkedArray.

source

pub fn to_vec(&self) -> Vec<Option<T::Native>>

Convert to a Vec of Option<T::Native>.

source

pub fn to_vec_null_aware( &self ) -> Either<Vec<T::Native>, Vec<Option<T::Native>>>

Convert to a Vec but don’t return Option<T::Native> if there are no null values

source

pub fn unset_fast_explode_list(&mut self)

source

pub fn get_flags(&self) -> Settings

source

pub fn is_sorted_flag(&self) -> IsSorted

source

pub fn set_sorted_flag(&mut self, sorted: IsSorted)

Set the ‘sorted’ bit meta info.

source

pub fn with_sorted_flag(&self, sorted: IsSorted) -> Self

Set the ‘sorted’ bit meta info.

source

pub fn first_non_null(&self) -> Option<usize>

Get the index of the first non null value in this ChunkedArray.

source

pub fn last_non_null(&self) -> Option<usize>

Get the index of the last non null value in this ChunkedArray.

source

pub fn iter_validities( &self ) -> Map<Iter<'_, ArrayRef>, fn(_: &ArrayRef) -> Option<&Bitmap>>

Get the buffer of bits representing null values

source

pub fn has_validity(&self) -> bool

Return if any the chunks in this ChunkedArray have a validity bitmap. no bitmap means no null values.

source

pub fn shrink_to_fit(&mut self)

Shrink the capacity of this array to fit its length.

source

pub fn clear(&self) -> Self

source

pub fn unpack_series_matching_type( &self, series: &Series ) -> PolarsResult<&ChunkedArray<T>>

Series to ChunkedArray<T>

source

pub fn chunk_id(&self) -> ChunkIdIter<'_>

Unique id representing the number of chunks

source

pub fn chunks(&self) -> &Vec<ArrayRef>

A reference to the chunks

source

pub unsafe fn chunks_mut(&mut self) -> &mut Vec<ArrayRef>

A mutable reference to the chunks

§Safety

The caller must ensure to not change the DataType or length of any of the chunks. And the null_count remains correct.

source

pub fn is_optimal_aligned(&self) -> bool

Returns true if contains a single chunk and has no null values

source

pub fn dtype(&self) -> &DataType

Get data type of ChunkedArray.

source

pub fn name(&self) -> &str

Name of the ChunkedArray.

source

pub fn ref_field(&self) -> &Field

Get a reference to the field.

source

pub fn rename(&mut self, name: &str)

Rename this ChunkedArray.

source

pub fn get(&self, idx: usize) -> Option<T::Physical<'_>>

Get a single value from this ChunkedArray. If the return values is None this indicates a NULL value.

§Panics

This function will panic if idx is out of bounds.

source

pub unsafe fn get_unchecked(&self, idx: usize) -> Option<T::Physical<'_>>

Get a single value from this ChunkedArray. If the return values is None this indicates a NULL value.

§Safety

It is the callers responsibility that the idx < self.len().

source

pub unsafe fn value_unchecked(&self, idx: usize) -> T::Physical<'_>

Get a single value from this ChunkedArray. Null values are ignored and the returned value could be garbage if it was masked out by NULL. Note that the value always is initialized.

§Safety

It is the callers responsibility that the idx < self.len().

source

pub fn last(&self) -> Option<T::Physical<'_>>

source

pub fn get_as_series(&self, idx: usize) -> Option<Series>

source

pub fn layout(&self) -> ChunkedArrayLayout<'_, T>

source

pub fn cont_slice(&self) -> PolarsResult<&[T::Native]>

Contiguous slice

source

pub fn data_views(&self) -> impl DoubleEndedIterator<Item = &[T::Native]>

Get slices of the underlying arrow data. NOTE: null values should be taken into account by the user of these slices as they are handled separately

source

pub fn into_no_null_iter( &self ) -> impl '_ + Send + Sync + ExactSizeIterator<Item = T::Native> + DoubleEndedIterator + TrustedLen

source

pub fn group_tuples_perfect( &self, max: usize, multithreaded: bool, group_capacity: usize ) -> GroupsProxy

Available on crate feature algorithm_group_by only.
source

pub fn lhs_sub<N: Num + NumCast>(&self, lhs: N) -> Self

Apply lhs - self

source

pub fn lhs_div<N: Num + NumCast>(&self, lhs: N) -> Self

Apply lhs / self

source

pub fn lhs_rem<N: Num + NumCast>(&self, lhs: N) -> Self

Apply lhs % self

Trait Implementations§

source§

impl<K: PolarsDataType, T: PolarsDataType> Clone for Logical<K, T>

source§

fn clone(&self) -> Self

Returns a copy of the value. Read more
1.0.0 · source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
source§

impl<K: PolarsDataType, T: PolarsDataType> Deref for Logical<K, T>

§

type Target = ChunkedArray<T>

The resulting type after dereferencing.
source§

fn deref(&self) -> &Self::Target

Dereferences the value.
source§

impl<K: PolarsDataType, T: PolarsDataType> DerefMut for Logical<K, T>

source§

fn deref_mut(&mut self) -> &mut Self::Target

Mutably dereferences the value.

Auto Trait Implementations§

§

impl<Logical, Physical> Freeze for Logical<Logical, Physical>

§

impl<Logical, Physical> !RefUnwindSafe for Logical<Logical, Physical>

§

impl<Logical, Physical> Send for Logical<Logical, Physical>

§

impl<Logical, Physical> Sync for Logical<Logical, Physical>

§

impl<Logical, Physical> Unpin for Logical<Logical, Physical>
where Logical: Unpin, Physical: Unpin,

§

impl<Logical, Physical> !UnwindSafe for Logical<Logical, Physical>

Blanket Implementations§

source§

impl<T> Any for T
where T: 'static + ?Sized,

source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
source§

impl<T> Borrow<T> for T
where T: ?Sized,

source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
source§

impl<T> DynClone for T
where T: Clone,

source§

fn __clone_box(&self, _: Private) -> *mut ()

source§

impl<T> From<T> for T

source§

fn from(t: T) -> T

Returns the argument unchanged.

source§

impl<T, U> Into<U> for T
where U: From<T>,

source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

source§

impl<T> IntoEither for T

source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
source§

impl<T> Pointable for T

source§

const ALIGN: usize = _

The alignment of pointer.
§

type Init = T

The type for initializers.
source§

unsafe fn init(init: <T as Pointable>::Init) -> usize

Initializes a with the given initializer. Read more
source§

unsafe fn deref<'a>(ptr: usize) -> &'a T

Dereferences the given pointer. Read more
source§

unsafe fn deref_mut<'a>(ptr: usize) -> &'a mut T

Mutably dereferences the given pointer. Read more
source§

unsafe fn drop(ptr: usize)

Drops the object pointed to by the given pointer. Read more
source§

impl<T> ToOwned for T
where T: Clone,

§

type Owned = T

The resulting type after obtaining ownership.
source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

§

type Error = Infallible

The type returned in the event of a conversion error.
source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

source§

fn vzip(self) -> V