Struct polars_core::datatypes::Logical
source · pub struct Logical<Logical: PolarsDataType, Physical: PolarsDataType>(pub ChunkedArray<Physical>, _, pub Option<DataType>);
Expand description
Maps a logical type to a chunked array implementation of the physical type. This saves a lot of compiler bloat and allows us to reuse functionality.
Tuple Fields§
§0: ChunkedArray<Physical>
§2: Option<DataType>
Implementations§
source§impl Logical<DecimalType, Int128Type>
impl Logical<DecimalType, Int128Type>
source§impl<K: PolarsDataType, T: PolarsDataType> Logical<K, T>where
Self: LogicalType,
impl<K: PolarsDataType, T: PolarsDataType> Logical<K, T>where
Self: LogicalType,
Methods from Deref<Target = ChunkedArray<T>>§
pub fn sum(&self) -> Option<IdxSize>
pub fn min(&self) -> Option<bool>
pub fn max(&self) -> Option<bool>
pub fn mean(&self) -> Option<f64>
sourcepub fn append(&mut self, other: &Self)
pub fn append(&mut self, other: &Self)
Append in place. This is done by adding the chunks of other
to this ChunkedArray
.
See also extend
for appends to the underlying memory
pub fn apply_values_generic<'a, U, K, F>(&'a self, op: F) -> ChunkedArray<U>
sourcepub fn apply_nonnull_values_generic<'a, U, K, F>(
&'a self,
dtype: DataType,
op: F
) -> ChunkedArray<U>where
U: PolarsDataType,
F: FnMut(T::Physical<'a>) -> K,
U::Array: ArrayFromIterDtype<K> + ArrayFromIterDtype<Option<K>>,
pub fn apply_nonnull_values_generic<'a, U, K, F>(
&'a self,
dtype: DataType,
op: F
) -> ChunkedArray<U>where
U: PolarsDataType,
F: FnMut(T::Physical<'a>) -> K,
U::Array: ArrayFromIterDtype<K> + ArrayFromIterDtype<Option<K>>,
Applies a function only to the non-null elements, propagating nulls.
sourcepub fn try_apply_nonnull_values_generic<'a, U, K, F, E>(
&'a self,
op: F
) -> Result<ChunkedArray<U>, E>where
U: PolarsDataType,
F: FnMut(T::Physical<'a>) -> Result<K, E>,
U::Array: ArrayFromIter<K> + ArrayFromIter<Option<K>>,
pub fn try_apply_nonnull_values_generic<'a, U, K, F, E>(
&'a self,
op: F
) -> Result<ChunkedArray<U>, E>where
U: PolarsDataType,
F: FnMut(T::Physical<'a>) -> Result<K, E>,
U::Array: ArrayFromIter<K> + ArrayFromIter<Option<K>>,
Applies a function only to the non-null elements, propagating nulls.
pub fn apply_generic<'a, U, K, F>(&'a self, op: F) -> ChunkedArray<U>where
U: PolarsDataType,
F: FnMut(Option<T::Physical<'a>>) -> Option<K>,
U::Array: ArrayFromIter<Option<K>>,
pub fn try_apply_generic<'a, U, K, F, E>(
&'a self,
op: F
) -> Result<ChunkedArray<U>, E>where
U: PolarsDataType,
F: FnMut(Option<T::Physical<'a>>) -> Result<Option<K>, E> + Copy,
U::Array: ArrayFromIter<Option<K>>,
sourcepub fn cast_and_apply_in_place<F, S>(&self, f: F) -> ChunkedArray<S>
pub fn cast_and_apply_in_place<F, S>(&self, f: F) -> ChunkedArray<S>
Cast a numeric array to another numeric data type and apply a function in place. This saves an allocation.
pub fn apply_mut<F>(&mut self, f: F)
pub fn apply_mut<'a, F>(&'a self, f: F) -> Self
sourcepub fn apply_to_buffer<'a, F>(&'a self, f: F) -> Self
pub fn apply_to_buffer<'a, F>(&'a self, f: F) -> Self
Utility that reuses an string buffer to amortize allocations.
Prefer this over an apply
that returns an owned String
.
pub fn apply_mut<'a, F>(&'a self, f: F) -> Self
pub fn apply_as_ints<F>(&self, f: F) -> Series
pub fn apply_as_ints<F>(&self, f: F) -> Series
sourcepub fn null_count(&self) -> usize
pub fn null_count(&self) -> usize
Return the number of null values in the ChunkedArray.
sourcepub unsafe fn set_null_count(&mut self, null_count: IdxSize)
pub unsafe fn set_null_count(&mut self, null_count: IdxSize)
Set the null count directly.
This can be useful after mutably adjusting the validity of the underlying arrays.
§Safety
The new null count must match the total null count of the underlying arrays.
pub fn rechunk(&self) -> Self
sourcepub fn slice(&self, offset: i64, length: usize) -> Self
pub fn slice(&self, offset: i64, length: usize) -> Self
Slice the array. The chunks are reallocated the underlying data slices are zero copy.
When offset is negative it will be counted from the end of the array. This method will never error, and will slice the best match when offset, or length is out of bounds
sourcepub fn limit(&self, num_elements: usize) -> Selfwhere
Self: Sized,
pub fn limit(&self, num_elements: usize) -> Selfwhere
Self: Sized,
Take a view of top n elements
sourcepub fn head(&self, length: Option<usize>) -> Selfwhere
Self: Sized,
pub fn head(&self, length: Option<usize>) -> Selfwhere
Self: Sized,
Get the head of the ChunkedArray
sourcepub fn tail(&self, length: Option<usize>) -> Selfwhere
Self: Sized,
pub fn tail(&self, length: Option<usize>) -> Selfwhere
Self: Sized,
Get the tail of the ChunkedArray
sourcepub fn prune_empty_chunks(&mut self)
pub fn prune_empty_chunks(&mut self)
Remove empty chunks.
sourcepub fn to_decimal(&self, infer_length: usize) -> PolarsResult<Series>
Available on crate feature dtype-decimal
only.
pub fn to_decimal(&self, infer_length: usize) -> PolarsResult<Series>
dtype-decimal
only.Convert an StringChunked
to a Series
of DataType::Decimal
.
Scale needed for the decimal type are inferred. Parsing is not strict.
Scale inference assumes that all tested strings are well-formed numbers,
and may produce unexpected results for scale if this is not the case.
If the decimal precision
and scale
are already known, consider
using the cast
method.
sourcepub fn extend(&mut self, other: &Self)
pub fn extend(&mut self, other: &Self)
Extend the memory backed by this array with the values from other
.
Different from ChunkedArray::append
which adds chunks to this ChunkedArray
extend
appends the data from other
to the underlying PrimitiveArray
and thus may cause a reallocation.
However if this does not cause a reallocation, the resulting data structure will not have any extra chunks and thus will yield faster queries.
Prefer extend
over append
when you want to do a query after a single append. For instance during
online operations where you add n
rows and rerun a query.
Prefer append
over extend
when you want to append many times before doing a query. For instance
when you read in multiple files and when to store them in a single DataFrame
.
In the latter case finish the sequence of append
operations with a rechunk
.
pub fn for_each<'a, F>(&'a self, op: F)
sourcepub fn is_null(&self) -> BooleanChunked
pub fn is_null(&self) -> BooleanChunked
Get a mask of the null values.
sourcepub fn is_not_null(&self) -> BooleanChunked
pub fn is_not_null(&self) -> BooleanChunked
Get a mask of the valid values.
sourcepub fn rolling_map_float<F>(
&self,
window_size: usize,
f: F
) -> PolarsResult<Self>
Available on crate feature rolling_window
only.
pub fn rolling_map_float<F>( &self, window_size: usize, f: F ) -> PolarsResult<Self>
rolling_window
only.Apply a rolling custom function. This is pretty slow because of dynamic dispatch.
sourcepub unsafe fn to_string(&self) -> StringChunked
pub unsafe fn to_string(&self) -> StringChunked
§Safety
String is not validated
pub fn as_binary(&self) -> BinaryChunked
sourcepub fn any(&self) -> bool
pub fn any(&self) -> bool
Returns whether any of the values in the column are true
.
Null values are ignored.
sourcepub fn all(&self) -> bool
pub fn all(&self) -> bool
Returns whether all values in the array are true
.
Null values are ignored.
sourcepub fn any_kleene(&self) -> Option<bool>
pub fn any_kleene(&self) -> Option<bool>
Returns whether any of the values in the column are true
.
The output is unknown (None
) if the array contains any null values and
no true
values.
sourcepub fn all_kleene(&self) -> Option<bool>
pub fn all_kleene(&self) -> Option<bool>
Returns whether all values in the column are true
.
The output is unknown (None
) if the array contains any null values and
no false
values.
pub fn is_nan(&self) -> BooleanChunked
pub fn is_not_nan(&self) -> BooleanChunked
pub fn is_finite(&self) -> BooleanChunked
pub fn is_infinite(&self) -> BooleanChunked
sourcepub fn none_to_nan(&self) -> Self
pub fn none_to_nan(&self) -> Self
Convert missing values to NaN
values.
pub fn to_canonical(&self) -> Self
pub fn par_iter(&self) -> impl ParallelIterator<Item = Option<Series>> + '_
pub fn par_iter_indexed( &mut self ) -> impl IndexedParallelIterator<Item = Option<Series>> + '_
pub fn par_iter_indexed( &self ) -> impl IndexedParallelIterator<Item = Option<&str>>
pub fn par_iter(&self) -> impl ParallelIterator<Item = Option<&str>> + '_
pub fn iter(&self) -> impl PolarsIterator<Item = Option<T::Physical<'_>>>
sourcepub fn to_ndarray(&self) -> PolarsResult<ArrayView1<'_, T::Native>>
Available on crate feature ndarray
only.
pub fn to_ndarray(&self) -> PolarsResult<ArrayView1<'_, T::Native>>
ndarray
only.If data is aligned in a single chunk and has no Null values a zero copy view is returned as an ndarray
sourcepub fn to_ndarray<N>(&self) -> PolarsResult<Array2<N::Native>>where
N: PolarsNumericType,
Available on crate feature ndarray
only.
pub fn to_ndarray<N>(&self) -> PolarsResult<Array2<N::Native>>where
N: PolarsNumericType,
ndarray
only.If all nested Series
have the same length, a 2 dimensional ndarray::Array
is returned.
pub fn to_bytes_hashes<'a>( &'a self, multithreaded: bool, hb: RandomState ) -> Vec<Vec<BytesHash<'a>>>
sourcepub unsafe fn with_chunks(&self, chunks: Vec<ArrayRef>) -> Self
pub unsafe fn with_chunks(&self, chunks: Vec<ArrayRef>) -> Self
§Safety
The Arrow datatype of all chunks must match the PolarsDataType
T
.
sourcepub unsafe fn amortized_iter(
&self
) -> AmortizedListIter<'_, impl Iterator<Item = Option<ArrayBox>> + '_>
pub unsafe fn amortized_iter( &self ) -> AmortizedListIter<'_, impl Iterator<Item = Option<ArrayBox>> + '_>
This is an iterator over a ListChunked
that save allocations.
A Series is:
1. Arc<ChunkedArray>
ChunkedArray is:
2. Vec< 3. ArrayRef>
The ArrayRef we indicated with 3. will be updated during iteration. The Series will be pinned in memory, saving an allocation for
- Arc<..>
- Vec<…>
§Warning
Though memory safe in the sense that it will not read unowned memory, UB, or memory leaks
this function still needs precautions. The returned should never be cloned or taken longer
than a single iteration, as every call on next
of the iterator will change the contents of
that Series.
§Safety
The lifetime of UnstableSeries is bound to the iterator. Keeping it alive longer than the iterator is UB.
sourcepub unsafe fn amortized_iter_with_name(
&self,
name: &str
) -> AmortizedListIter<'_, impl Iterator<Item = Option<ArrayBox>> + '_>
pub unsafe fn amortized_iter_with_name( &self, name: &str ) -> AmortizedListIter<'_, impl Iterator<Item = Option<ArrayBox>> + '_>
§Safety
The lifetime of UnstableSeries is bound to the iterator. Keeping it alive longer than the iterator is UB.
sourcepub fn apply_amortized_generic<'a, F, K, V>(&'a self, f: F) -> ChunkedArray<V>where
V: PolarsDataType,
F: FnMut(Option<UnstableSeries<'a>>) -> Option<K> + Copy,
V::Array: ArrayFromIter<Option<K>>,
pub fn apply_amortized_generic<'a, F, K, V>(&'a self, f: F) -> ChunkedArray<V>where
V: PolarsDataType,
F: FnMut(Option<UnstableSeries<'a>>) -> Option<K> + Copy,
V::Array: ArrayFromIter<Option<K>>,
Apply a closure F
elementwise.
pub fn try_apply_amortized_generic<'a, F, K, V>(
&'a self,
f: F
) -> PolarsResult<ChunkedArray<V>>where
V: PolarsDataType,
F: FnMut(Option<UnstableSeries<'a>>) -> PolarsResult<Option<K>> + Copy,
V::Array: ArrayFromIter<Option<K>>,
pub fn for_each_amortized<'a, F>(&'a self, f: F)
sourcepub fn zip_and_apply_amortized<'a, T, I, F>(
&'a self,
ca: &'a ChunkedArray<T>,
f: F
) -> Selfwhere
T: PolarsDataType,
&'a ChunkedArray<T>: IntoIterator<IntoIter = I>,
I: TrustedLen<Item = Option<T::Physical<'a>>>,
F: FnMut(Option<UnstableSeries<'a>>, Option<T::Physical<'a>>) -> Option<Series>,
pub fn zip_and_apply_amortized<'a, T, I, F>(
&'a self,
ca: &'a ChunkedArray<T>,
f: F
) -> Selfwhere
T: PolarsDataType,
&'a ChunkedArray<T>: IntoIterator<IntoIter = I>,
I: TrustedLen<Item = Option<T::Physical<'a>>>,
F: FnMut(Option<UnstableSeries<'a>>, Option<T::Physical<'a>>) -> Option<Series>,
Zip with a ChunkedArray
then apply a binary function F
elementwise.
pub fn binary_zip_and_apply_amortized<'a, T, U, F>(
&'a self,
ca1: &'a ChunkedArray<T>,
ca2: &'a ChunkedArray<U>,
f: F
) -> Selfwhere
T: PolarsDataType,
U: PolarsDataType,
F: FnMut(Option<UnstableSeries<'a>>, Option<T::Physical<'a>>, Option<U::Physical<'a>>) -> Option<Series>,
pub fn try_zip_and_apply_amortized<'a, T, I, F>(
&'a self,
ca: &'a ChunkedArray<T>,
f: F
) -> PolarsResult<Self>where
T: PolarsDataType,
&'a ChunkedArray<T>: IntoIterator<IntoIter = I>,
I: TrustedLen<Item = Option<T::Physical<'a>>>,
F: FnMut(Option<UnstableSeries<'a>>, Option<T::Physical<'a>>) -> PolarsResult<Option<Series>>,
sourcepub fn apply_amortized<'a, F>(&'a self, f: F) -> Self
pub fn apply_amortized<'a, F>(&'a self, f: F) -> Self
Apply a closure F
elementwise.
pub fn try_apply_amortized<'a, F>(&'a self, f: F) -> PolarsResult<Self>
sourcepub fn inner_dtype(&self) -> DataType
pub fn inner_dtype(&self) -> DataType
Get the inner data type of the list.
pub fn set_inner_dtype(&mut self, dtype: DataType)
pub fn set_fast_explode(&mut self)
pub fn _can_fast_explode(&self) -> bool
sourcepub unsafe fn to_logical(&mut self, inner_dtype: DataType)
pub unsafe fn to_logical(&mut self, inner_dtype: DataType)
Set the logical type of the ListChunked
.
§Safety
The caller must ensure that the logical type given fits the physical type of the array.
sourcepub fn apply_to_inner(
&self,
func: &dyn Fn(Series) -> PolarsResult<Series>
) -> PolarsResult<ListChunked>
pub fn apply_to_inner( &self, func: &dyn Fn(Series) -> PolarsResult<Series> ) -> PolarsResult<ListChunked>
Ignore the list indices and apply func
to the inner type as Series
.
sourcepub unsafe fn get_object_unchecked(
&self,
index: usize
) -> Option<&dyn PolarsObjectSafe>
Available on crate feature object
only.
pub unsafe fn get_object_unchecked( &self, index: usize ) -> Option<&dyn PolarsObjectSafe>
object
only.Get a hold to an object that can be formatted or downcasted via the Any trait.
§Safety
No bounds checks
sourcepub fn get_object(&self, index: usize) -> Option<&dyn PolarsObjectSafe>
Available on crate feature object
only.
pub fn get_object(&self, index: usize) -> Option<&dyn PolarsObjectSafe>
object
only.Get a hold to an object that can be formatted or downcasted via the Any trait.
sourcepub fn sample_n(
&self,
n: usize,
with_replacement: bool,
shuffle: bool,
seed: Option<u64>
) -> PolarsResult<Self>
Available on crate feature random
only.
pub fn sample_n( &self, n: usize, with_replacement: bool, shuffle: bool, seed: Option<u64> ) -> PolarsResult<Self>
random
only.Sample n datapoints from this ChunkedArray
.
sourcepub fn sample_frac(
&self,
frac: f64,
with_replacement: bool,
shuffle: bool,
seed: Option<u64>
) -> PolarsResult<Self>
Available on crate feature random
only.
pub fn sample_frac( &self, frac: f64, with_replacement: bool, shuffle: bool, seed: Option<u64> ) -> PolarsResult<Self>
random
only.Sample a fraction between 0.0-1.0 of this ChunkedArray
.
sourcepub fn to_vec_null_aware(
&self
) -> Either<Vec<T::Native>, Vec<Option<T::Native>>>
pub fn to_vec_null_aware( &self ) -> Either<Vec<T::Native>, Vec<Option<T::Native>>>
Convert to a Vec
but don’t return Option<T::Native>
if there are no null values
pub fn unset_fast_explode_list(&mut self)
pub fn get_flags(&self) -> Settings
pub fn is_sorted_flag(&self) -> IsSorted
sourcepub fn set_sorted_flag(&mut self, sorted: IsSorted)
pub fn set_sorted_flag(&mut self, sorted: IsSorted)
Set the ‘sorted’ bit meta info.
sourcepub fn with_sorted_flag(&self, sorted: IsSorted) -> Self
pub fn with_sorted_flag(&self, sorted: IsSorted) -> Self
Set the ‘sorted’ bit meta info.
sourcepub fn first_non_null(&self) -> Option<usize>
pub fn first_non_null(&self) -> Option<usize>
Get the index of the first non null value in this ChunkedArray
.
sourcepub fn last_non_null(&self) -> Option<usize>
pub fn last_non_null(&self) -> Option<usize>
Get the index of the last non null value in this ChunkedArray
.
sourcepub fn iter_validities(
&self
) -> Map<Iter<'_, ArrayRef>, fn(_: &ArrayRef) -> Option<&Bitmap>> ⓘ
pub fn iter_validities( &self ) -> Map<Iter<'_, ArrayRef>, fn(_: &ArrayRef) -> Option<&Bitmap>> ⓘ
Get the buffer of bits representing null values
sourcepub fn has_validity(&self) -> bool
pub fn has_validity(&self) -> bool
Return if any the chunks in this ChunkedArray
have a validity bitmap.
no bitmap means no null values.
sourcepub fn shrink_to_fit(&mut self)
pub fn shrink_to_fit(&mut self)
Shrink the capacity of this array to fit its length.
pub fn clear(&self) -> Self
sourcepub fn unpack_series_matching_type(
&self,
series: &Series
) -> PolarsResult<&ChunkedArray<T>>
pub fn unpack_series_matching_type( &self, series: &Series ) -> PolarsResult<&ChunkedArray<T>>
Series to ChunkedArray<T>
sourcepub fn chunk_id(&self) -> ChunkIdIter<'_>
pub fn chunk_id(&self) -> ChunkIdIter<'_>
Unique id representing the number of chunks
sourcepub unsafe fn chunks_mut(&mut self) -> &mut Vec<ArrayRef>
pub unsafe fn chunks_mut(&mut self) -> &mut Vec<ArrayRef>
sourcepub fn is_optimal_aligned(&self) -> bool
pub fn is_optimal_aligned(&self) -> bool
Returns true if contains a single chunk and has no null values
sourcepub fn dtype(&self) -> &DataType
pub fn dtype(&self) -> &DataType
Get data type of ChunkedArray
.
sourcepub fn name(&self) -> &str
pub fn name(&self) -> &str
Name of the ChunkedArray
.
sourcepub fn rename(&mut self, name: &str)
pub fn rename(&mut self, name: &str)
Rename this ChunkedArray
.
sourcepub fn get(&self, idx: usize) -> Option<T::Physical<'_>>
pub fn get(&self, idx: usize) -> Option<T::Physical<'_>>
Get a single value from this ChunkedArray
. If the return values is None
this
indicates a NULL value.
§Panics
This function will panic if idx
is out of bounds.
sourcepub unsafe fn get_unchecked(&self, idx: usize) -> Option<T::Physical<'_>>
pub unsafe fn get_unchecked(&self, idx: usize) -> Option<T::Physical<'_>>
Get a single value from this ChunkedArray
. If the return values is None
this
indicates a NULL value.
§Safety
It is the callers responsibility that the idx < self.len()
.
sourcepub unsafe fn value_unchecked(&self, idx: usize) -> T::Physical<'_>
pub unsafe fn value_unchecked(&self, idx: usize) -> T::Physical<'_>
Get a single value from this ChunkedArray
. Null values are ignored and the returned
value could be garbage if it was masked out by NULL. Note that the value always is initialized.
§Safety
It is the callers responsibility that the idx < self.len()
.
pub fn last(&self) -> Option<T::Physical<'_>>
pub fn get_as_series(&self, idx: usize) -> Option<Series>
pub fn layout(&self) -> ChunkedArrayLayout<'_, T>
sourcepub fn cont_slice(&self) -> PolarsResult<&[T::Native]>
pub fn cont_slice(&self) -> PolarsResult<&[T::Native]>
Contiguous slice
sourcepub fn data_views(&self) -> impl DoubleEndedIterator<Item = &[T::Native]>
pub fn data_views(&self) -> impl DoubleEndedIterator<Item = &[T::Native]>
Get slices of the underlying arrow data. NOTE: null values should be taken into account by the user of these slices as they are handled separately
pub fn into_no_null_iter( &self ) -> impl '_ + Send + Sync + ExactSizeIterator<Item = T::Native> + DoubleEndedIterator + TrustedLen
pub fn group_tuples_perfect( &self, max: usize, multithreaded: bool, group_capacity: usize ) -> GroupsProxy
algorithm_group_by
only.Trait Implementations§
source§impl<K: PolarsDataType, T: PolarsDataType> Clone for Logical<K, T>
impl<K: PolarsDataType, T: PolarsDataType> Clone for Logical<K, T>
source§impl<K: PolarsDataType, T: PolarsDataType> Deref for Logical<K, T>
impl<K: PolarsDataType, T: PolarsDataType> Deref for Logical<K, T>
source§impl<K: PolarsDataType, T: PolarsDataType> DerefMut for Logical<K, T>
impl<K: PolarsDataType, T: PolarsDataType> DerefMut for Logical<K, T>
Auto Trait Implementations§
impl<Logical, Physical> Freeze for Logical<Logical, Physical>
impl<Logical, Physical> !RefUnwindSafe for Logical<Logical, Physical>
impl<Logical, Physical> Send for Logical<Logical, Physical>
impl<Logical, Physical> Sync for Logical<Logical, Physical>
impl<Logical, Physical> Unpin for Logical<Logical, Physical>
impl<Logical, Physical> !UnwindSafe for Logical<Logical, Physical>
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more