Struct arrow::buffer::Buffer

pub struct Buffer { /* fields omitted */ }

Buffer represents a contiguous memory region that can be shared with other buffers and across thread boundaries.

Implementations

impl Buffer

pub fn from_bytes(bytes: Bytes) -> Self

Auxiliary method to create a new Buffer

pub fn from_slice_ref<U: ArrowNativeType, T: AsRef<[U]>>(items: &T) -> Self

Initializes a Buffer from a slice of items.

pub unsafe fn from_raw_parts(
    ptr: NonNull<u8>,
    len: usize,
    capacity: usize
) -> Self

Creates a buffer from an existing memory region (must already be byte-aligned), this Buffer will free this piece of memory when dropped.

Arguments

• ptr - Pointer to raw parts
• len - Length of raw parts in bytes
• capacity - Total allocated memory for the pointer ptr, in bytes

Safety

This function is unsafe as there is no guarantee that the given pointer is valid for len bytes. If the ptr and capacity come from a Buffer, then this is guaranteed.

pub unsafe fn from_unowned(
    ptr: NonNull<u8>,
    len: usize,
    data: Arc<FFI_ArrowArray>
) -> Self

Creates a buffer from an existing memory region (must already be byte-aligned), this Buffer does not free this piece of memory when dropped.

Arguments

• ptr - Pointer to raw parts
• len - Length of raw parts in bytes
• data - An ffi::FFI_ArrowArray with the data

Safety

This function is unsafe as there is no guarantee that the given pointer is valid for len bytes and that the foreign deallocator frees the region.

pub fn len(&self) -> usize

Returns the number of bytes in the buffer

pub fn capacity(&self) -> usize

Returns the capacity of this buffer. For exernally owned buffers, this returns zero

pub fn is_empty(&self) -> bool

Returns whether the buffer is empty.

pub fn as_slice(&self) -> &[u8]

Returns the byte slice stored in this buffer

pub fn slice(&self, offset: usize) -> Self

Returns a new Buffer that is a slice of this buffer starting at offset. Doing so allows the same memory region to be shared between buffers.

Panics

Panics iff offset is larger than len.

pub fn as_ptr(&self) -> *const u8

Returns a pointer to the start of this buffer.

Note that this should be used cautiously, and the returned pointer should not be stored anywhere, to avoid dangling pointers.

pub unsafe fn typed_data<T: ArrowNativeType + Num>(&self) -> &[T]

View buffer as typed slice.

Safety

ArrowNativeType is public so that it can be used as a trait bound for other public components, such as the ToByteSlice trait. However, this means that it can be implemented by user defined types, which it is not intended for.

Also typed_data::<bool> is unsafe as 0x00 and 0x01 are the only valid values for bool in Rust. However, bool arrays in Arrow are bit-packed which breaks this condition. View buffer as typed slice.

pub fn bit_slice(&self, offset: usize, len: usize) -> Self

Returns a slice of this buffer starting at a certain bit offset. If the offset is byte-aligned the returned buffer is a shallow clone, otherwise a new buffer is allocated and filled with a copy of the bits in the range.

pub fn bit_chunks(&self, offset: usize, len: usize) -> BitChunks<'_>

Returns a BitChunks instance which can be used to iterate over this buffers bits in larger chunks and starting at arbitrary bit offsets. Note that both offset and length are measured in bits.

pub fn count_set_bits(&self) -> usize

Returns the number of 1-bits in this buffer.

pub fn count_set_bits_offset(&self, offset: usize, len: usize) -> usize

Returns the number of 1-bits in this buffer, starting from offset with length bits inspected. Note that both offset and length are measured in bits.

impl Buffer

pub unsafe fn from_trusted_len_iter<T: ArrowNativeType, I: Iterator<Item = T>>(
    iterator: I
) -> Self

Creates a Buffer from an Iterator with a trusted (upper) length. Prefer this to collect whenever possible, as it is ~60% faster.

Example

let v = vec![1u32];
let iter = v.iter().map(|x| x * 2);
let buffer = unsafe { Buffer::from_trusted_len_iter(iter) };
assert_eq!(buffer.len(), 4) // u32 has 4 bytes

Safety

This method assumes that the iterator’s size is correct and is undefined behavior to use it on an iterator that reports an incorrect length.

pub unsafe fn try_from_trusted_len_iter<E, T: ArrowNativeType, I: Iterator<Item = Result<T, E>>>(
    iterator: I
) -> Result<Self, E>

Creates a Buffer from an Iterator with a trusted (upper) length or errors if any of the items of the iterator is an error. Prefer this to collect whenever possible, as it is ~60% faster.

Safety

This method assumes that the iterator’s size is correct and is undefined behavior to use it on an iterator that reports an incorrect length.

Trait Implementations

impl<'a, 'b> BitAnd<&'b Buffer> for &'a Buffer

type Output = Result<Buffer>

The resulting type after applying the & operator.

fn bitand(self, rhs: &'b Buffer) -> Result<Buffer>

Performs the & operation.

impl<'a, 'b> BitOr<&'b Buffer> for &'a Buffer

type Output = Result<Buffer>

The resulting type after applying the | operator.

fn bitor(self, rhs: &'b Buffer) -> Result<Buffer>

Performs the | operation.

impl Clone for Buffer

fn clone(&self) -> Buffer

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Buffer

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Deref for Buffer

type Target = [u8]

The resulting type after dereferencing.

fn deref(&self) -> &[u8]

Dereferences the value.

impl From<BooleanBufferBuilder> for Buffer

fn from(builder: BooleanBufferBuilder) -> Self

Performs the conversion.

impl From<Buffer> for Bitmap

fn from(buf: Buffer) -> Self

Performs the conversion.

impl From<MutableBuffer> for Buffer

fn from(buffer: MutableBuffer) -> Self

Performs the conversion.

impl<T: AsRef<[u8]>> From<T> for Buffer

Creating a Buffer instance by copying the memory from a AsRef<[u8]> into a newly allocated memory region.

fn from(p: T) -> Self

Performs the conversion.

impl<T: ArrowNativeType> FromIterator<T> for Buffer

fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self

Creates a value from an iterator.

impl FromIterator<bool> for Buffer

Creating a Buffer instance by storing the boolean values into the buffer

fn from_iter<I>(iter: I) -> Self where
    I: IntoIterator<Item = bool>, 

Creates a value from an iterator.

impl Not for &Buffer

type Output = Buffer

The resulting type after applying the ! operator.

fn not(self) -> Buffer

Performs the unary ! operation.

impl PartialEq<Buffer> for Buffer

fn eq(&self, other: &Buffer) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Buffer) -> bool

This method tests for !=.

impl Send for Buffer

impl StructuralPartialEq for Buffer

impl Sync for Buffer