Module arrow_c_ffi 

Arrow-C-FFI Module - Share data to another language and/or run-time*

Implements the Apache Arrow C Data Interface for Minarrow, enabling zero-copy data exchange across language boundaries.
Compatible with any runtime implementing the Arrow C interface, including Python, C++, Java, and others.

Features

• Export: Convert Minarrow arrays into Arrow C ArrowArray and ArrowSchema pointers.
• Import: Construct Minarrow arrays directly from Arrow C pointers.
• Type fidelity: Preserves physical layout, logical type metadata, and temporal units.
• FFI safety: Backing buffers are reference-counted to ensure lifetime validity across the boundary, with explicit release functions for deallocation.

Usage

1. Ensure the array has an associated Field or is wrapped in a FieldArray.
2. For temporal arrays, set the correct ArrowType with appropriate TimeUnit or IntervalUnit.
3. Call export_to_c to obtain Arrow-compatible pointers for FFI transmission.
4. On the receiving side, reconstruct a Minarrow array with import_from_c.

Examples

• Rust -> Arrow: cargo run --example apache-ffi
• Planned: Python, C++, and other language bindings.

Notes

• Dictionary-encoded (categorical) arrays are supported, including index type mapping.
• UTF-8 and large UTF-8 string arrays preserve offset and value buffer ordering.
• Temporal arrays validate logical type <-> physical storage alignment prior to export.
• pyo3 normally abstracts pointer handling and lifetime management when integrating with Python; we do not yet use it, but once integrated, instead of manual Arc reference count handling and explicit clean-up, one will be able to instead leverage automatic, Python-owned lifetimes.

Trademark Notice

Apache Arrow is a trademark of the Apache Software Foundation, used here under fair-use to implement its published interoperability standard as per https://www.apache.org/foundation/marks/ .

Structs

ArrowArray

ArrowArray as per the Arrow C spec

ArrowArrayStream

ArrowArrayStream as per the Arrow C Stream Interface spec.

ArrowSchema

ArrowSchema as per the Arrow C spec

Functions

decode_arrow_metadata^⚠

Decodes Arrow C Data Interface metadata from a raw pointer into key-value pairs.

export_array_stream

Creates an ArrowArrayStream that yields plain arrays (one per chunk).

export_record_batch_stream

Creates an ArrowArrayStream that yields record batches as struct arrays.

export_record_batch_stream_with_metadata

Like export_record_batch_stream, but attaches key-value metadata to the struct-level schema. Use this to preserve a table name or other metadata across the FFI boundary.

export_struct_to_c

Exports a single record batch (a set of column arrays with a shared schema) as an Arrow struct array + struct schema pair.

export_to_c

Exports a Minarrow array to Arrow C Data Interface pointers.

fmt_c

Constructs the Arrow C FFI format string for the given ArrowType.

import_array_stream^⚠

Consumes an ArrowArrayStream that yields plain arrays and returns the imported arrays along with the field metadata.

import_from_c^⚠

Imports a Minarrow array from ArrowArray and ArrowSchema C pointers.

import_from_c_owned^⚠

Imports a Minarrow array from owned ArrowArray and ArrowSchema C pointers.

import_record_batch_stream^⚠

Consumes an ArrowArrayStream that yields struct arrays (record batches) and returns a list of Tables.

import_record_batch_stream_with_metadata^⚠

Like import_record_batch_stream, but also returns the schema-level metadata as key-value pairs. Returns None for metadata when the schema has no metadata attached.