Module arrow::ffi_stream

Contains declarations to bind to the C Stream Interface.

This module has two main interfaces: One interface maps C ABI to native Rust types, i.e. convert c-pointers, c_char, to native rust. This is handled by FFI_ArrowArrayStream.

The second interface is used to import FFI_ArrowArrayStream as Rust implementation RecordBatch reader. This is handled by ArrowArrayStreamReader.

// create an record batch reader natively
let file = File::open("arrow_file").unwrap();
let reader = Box::new(FileReader::try_new(file).unwrap());

// export it
let stream = Box::new(FFI_ArrowArrayStream::empty());
let stream_ptr = Box::into_raw(stream) as *mut FFI_ArrowArrayStream;
unsafe { export_reader_into_raw(reader, stream_ptr) };

// consumed and used by something else...

// import it
let stream_reader = unsafe { ArrowArrayStreamReader::from_raw(stream_ptr).unwrap() };
let imported_schema = stream_reader.schema();

let mut produced_batches = vec![];
for batch in stream_reader {
     produced_batches.push(batch.unwrap());
}

// (drop/release)
unsafe {
  Box::from_raw(stream_ptr);
}
Ok(())
}

Structs

ArrowArrayStreamReader

A RecordBatchReader which imports Arrays from FFI_ArrowArrayStream. Struct used to fetch RecordBatch from the C Stream Interface. Its main responsibility is to expose RecordBatchReader functionality that requires FFI_ArrowArrayStream.

FFI_ArrowArrayStream

ABI-compatible struct for ArrayStream from C Stream Interface See https://arrow.apache.org/docs/format/CStreamInterface.html#structure-definitions This was created by bindgen

Functions

export_reader_into_raw^⚠

Exports a record batch reader to raw pointer of the C Stream Interface provided by the consumer.