Module arrow::ffi_stream
source · [−]Expand description
Contains declarations to bind to the C Stream Interface.
This module has two main interfaces: One interface maps C ABI to native Rust types, i.e. convert c-pointers, c_char, to native rust. This is handled by FFI_ArrowArrayStream.
The second interface is used to import FFI_ArrowArrayStream
as Rust implementation RecordBatch
reader.
This is handled by ArrowArrayStreamReader
.
// create an record batch reader natively
let file = File::open("arrow_file").unwrap();
let reader = Box::new(FileReader::try_new(file).unwrap());
// export it
let stream = Box::new(FFI_ArrowArrayStream::empty());
let stream_ptr = Box::into_raw(stream) as *mut FFI_ArrowArrayStream;
unsafe { export_reader_into_raw(reader, stream_ptr) };
// consumed and used by something else...
// import it
let stream_reader = unsafe { ArrowArrayStreamReader::from_raw(stream_ptr).unwrap() };
let imported_schema = stream_reader.schema();
let mut produced_batches = vec![];
for batch in stream_reader {
produced_batches.push(batch.unwrap());
}
// (drop/release)
unsafe {
Box::from_raw(stream_ptr);
}
Ok(())
}
Structs
A RecordBatchReader
which imports Arrays from FFI_ArrowArrayStream
.
Struct used to fetch RecordBatch
from the C Stream Interface.
Its main responsibility is to expose RecordBatchReader
functionality
that requires FFI_ArrowArrayStream.
ABI-compatible struct for ArrayStream
from C Stream Interface
See https://arrow.apache.org/docs/format/CStreamInterface.html#structure-definitions
This was created by bindgen
Functions
Exports a record batch reader to raw pointer of the C Stream Interface provided by the consumer.