pub fn decode_arrow_ipc_zero_copy(payload: IpcPayload) -> Result<DoraArray>Expand description
Decode an Arrow IPC stream from an Arrow [Buffer] without copying the
payload buffers when they are properly aligned.
Unlike decode_arrow_ipc, which reads from a byte slice through
StreamReader (and therefore allocates a fresh buffer and copies every
array buffer out of the stream), this uses
arrow::ipc::reader::StreamDecoder, which slices the array buffers
directly out of the provided [Buffer]. When the input buffer is suitably
aligned — as Dora’s shared-memory payloads always are (128-byte AVec /
page-aligned Zenoh SHM) — the decoded array aliases the input and no payload
copy happens.
The decoder runs with the default require_alignment = false, so an
under-aligned input (e.g. an arbitrary heap Vec) is handled gracefully by
copying just the misaligned buffers rather than erroring. This keeps the
receive path robust while preserving zero-copy for the common SHM case.
§Errors
Like decode_arrow_ipc, returns an error for an empty, truncated, or
otherwise malformed stream, for a batch whose column count is not exactly
one, and for any payload larger than 256 MB — it never panics on bad input.
§Example
use dora_node_api::IntoArrow;
use dora_node_api::arrow_utils::{IpcPayload, decode_arrow_ipc_zero_copy, encode_arrow_ipc};
let ipc = encode_arrow_ipc(&vec![1u64, 2, 3].into_arrow())?;
let decoded = decode_arrow_ipc_zero_copy(IpcPayload::from_vec(ipc))?;
let values: Vec<u64> = (&decoded).try_into()?;
assert_eq!(values, vec![1, 2, 3]);
// A malformed stream is rejected with an error rather than panicking.
assert!(decode_arrow_ipc_zero_copy(IpcPayload::from_vec(vec![0u8; 8])).is_err());