Crate json

Transfer data between the Arrow memory format and JSON line-delimited records.

See the module level documentation for the reader and writer for usage examples.

Binary Data

As per RFC7159 JSON cannot encode arbitrary binary data. A common approach to workaround this is to use a binary-to-text encoding scheme, such as base64, to encode the input data and then decode it on output.

// The data we want to write
let input = BinaryArray::from(vec![b"\xDE\x00\xFF".as_ref()]);

// Base64 encode it to a string
let encoded: StringArray = b64_encode(&BASE64_STANDARD, &input);

// Write the StringArray to JSON
let batch = RecordBatch::try_from_iter([("col", Arc::new(encoded) as _)]).unwrap();
let mut buf = Vec::with_capacity(1024);
let mut writer = LineDelimitedWriter::new(&mut buf);
writer.write(&batch).unwrap();
writer.finish().unwrap();

// Read the JSON data
let cursor = Cursor::new(buf);
let mut reader = ReaderBuilder::new(batch.schema()).build(cursor).unwrap();
let batch = reader.next().unwrap().unwrap();

// Reverse the base64 encoding
let col: BinaryArray = batch.column(0).as_string::<i32>().clone().into();
let output = b64_decode(&BASE64_STANDARD, &col).unwrap();

assert_eq!(input, output);

Modules

reader

JSON reader

writer

JSON Writer

Structs

Reader

Reads JSON data with a known schema directly into arrow RecordBatch

ReaderBuilder

A builder for Reader and Decoder

Writer

A JSON writer which serializes RecordBatches to a stream of u8 encoded JSON objects.

WriterBuilder

JSON writer builder.

Enums

StructMode

Specifies what is considered valid JSON when reading or writing RecordBatches or StructArrays.

Traits

JsonSerializable

Trait declaring any type that is serializable to JSON. This includes all primitive types (bool, i32, etc.).

Type Aliases

ArrayWriter

A JSON writer which serializes RecordBatches to JSON arrays.

LineDelimitedWriter

A JSON writer which serializes RecordBatches to newline delimited JSON objects.