Module writer 

JSON Writer

This JSON writer converts Arrow RecordBatches into arrays of JSON objects or JSON formatted byte streams.

Writing JSON formatted byte streams

To serialize RecordBatches into line-delimited JSON bytes, use LineDelimitedWriter:

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as JSON
let buf = Vec::new();
let mut writer = arrow_json::LineDelimitedWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();

// Get the underlying buffer back,
let buf = writer.into_inner();
assert_eq!(r#"{"a":1}
{"a":2}
{"a":3}
"#, String::from_utf8(buf).unwrap())

To serialize RecordBatches into a well formed JSON array, use ArrayWriter:

use arrow_schema::{DataType, Field, Schema};

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as a JSON array
let buf = Vec::new();
let mut writer = arrow_json::ArrayWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();

// Get the underlying buffer back,
let buf = writer.into_inner();
assert_eq!(r#"[{"a":1},{"a":2},{"a":3}]"#, String::from_utf8(buf).unwrap())

LineDelimitedWriter and ArrayWriter will omit writing keys with null values. In order to explicitly write null values for keys, configure a custom Writer by using a WriterBuilder to construct a Writer.

Writing to serde_json JSON Objects

To serialize RecordBatches into an array of JSON objects you can reparse the resulting JSON string. Note that this is less efficient than using the Writer API.

let schema = Schema::new(vec![Field::new("a", DataType::Int32, false)]);
let a = Int32Array::from(vec![1, 2, 3]);
let batch = RecordBatch::try_new(Arc::new(schema), vec![Arc::new(a)]).unwrap();

// Write the record batch out as json bytes (string)
let buf = Vec::new();
let mut writer = arrow_json::ArrayWriter::new(buf);
writer.write_batches(&vec![&batch]).unwrap();
writer.finish().unwrap();
let json_data = writer.into_inner();

// Parse the string using serde_json
use serde_json::{Map, Value};
let json_rows: Vec<Map<String, Value>> = serde_json::from_reader(json_data.as_slice()).unwrap();
assert_eq!(
    serde_json::Value::Object(json_rows[1].clone()),
    serde_json::json!({"a": 2}),
);

Customizing the encoder

The output produced for each data type can be customized using WriterBuilder::with_encoder_factory. For example, you can override the default hex encoding of binary data to use Base64 instead, or provide encoders for types with no built-in encoding, such as unions. See the example on EncoderFactory.

Structs

EncoderOptions

Configuration options for the JSON encoder.

JsonArray

Produces JSON output as a single JSON array.

LineDelimited

Produces JSON output with one record per line.

NullableEncoder

An encoder + a null buffer. This is packaged together into a wrapper struct to minimize dynamic dispatch for null checks.

Writer

A JSON writer which serializes RecordBatches to a stream of u8 encoded JSON objects.

WriterBuilder

JSON writer builder.

Traits

Encoder

A trait to format array values as JSON values

EncoderFactory

Creates custom encoders for specific data types when writing JSON data.

JsonFormat

This trait defines how to format a sequence of JSON objects to a byte stream.

Functions

make_encoder

Creates an encoder for the given array and field.

Type Aliases

ArrayWriter

A JSON writer which serializes RecordBatches to JSON arrays.

LineDelimitedWriter

A JSON writer which serializes RecordBatches to newline delimited JSON objects.