json-slabs 0.2.0

Read and write JSLB (JsonSlabs) binary container files.
Documentation
  • Coverage
  • 100%
    72 out of 72 items documented1 out of 1 items with examples
  • Size
  • Source code size: 68.3 kB This is the summed size of all the files inside the crates.io package for this release.
  • Documentation size: 698.1 kB This is the summed size of all files generated by rustdoc for all configured targets
  • Ø build duration
  • this release: 5s Average build duration of successful builds.
  • all releases: 6s Average build duration of successful builds in releases after 2024-10-23.
  • Links
  • Homepage
  • mstange/json-slabs
    3 2 0
  • crates.io
  • Dependencies
  • Versions
  • Owners
  • mstange

json-slabs

crates.io docs.rs

Read and write JsonSlabs (JSLB) files — a binary container format for JSON-shaped data with embedded typed-array slabs. The array data is stored out-of-line and at its natural alignment, so it can be read as a zero-copy view into the file buffer and its size doesn't affect the parsing cost of the object structure.

This crate contains no JSON parser or writer. For JSON slabs it works with raw bytes and leaves parsing and placeholder substitution to the consumer.

Writing

use json_slabs::Builder;

let mut b = Builder::new();
let numbers = b.add_slab(&[10i32, 20, 30]);
let sub_json = b.add_json_slab(br#"["hello", "world"]"#.to_vec());
// `{:#}` on a placeholder prints the whole `{"$s": N}` reference.
let skeleton = format!(r#"{{"numbers":{numbers:#},"splitOutArray":{sub_json:#}}}"#);
let bytes = b.finish(skeleton.as_bytes());

The slice passed to Builder::add_slab needs to outlive the builder. You can use Builder::add_slab_from_vec(values) instead if you want the builder to hold on to the data.

For columns that don't already exist in memory, use Builder::add_slab_from_iter(count, iter). The iterator is consumed during Builder::finish / Builder::to_writer when the file bytes are written.

Reading

use json_slabs::{ParsedFile, SlabPlaceholder};

let parsed = ParsedFile::parse(&bytes).unwrap();
// The root JSON slab, as raw JSON bytes:
// br#"{"numbers":{"$s":1},"splitOutArray":{"$s":2}}"#
let root: &[u8] = parsed.root_json_bytes();
// In practice you'd parse that JSON and get the slab index from the
// parsed `{"$s": N}` placeholder.
let numbers: Vec<i32> = parsed.read(SlabPlaceholder(1)).unwrap();
assert_eq!(numbers, vec![10, 20, 30]);

See the API docs for the full API, and FORMAT.md for the on-disk layout.

License

MIT. See LICENSE.