Actson

Actson is a reactive JSON parser (sometimes referred to as non-blocking or asynchronous). It is event-based and can be used in asynchronous code (for example in combination with Tokio).

Why another JSON parser?

• Non-blocking. Other JSON parsers use blocking I/O. If you want to develop a reactive application you should use non-blocking I/O (see the Reactive Manifesto).
• Big Data. Most parsers read the full JSON text into memory to map it to a struct. Actson can handle arbitrarily large JSON text. It is event-based and can be used for streaming.
• GeoRocket. Actson was primarily developed for the GeoJSON support in GeoRocket, a high-performance reactive data store for geospatial files.

Usage

Push-based parsing

Push-based parsing is the most flexible way of using Actson. Push new bytes into a PushJsonFeeder and then let the parser consume them until it returns JsonEvent::NeedMoreInput. Repeat this process until you receive JsonEvent::Eof or JsonEvent::Error.

This approach is very low-level but gives you the freedom to provide new bytes to the parser whenever they are available and to generate JSON events whenever you need them.

use actson::{JsonParser, JsonEvent};
use actson::feeder::{PushJsonFeeder, JsonFeeder};

let json = r#"{"name": "Elvis"}"#.as_bytes();

let feeder = PushJsonFeeder::new();
let mut parser = JsonParser::new(feeder);
let mut i = 0;
loop {
    // feed as many bytes as possible to the parser
    let mut event = parser.next_event();
    while event == JsonEvent::NeedMoreInput {
        i += parser.feeder.push_bytes(&json[i..]);
        if i == json.len() {
            parser.feeder.done();
        }
        event = parser.next_event();
    }

    // process event
    match event {
        JsonEvent::FieldName => assert!(matches!(parser.current_str(), Ok("name"))),
        JsonEvent::ValueString => assert!(matches!(parser.current_str(), Ok("Elvis"))),
        JsonEvent::Eof => break,
        JsonEvent::Error(kind) => panic!("Parser error: {:?}", kind),
        _ => {} // there are many other event types you may process here
    }
}

Asynchronous parsing with Tokio

Actson can be used with Tokio to parse JSON asynchronously.

The main idea here is to call JsonParser::next_event() in a loop to parse the JSON document and to produce events. Whenever you get JsonEvent::NeedMoreInput, call AsyncBufReaderJsonFeeder::fill_buf() to asynchronously read more bytes from the input and to provide them to the parser.

Heads up: The tokio feature has to be enabled for this. It is disabled by default.

use tokio::fs::File;
use tokio::io::{self, AsyncReadExt, BufReader};

use actson::{JsonParser, JsonEvent};
use actson::tokio::AsyncBufReaderJsonFeeder;

#[tokio::main]
async fn main() {
    let file = File::open("tests/fixtures/pass1.txt").await.unwrap();
    let reader = BufReader::new(file);

    let feeder = AsyncBufReaderJsonFeeder::new(reader);
    let mut parser = JsonParser::new(feeder);
    loop {
        let mut event = parser.next_event();
        if event == JsonEvent::NeedMoreInput {
            parser.feeder.fill_buf().await.unwrap();
            event = parser.next_event();
        }

        match event {
            JsonEvent::Eof => break,
            JsonEvent::Error(kind) => panic!("Parser error: {:?}", kind),
            _ => {} // do something useful with the event
        }
    }
}

Parsing from a BufReader

BufReaderJsonFeeder allows you to feed the parser from a std::io::BufReader.

Note: By following this synchronous and blocking approach, you are missing out on Actson's reactive properties. We recommend using Actson together with Tokio instead to parse JSON asynchronously (see above).

use actson::{JsonParser, JsonEvent};

use std::fs::File;
use std::io::BufReader;

let file = File::open("tests/fixtures/pass1.txt").unwrap();
let reader = BufReader::new(file);

let feeder = actson::feeder::BufReaderJsonFeeder::new(reader);
let mut parser = JsonParser::new(feeder);
loop {
    let mut event = parser.next_event();
    if event == JsonEvent::NeedMoreInput {
        parser.feeder.fill_buf().unwrap();
        event = parser.next_event();
    }

    match event {
        JsonEvent::Eof => break,
        JsonEvent::Error(kind) => panic!("Parser error: {:?}", kind),
        _ => {} // do something useful with the event
    }
}

Parsing a slice of bytes

For convenience, SliceJsonFeeder allows you to feed the parser from a slice of bytes.

use actson::{JsonParser, JsonEvent};
use actson::feeder::SliceJsonFeeder;

let json = r#"{"name": "Elvis"}"#.as_bytes();

let feeder = SliceJsonFeeder::new(json);
let mut parser = JsonParser::new(feeder);
loop {
    match parser.next_event() {
        JsonEvent::FieldName => assert!(matches!(parser.current_str(), Ok("name"))),
        JsonEvent::ValueString => assert!(matches!(parser.current_str(), Ok("Elvis"))),
        JsonEvent::Eof => break,
        JsonEvent::Error(kind) => panic!("Parser error: {:?}", kind),
        _ => {}
    }
}

Parsing into a Serde JSON Value

For testing and compatibility reasons, Actson is able to parse a byte slice into a Serde JSON Value.

Heads up: You need to enable the serde_json feature for this.

use actson::serde_json::from_slice;

let json = r#"{"name": "Elvis"}"#.as_bytes();
let value = from_slice(json).unwrap();

assert!(value.is_object());
assert_eq!(value["name"], "Elvis");

However, if you find yourself doing this, you probably don't need the reactive features of Actson and your data seems to completely fit into memory. In this case, you're most likely better off using Serde JSON directly.

Compliance

We test Actson thoroughly to make sure it is compliant with RFC 8259, can parse valid JSON documents, and rejects invalid ones.

Besides own unit tests, Actson passes the tests from JSON_checker.c and all 283 accept and reject tests from the very comprehensive JSON Parsing Test Suite.

Other languages

Besides this implementation in Rust here, there is a Java implementation.

Acknowledgments

The event-based parser code and the JSON files used for testing are largely based on the file JSON_checker.c and the JSON test suite from JSON.org originally released under this license (basically MIT license).

The directory tests/json_test_suite is a Git submodule pointing to the JSON Parsing Test Suite curated by Nicolas Seriot and released under the MIT license.

License

Actson is released under the MIT license. See the LICENSE file for more information.