rsonpath-lib
– SIMD-powered JSONPath, as a library 🚀
Library for rsonpath
, the JSONPath engine for querying massive streamed datasets.
The main target of this crate is the rsonpath
CLI tool. Note that this API is unstable until we reach
v1.0.0.
Unsafe
The library uses unsafe
for SIMD operations, because it has to, at least until portable-simd
gets stabilized.
Because of this, a compiled library is not portable – if you build on a platform supporting
AVX2 and then use the same compiled code on an ARM platform, it will crash.
We put special care to not use unsafe
code anywhere else – in fact, the crate uses #[forbid(unsafe_code)]
when compiled without the default simd
feature.
Build & test
The dev workflow utilizes just
.
Use the included Justfile
. It will automatically install Rust for you using the rustup
tool if it detects there is no Cargo in your environment.
Architecture diagram
Below is a simplified overview of the module interactions and interfaces, and how data flows from the user's input (query, document) through the pipeline to produce results.
Dependencies
Showing direct dependencies.
rsonpath-lib v0.3.3 (/home/mat/rsonpath/crates/rsonpath-lib)
├── aligners v0.0.10
├── cfg-if v1.0.0
├── log v0.4.17
├── memchr v2.5.0
├── nom v7.1.3
├── replace_with v0.1.7
├── smallvec v1.10.0
├── thiserror v1.0.40
└── vector-map v1.0.1
Justification
aligners
– SIMD operations require correct input data alignment, putting those requirements at type level makes our code more robust.cfg-if
– used to support SIMD and no-SIMD versions.log
– diagnostic logs during compilation and execution.memchr
– rapid, SIMDified substring search for fast-forwarding to labels.nom
– for parser implementation.replace_with
– for safe handling of internal classifier state when switching classifiers.smallvec
– crucial for small-stack performance.thiserror
– idiomaticError
implementations.vector_map
– used in the query compiler for measurably better performance.