jshift 0.7.0

Schema-guided typed JSON on raw bytes: path get/mutate, open views, project — without a Value tree.
Documentation

jshift

jshift is a schema-guided, 100% safe Rust JSON path reader and in-place mutator.

safe Rust zero-copy raw-bytes in-place fast-json

It's built for high-performance; selectively query and modify JSON on raw byte buffers without the full AST allocate / serialize cycle of traditional parsers. In other words:

jshift can be used in place of serde for

🚀 +2x to +20000x speed gains 🚀

on common JSON tasks.

jshift is ideal for optimizing data engineering pipelines, API gateways and request/response transformers, webhook routers, event processors, telemetry, edge filters, and any system that handles JSON.

  • If you only need “read a field from a JSON message” other path engines exist, like gjson.
  • If you need read + rewrite the same buffer under real production constraints (safe Rust, no second serialize pass), jshift is the product.

A simple analogy: most JSON libraries ask you to unpack the whole filing cabinet to change 1 paper document. jshift is built for the scenario where you already know which cabinet drawer and folder you need, and you want to make your find/edit/extract without unpacking and repacking every paper in every drawer.

Simple mental model:

  • Find: Extract the value(s) you're looking for, via a slice of the original bytes
  • Mutate: Edit the same Vec of raw bytes without making a copy or reading into memory
  • Project (as in field projection): Output new smaller JSON document (or NDJSON lines) efficiently
  • TypedDoc (0.5+): hold the buffer, get::<T>(path), stream arrays with each_*, exclusive mutate() — typed fields without serde_json::Value
  • Enums / bridges / flatten (0.7+): external / internal / adjacent / untagged enums; #[json(flatten)]; optional serde / axum
  • API guide: which surface for ingest vs JsonView vs project vs raw land (rustdoc jshift::api_guide)

That’s the whole product: peek, patch same buffer, and/or project thinner JSON, without unpacking and reading the whole filing cabinet.

TypedDoc (typed without Value)

Bring the JsonDoc trait into scope for path get / each_* methods (implemented for TypedDoc, TypedDocRef, SharedDocument, and Vec<u8>).

use jshift::{JsonDoc, TypedDoc};

let mut doc = TypedDoc::from_slice(
    br#"{"status":"ok","items":[{"id":1},{"id":2}],"extra":true}"#,
);

assert_eq!(doc.get::<u64>("items[0].id").unwrap(), 1);
assert_eq!(doc.get_str("status").unwrap(), "ok");

// sparse array field stream — relative path parsed once (beats full-card views)
let ids: Vec<u64> = doc.collect_each_get("items", "id").unwrap();
assert_eq!(ids, vec![1, 2]);

// exclusive mutate: zero-copy borrows cannot coexist (Rust borrow law)
doc.mutate().set("status", "accepted").unwrap();
assert!(doc.contains("extra").unwrap()); // unknowns preserved

0.5–0.7 typed stack: ViewList / ValueList (+ index()), NestedView / MapView / IndexedMapView, CollectPolicy (stream|owned|projected), ObjectBuilder / JsonWriter, to_schema_bytes, batch mutate, validate, typed JSONL, externally tagged enums, serde / axum bridges.

Enums (external / internal / adjacent / untagged)

Byte-level tagging (no Value), aligned with serde’s common representations:

Style Derive Example wire
External (default) #[derive(JsonView)] "Ready" / {"Click":{"x":1}} / {"Id":7}
Internal #[json(tag = "type")] {"type":"Click","x":1}
Adjacent #[json(tag = "t", content = "c")] {"t":"Id","c":7}
Untagged #[json(untagged)] try variants in order
use jshift::JsonView;

#[derive(Debug, PartialEq, JsonView)]
enum Status {
    Ready,
    #[json(rename = "in_progress")]
    Running,
}

#[derive(Debug, PartialEq, JsonView)]
#[json(tag = "type")]
enum InternalEv {
    Ready,
    Click {
        #[json(path = "x")]
        x: u64,
        #[json(path = "y")]
        y: u64,
    },
}

assert_eq!(Status::read_from(br#""in_progress""#).unwrap(), Status::Running);
assert_eq!(
    InternalEv::read_from(br#"{"type":"Click","x":1,"y":2}"#).unwrap(),
    InternalEv::Click { x: 1, y: 2 }
);

Free functions: read_external_*, read_internal, read_adjacent, read_untagged, and matching write_*. See rustdoc jshift::enums and the API guide (when to use what).

Nested views, maps, and flatten (Track B)

Nested JsonView fields (serde-like default)

Serde deserializes nested structs as nested objects by default. jshift mirrors that on bytes: a field whose type is not a built-in scalar / map is a nested [JsonView] at path ("meta": { … }), not flattened.

use jshift::JsonView;

#[derive(JsonView)]
struct Meta {
    #[json(path = "rank")]
    rank: u64,
}

#[derive(JsonView)]
struct Product {
    #[json(path = "id")]
    id: u64,
    #[json(path = "meta")]
    meta: Meta, // wire: {"id":…,"meta":{"rank":…}} — Meta must #[derive(JsonView)]
}
You want… Attribute / type
Nested object (serde default) meta: Meta with Meta: JsonView (auto)
Force nested #[json(nested)]
Spread into parent #[json(flatten)]
Custom scalar newtype (FromJsonSlice) #[json(from_slice)] on the field
Built-in scalar String, u64, bool, … (auto)

Missing JsonView on the nested type is a compile error with a diagnostic note (not a late runtime surprise). Option<Meta>: missing path or JSON null → None.

Open vs closed nested write

API Behavior
Parent write_into / write_view_at Open: child write_into on the nested span — unknowns inside meta stay
write_view_at_closed / to_schema_bytes Closed: replace child with schema-only card — unknowns dropped
// open (default for derive write_into)
write_view_at(&mut buf, &parse_path("meta"), &meta)?;
// closed card
write_view_at_closed(&mut buf, &parse_path("meta"), &meta)?;

Map fields (HashMap / BTreeMap)

use std::collections::HashMap;
use jshift::JsonView;

#[derive(JsonView)]
struct Board {
    #[json(path = "scores")]
    scores: HashMap<String, u64>, // wire: {"scores":{"alice":10,"bob":20}}
}

V must implement FromJsonSlice (numbers, strings, bools, …). Runtime helpers: MapView, read_hashmap_at, write_hashmap_at.

Duplicate-key policy

JSON allows repeated keys; jshift path finds use first match. Explicit APIs never no-op:

use jshift::{
    check_duplicate_keys, validate_unique_keys, validate_unique_keys_deep, DuplicateKeyPolicy,
};

// Security: reject any repeated key
validate_unique_keys(json)?;
validate_unique_keys_deep(json)?; // every nested object

// Resolve: report which value wins (borrowed key spans; no per-key Vec alloc)
let report = check_duplicate_keys(json, DuplicateKeyPolicy::Last)?;
if report.has_duplicates {
    let a = report.get("a").unwrap();
    let _winning = a.value(json); // or &json[a.value_start..a.value_end]
}
// First | Last → ObjectKeyReport<'_>; Error → Err(DuplicateKey) (quote offsets)
// validate_unique_keys / _deep: light unique-only scan (no report)

Byte-level flatten (0.7 / Track B)

Serde-style field spreading on raw buffers — no nested "meta":{...} wrapper, no Value tree:

use jshift::JsonView;

#[derive(JsonView)]
struct Meta {
    #[json(path = "created_at")]
    created_at: String,
    #[json(path = "source")]
    source: String,
}

#[derive(JsonView)]
struct Event {
    #[json(path = "id")]
    id: u64,
    #[json(flatten)]
    meta: Meta, // reads/writes created_at + source at parent root
}
Operation Behavior
Read Child JsonView decodes from the same parent object bytes (open: only named paths)
Write Child write_into open-upserts into parent (unknowns preserved)
Schema emit ObjectBuilder::flatten_object / flatten_view copies on-wire key+value spans (no re-escape)
Project Keep-list = parent paths ∪ child FIELD_PATHS

Also: Option<T> + flatten (None if required child path missing), multiple flattens (later fields win on key collision), free helpers read_flattened / write_flattened.

Flatten benches vs serde(flatten)

Release timings on a typical developer Linux machine; prefer ratios over absolute ns. Schema keys (id, created_at, source) are placed first in the object (realistic “card then payload” layout).

cargo run --release --example flatten_bench --features "derive,serde"
cargo run --release --example flatten_50mb_bench --features "derive,serde"
# 50 MiB fixture: benches/data/flatten_50mb.json (gitignored; auto-generated)

Micro / small-fat objects (flatten_bench)

Workload jshift serde serde/jshift
Decode small Event (id + flatten meta) ~560 ns ~560 ns ~1.0×
Decode on fat (+16 noise + 256 B blob) ~495 ns ~2.8 µs ~5.7×
Decode on fat (+64 noise) ~500 ns ~8.0 µs ~16×
Decode on fat (+256 noise) ~560 ns ~29 µs ~51×
Dual flatten decode (+64 noise) ~890 ns ~7.1 µs ~8×
Open mutate source on fat (+128 noise) ~1.1 µs ~10 µs ~9×
Project thin card vs serde roundtrip ~1.3 µs ~9.0 µs ~6.7×
Encode Event (schema / to_vec) ~530 ns ~140 ns ~0.3× (serde wins)
ObjectBuilder::flatten_object emit ~200 ns ~105 ns ~0.5× (serde wins)

~50 MiB single object (flatten_50mb_bench) — same Event shape; ~50 MiB of noise strings after the schema keys

Workload jshift serde serde/jshift
Decode Event (keys first) ~0.5–1 µs ~32 ms ~30 000–60 000×
In-place write_into (owned buffer, keep noise) ~1 µs ~32 ms ~30 000×
Project thin card vs serde roundtrip ~1.5 µs ~18 ms ~12 000×
Clone body + mutate ~50–70 ms (memcpy) ~24 ms ~0.3× *
Encode card only (not 50 MiB) ~330 ns ~100 ns ~0.3× (serde wins)

*Clone+mutate: jshift keeps the full ~50 MiB (noise preserved). Serde from_slice → change → to_vec rebuilds a ~59 B card and drops all noise — faster wall clock, different product. Pure Vec clone of 50 MiB is ~50 ms on this machine (often the whole “clone + mutate” cost).

Interpretation
You care about… Winner Why
Tiny pure JSON (few KB, no junk) Tie / slight serde Full parse is already cheap; flatten visitors are thin
Fat / multi‑MB docs with unknown fields jshift Path scan stops after named keys; cost stays ~µs while serde tokenizes the whole buffer
Open patch (change source, forward rest) jshift In-place splice; unknowns never materialised
Closed emit of a small card from a Rust value serde Serializer is optimized for tiny graphs; jshift schema emit is competitive but not the win path
“Clone the HTTP body then patch” Depends Memcpy of 50 MiB (~50 ms) dominates jshift; serde can look faster only by throwing away the body on re-serialize

Scaling story: decode cost for jshift stayed ~0.5–1 µs from a few-KB fat object through 50 MiB when schema keys are early. Serde grew from ~µs to ~32 ms. That is the same selective-path law as TypedDoc sparse get — flatten does not break it.

Fairness caveats

  1. Schema-first layout favors early exit; keys-last would force jshift to skip more of the object (still no tree, still no string materialisation of noise).
  2. Serde “mutate” on 50 MiB is not open-document: the re-encoded output is ~59 B, not 50 MiB.
  3. Numbers are one machine / release build; re-run the examples above on your hardware.

Ecosystem bridges (0.7)

jshift = { version = "0.7", features = ["derive", "serde", "axum"] }
Feature API Role
serde AsJshiftView<T> (&RawValue, zero-copy from_str/from_slice), AsJshiftViewOwned (Box for readers), from_value / to_value Gradual migration; sparse path decode without a Value domain type
axum JshiftJson<T>, JshiftDoc Body → JsonView / TypedDoc without serde_json::Value; open/closed response

Serde bridge guide (rustdoc): jshift::serde_bridge_guide (feature serde) — why the bridge exists, best practices, benches vs native serde, and side-by-side code.

Multi-file JSON / JSONL (paginated APIs, shards, chunks)

Stream records across many buffers without merging into one Value or one mega-string:

Workload API
Many JSONL shards for_each_jsonl_multi / for_each_jsonl_view_multi
Chunked download (split mid-line) JsonlDecoder
Paginated {"items":[…]} pages for_each_page_items
Root array per page for_each_page_array
One object per page/file for_each_page_doc
use jshift::{for_each_page_items, JsonView};

#[derive(JsonView)]
struct Row {
    #[json(path = "id")]
    id: u64,
}

// page1.json, page2.json, …
for_each_page_items::<Row, _, _, _>([p1, p2], "items", |pos, row| {
    // pos.source_index = page, pos.record_index = item within page
    Ok(())
})?;

Bench vs serde (multi_json_bench)

cargo run --release --example multi_json_bench --features derive
Workload jshift serde serde/jshift
Pages sparse each_get id (10k rows, ~3 MiB, 256 B blob) ~8.5 ms ~7.3 ms ~0.9× (serde wins)
Multi JSONL sparse id (8k lines, ~2.4 MiB) ~2.9 ms ~6.8 ms ~2.4×
Chunked JSONL 4 KiB feeds (3k lines) ~1.1 ms ~2.3 ms ~2.0×

Interpretation: On moderate paginated pages, serde’s full-page Deserialize is highly optimized and can edge out per-row path views when every row is visited. Sharded / chunked JSONL favors jshift (sparse path decode per line, no whole-page struct materialization). Product wins beyond raw ns: no buffer merge, StreamPos provenance, open docs, and JsonlDecoder for incremental feeds.

Live TeeFury catalog ingest (teefury_ingest_bench)

Real Shopify products.json — GET teefury.com/products.json?page=1 … page 10, project listing cards (id / title / handle / vendor / product_type) and drop body_html + all variants / images / options.

# HTTP fetch pages 1–10 → benches/data/ (gitignored), then CPU bench:
cargo run --release --example teefury_ingest_bench --features derive
# Offline (fixtures already present):
TEEFURY_OFFLINE=1 cargo run --release --example teefury_ingest_bench --features derive
Workload (300 products, ~3.6 MiB / 10 pages) jshift serde serde/jshift
ProductCard JsonView vs serde sparse card ~8.8 ms ~4.1 ms ~0.5× (serde wins typed card)
ProductCard vs serde full product (variants/images as Value) ~8.8 ms ~29 ms ~3.4×
ProductCard vs serde Value + field index ~8.8 ms ~29 ms ~3.3×
ProjectPlan emit card arrays vs serde full tree ~4.9 ms ~29 ms ~6×
JMESPath project cards (+fold) vs serde map→JSON ~4.7 ms ~4.3 ms ~0.9× (wash)

Payload: card JSON / NDJSON ≈ 1.2% of the raw catalog pages (~43 KiB from ~3.6 MiB).

How to read this: If the pipeline already deserializes the whole Shopify product (or a Value tree) “just in case,” switching to a sparse jshift card or ProjectPlan is a clear win on this corpus. If you already have a minimal serde struct with only card fields, serde’s single-pass Deserialize is hard to beat for pure decode — jshift still wins on no DOM, multi-page streaming without merge, open docs, and byte-level project emit. Numbers are one Linux release build; re-run the example on your machine.

When does ingestion want edge decode?

Edge decode = extract typed fields (and/or project) before/while landing—not “only dump bytes.”

Prefer edge decode when… Prefer raw / delayed decode when…
Hot path needs fields (filter, route, trade) before land finishes Archive / compliance “exact wire bytes” is the only goal
Bronze/stage is schema-on-write (Parquet/Arrow rows) and you accept that tax at the edge Bronze is true raw JSON and silver has spare CPU
You must drop most of the payload (PII, fat variants) for storage $ Schema is unstable; you reprocess raw often
Dedupe / tenant routing needs id / account at the collector Feed is already binary / Arrow / Flight

jshift = nested JSON still on the wire: sparse paths + raw element spans for a raw_json column without a Value tree. Arrow/Parquet = columnar stage after you have rows.

Bronze land: true raw vs Arrow IPC (bronze_land_bench)

cargo run --release --example bronze_land_bench --features derive

Disk+fsync is usually I/O bound (raw ≈ Arrow payload). Product-row IPC always includes a raw_json column in the corrected paths. Prefer each_with + find_value over JsonView for ingest (see note below).

JSON → Parquet stage tables S/M/L (json_parquet_stage_bench)

Throughput architecture probe: synthetic Shopify-shaped pages (fat body_html / variants), always stage schema
page, id, title, handle, vendor, product_type, raw_json (full product JSON).

cargo run --release --example json_parquet_stage_bench --features derive
Path (memory → Snappy Parquet) small ~14 KiB medium ~367 KiB large ~6.5 MiB Notes
jshift span+find (5 fields+raw) ~80 MiB/s ~220 MiB/s ~260 MiB/s preferred jshift; 5× object scans
jshift early id+title+raw faster ~+5% ~+13% stops before fat body
jshift raw_json spans only fastest jshift ~+30% ~+18% row-per-product JSON, no field decode
serde RawValue + re-parse card ~+10% ~+35% ~+30% two passes; tight tokenizer
serde sparse + FAKE raw_json fast ~3× ~8× not full fidelity (drops variants)
serde Value DOM slower slower ~parity tree tax

Why serde often beats jshift on “5 listing fields + raw”:
Serde’s single-pass Deserialize is extremely optimized. jshift’s public find_value restarts the object scan per key (no multi-key single-pass API yet). That is an honest limit—not “jshift is bad at JSON.” jshift still wins the product axes: open docs, raw spans without DOM, early-exit sparse reads, project-without-tree, in-place mutate. JsonView is the ergonomic API; each_with + static PathSegments is the ingest API.

Architecture takeaway for max throughput to Parquet stage:

  1. Decide if stage needs true raw_json (full product) or projected columns only.
  2. If true raw + few columns: jshift span walk is competitive and DOM-free; serde RawValue is often slightly faster today.
  3. If you drop fidelity (FAKE raw), anything “sparse-only” looks faster—don’t confuse that with bronze quality.
  4. True raw .json land remains cheapest when silver does the Parquet convert later.
// Axum handler (feature = "axum")
// Return the extractor so retained body is open-written (unknowns kept).
async fn create(mut req: JshiftJson<CreateUser>) -> JshiftJson<CreateUser> {
    req.value.email = req.value.email.to_lowercase();
    req
}
// Closed card (no body): JshiftJson::closed(CreateUser { ... })

Example: cargo run --example axum_jshift --features "axum,derive".

v0.7 benches (enums + bridges)

cargo run --release --example v07_ecosystem_bench --features "derive,serde"

Release timings on a typical developer machine (Linux); prefer ratios over absolute ns.

Workload jshift serde serde/jshift
Unit enum decode ("in_progress") ~36 ns ~38 ns ~1.1×
Struct-variant decode (Click + noise) ~194 ns ~131 ns ~0.7× (serde wins tiny)
Newtype enum decode {"Id":42} ~52 ns ~42 ns ~0.8×
Mixed unit|object dispatch (pair) ~71 ns ~79 ns ~1.1×
Sparse card on fat object (id+title) ~141 ns ~378 ns ~2.7×
…vs full fat struct deserialize ~141 ns ~741 ns ~5.3×
…vs Value + index ~141 ns ~1.5 µs ~10×
AsJshiftView sparse via from_str ~448 ns ~378 ns ~0.8× (bridge tax)
Schema emit 2-field card ~85 ns ~76 ns ~0.9×
Unit / newtype encode ~47–54 ns ~24–31 ns ~0.5–0.6× (serde wins)
Flatten decode fat (+256 noise) ~560 ns ~29 µs ~51× — see flatten table

How to read this

  • Tiny external enums are a wash (or slight serde win) — both paths are already thin.
  • Sparse views on fat JSON remain jshift’s win (same story as 0.5–0.6).
  • AsJshiftView pays a RawValue boundary; use native JsonView::read_from or JshiftJson when you control the stack.
  • Axum JshiftJson is body buffer + read_from (same cost class as sparse decode once bytes are in RAM) — not micro-benched as HTTP.
  • Flatten details: cargo run --release --example flatten_bench --features "derive,serde".

Typed bytes epic (v0.5–v0.6)

Goal: typed JSON on raw buffers so most production work never needs serde_json::Value — sparse path decode, open in-place mutate, streamable collections, schema emit, and honest “when serde still wins” boundaries.

Repro:

cargo run --release --example typed_doc_bench      # v0.5 spine
cargo run --release --example v06_collections_bench # v0.6 collections

Numbers below are release timings on a typical developer machine (Linux); ratios will move with CPU and payload shape. Prefer ratios over absolute ns.

v0.5 — TypedDoc spine vs serde_json

Workload jshift serde serde/jshift
Small sparse get id (~100 B) ~100–170 ns ~1.3–2.6 µs ~13–15×
Med sparse products[0].id (~111 KB) ~150–270 ns ~2.4–4.7 ms ~15 000–17 000×
Large sparse first product (~2 MB) ~160–280 ns ~39–72 ms ~240 000–260 000×
Large key-last status (after bulk) ~4–7 ms ~70–92 ms ~13–18×
Stream each_get sum ids (2k) ~370–430 µs ~1.3 ms ~3–4×
Stream each_get sum ids (50k) ~10–12 ms ~70–75 ms ~6–7×
Sparse id stream (500 cards) ~87–100 µs ~97–130 µs ~1.0–1.3×
Full Card each_view (500) ~180–190 µs ~100–130 µs ~0.5–0.7× (serde wins)
Open mutate status (med ~111 KB) ~8 µs ~1.5–1.6 ms ~180–200×
Open mutate status (large ~2 MB) ~280 µs ~46 ms ~160×
ViewList::index mid get (5k, after index) ~80 ns ~760 µs full parse ~9 000×
64× indexed multi-get (5k) ~5 µs ~760 µs ~150×
JsonWriter build (2k ids + nest) ~26 µs ~35 µs ~1.3×
RawJson pocket products[0]→id ~260 ns ~1.3 ms ~5 000×
object_entries sum 200 keys ~10 µs ~42 µs ~4×
get_nullable (tiny doc) ~parity ~parity ~1×

v0.6 — Nested / Map / collect policies vs serde_json

Payloads used in v06_collections_bench: ~450 KB catalog (5k products), ~31 KB score map (2k string keys).

Workload jshift serde serde/jshift
NestedView / path products[0].meta.rank ~300–370 ns ~6.4 ms ~17 000–21 000×
MapView get mid-key (2k keys) ~23–26 µs ~575 µs ~22–25×
MapView each sum 2k keys ~160–240 µs ~560 µs ~2–4×
Map collect_owned vs HashMap deserialize ~230–250 µs ~340 µs ~1.4–1.5×
each_field / collect_field ids (5k) ~0.82–1.0 ms ~1.2 ms ~1.2–1.4×
collect_owned IdCard (5k) ~1.1 ms ~1.2 ms ~1.1×
collect_projected thin cards (5k) ~3.1 ms ~1.2 ms ~0.4× (serde wins full typed)
Nested meta.rank sum — manual nest ~2.2 ms ~1.3 ms ~0.6×
Nested meta.rank sum — sum_nested_field_u64 ~1.5–1.7 ms ~1.3 ms ~0.8–0.9×
64× map get linear ~1.5–1.6 ms ~335–350 µs* ~0.2×
64× map get after IndexedMapView ~2.6–3.4 µs ~335–350 µs* ~100–130×
Flat to_schema_bytes (2 fields) ~97–104 ns ~65–70 ns ~0.6–0.7× (near parity)
64× ViewList indexed random get ~5–6 µs ~1.2 ms ~210–250×
RFC 7396 merge_patch (tiny) ~1.5 µs ~1.1 µs† ~0.8×
check_document depth+size (~450 KB) ~690 µs ~7 ms full parse ~10×
project_view_each IdCard sum (5k) ~1.8 ms ~1.2 ms ~0.7×
JsonWriter::pretty tiny object ~115 ns ~176 ns ~1.5×

*serde multi-get includes full HashMap deserialize each iteration (fair “cold” cost). After you already hold a map, pure HashMap get is faster — use IndexedMapView when you keep the buffer and probe often.

†serde merge_patch column is a hand-rolled Value merge (no std RFC 7396 API).

How to read the epic tables

Prefer jshift when… Prefer serde when…
Sparse path / first-card / key-first on large buffers You need every field of every element
Open mutate / batch patch without rebuild Tiny closed structs, pure encode firehose
Stream one field (each_field / each_get) Full Vec<Struct> is the product
Multi-get after index() / IndexedMapView You already built a DOM / HashMap for other reasons
Safe Rust + same buffer Multi-format (bincode, etc.)

Benchmarks on JSON tasks (path engine matrix)

The bench is designed to provide a fair comparison between jshift, serde, gjson, and sonic-rs.

JSON files used on each bench task:

Scale Size
Small ~500 KiB
Medium ~10 MiB
Large (find/mutate) ~50 MiB
Large (catalog) ~338 MiB, ~25k products from a Shopify public API

Benchmark Task 1: Key-first find

Hot field is the first key; a bulk array trails behind.

  • APIs

    • jshift: find_value(json, &parse_path("target"))
    • serde: serde_json::from_slice → v["target"]
    • gjson: gjson::get(s, "target")
    • sonic-rs: sonic_rs::get(json, &pointer!["target"])
  • Why jshift is fast: stops at the first key; never materializes the trailing bulk. Serde always builds a full tree of the whole buffer.

  • Ratios (medium): vs serde ~3 000 000×; vs gjson ~1.5×; vs sonic ~2.1×

  • Timings

    Engine ~500 KiB ~10 MiB Large (~50 MiB)
    jshift ~80 ns ~80 ns ~80 ns
    serde ~12 ms ~253 ms ~856 ms
    gjson ~140 ns ~120 ns ~140 ns
    sonic ~190 ns ~171 ns ~241 ns
  • When to use: default API / gateway shape — “Is status ok?” near the root. Common production case; not the adversarial key-last row.

Code Example of Key-first find

use jshift::{find_value, parse_path};

let json = br#"{"status":"ok","items":[1,2,3,4,5]}"#;

// jshift makes `v` a zero-copy slice into `json`:
let v = find_value(json, &parse_path("status")).unwrap();

assert_eq!(v, br#""ok""#);

The jshift concept: stop at status; never build a tree of items (ie what serde does).


Benchmark Task 2: Key-last find

Hot field sits after a giant array (adversarial skip).

  • APIs

    • jshift: find_value(…, "target") after bulk data
    • serde: full from_slice + index
    • gjson: gjson::get(s, "target")
    • sonic-rs: pointer get "target"
  • Why: everyone must skip the bulk. gjson’s unsafe-hot skip can win pure find; jshift stays safe and multi-× vs full parse.

  • Ratios (medium): vs serde ~10×; vs gjson ~0.38× (gjson faster); vs sonic ~2.4×

  • Timings

    Engine ~500 KiB ~10 MiB Large (~50 MiB)
    jshift ~655 µs ~13.5 ms ~66 ms
    serde ~6.3 ms ~140 ms ~717 ms
    gjson ~259 µs ~5.2 ms ~26 ms
    sonic ~1.6 ms ~32 ms ~152 ms
  • When to use: the key you're looking up is structured to come after other large fields, especially when you still need safe mutate / project on the same buffer (jshift's forte), not only a get (gjson's forte).

Code Example of Key-last find

use jshift::{find_value, parse_path};

let json = br#"{"items":[0,1,2,3,4,5,6,7,8,9],"target":123456}"#;

let v = find_value(json, &parse_path("target")).unwrap();
assert_eq!(v, b"123456");

The jshift concept: this is still a path scan, ie skip the array as one value, then read target. Cost is higher when the bulk is huge. This is one of the few scenario's in which gjson is more performant, using unsafe rust to achieve the speed, but jshift is still extremely performant.


Benchmark Task 3: 🏆 In-place mutate 🏆

Same-length overwrite of one field.

This is the mutator feature that jshift provides, which no other rust crate offers.

  • APIs

    • jshift: mutate_value(&mut buf, &path, b"654321")
    • serde: from_slice → set → to_vec (new document)
    • gjson: — no in-place mutate
    • sonic-rs: — no jshift-style splice mutate
  • Why: jshift splices bytes in the same Vec<u8>. Serde rebuilds a full document. Peers are not mutators.

  • Ratios (medium): vs serde ~9.3×; gjson/sonic N/A

  • Timings

    Engine ~500 KiB ~10 MiB Large (~50 MiB)
    jshift ~1.3 ms ~28 ms ~139 ms
    serde ~7.4 ms ~257 ms ~1.0 s
    gjson — — —
    sonic — — —
  • When to use: gateways, JSONL cleaners, feature flags — change status, keep shipping the rest of the bytes. Prefer mutate_value_checked for untrusted payloads.

Code Example of In-place Mutate

use jshift::{find_value, mutate_value, parse_path};

let mut json = br#"{"status":"new","id":7}"#.to_vec();

mutate_value(&mut json, &parse_path("status"), br#""accepted""#).unwrap();

assert_eq!(&json[..], br#"{"status":"accepted","id":7}"#);
// output document (same Vec): {"status":"accepted","id":7}

assert_eq!(find_value(&json, &parse_path("id")).unwrap(), b"7");

The jshift concept: splice bytes in place. (Same-length edits are the cheapest case; longer/shorter values still work via tail rotate.)


Benchmark Task 4: Sparse find first array element

products[0].title without walking the catalog.

  • APIs

    • jshift: find_value(…, "products[0].title")
    • serde: parse → v["products"][0]["title"]
    • gjson: products.0.title
    • sonic-rs: pointer!["products", 0, "title"]
  • Why: early exit into the first element; no catalog DOM. Path engines stay ns–µs; full parse pays for every product.

  • Ratios (medium): vs serde ~1 300 000×; vs gjson ~2.5×; vs sonic ~1.9×

  • Timings

    Engine ~500 KiB ~10 MiB Large (338 MiB catalog)
    jshift ~120 ns ~120 ns ~150 ns
    serde ~6.3 ms ~153 ms ~3.4 s
    gjson ~301 ns ~301 ns ~3.8 µs
    sonic ~231 ns ~231 ns ~270 ns
  • When to use: sample one record from a huge export, health checks, “show me the first listing.” On huge real dumps serde is multi-second; jshift stays in nanoseconds.

Code Example of Sparse find first array element

use jshift::{find_value, parse_path};

let json = br#"{
  "products": [
    {"id":1,"title":"Hat","noise":true},
    {"id":2,"title":"Mug","noise":false}
  ]
}"#;

// find products[0].title without caring about the rest of the catalog.
let v = find_value(json, &parse_path("products[0].title")).unwrap();
assert_eq!(v, br#""Hat""#);

The jshift concept: walk only to the first product’s title; leave later products unread.


Benchmark Task 5: Sparse find mid element (linear)

products[mid].title without an index (sibling skip).

  • APIs

    • jshift: find_value(…, "products[N].title") scan
    • serde: full parse + index
    • gjson: products.N.title
    • sonic-rs: parse or pointer walk
  • Why: linear sibling skip is O(N). Correct for one-shot streaming; painful for random access. gjson is a strong pure scanner.

  • Ratios (medium): vs serde ~26×; vs gjson ~0.59× (gjson faster); vs sonic ~3.5×

  • Timings

    Engine ~500 KiB ~10 MiB Large (338 MiB)
    jshift ~292 µs ~6.0 ms ~178 ms
    serde ~7.0 ms ~154 ms ~3.4 s
    gjson ~173 µs ~3.6 ms ~109 ms
    sonic ~986 µs ~21 ms ~469 ms
  • When to use: rare one-off mid hits only. If you query mid/last often, pay the upfront cost for the indexed entry which provides blazing fast speeds after the initial index creation (examples provided lower in the doc).

Code Example of Sparse find mid element (linear)

use jshift::{find_value, parse_path};

let json = br#"{
  "products": [
    {"id":0,"title":"A"},
    {"id":1,"title":"B"},
    {"id":2,"title":"C"}
  ]
}"#;

let v = find_value(json, &parse_path("products[2].title")).unwrap();
assert_eq!(v, br#""C""#);

The jshift concept: fine for a one-off, however each mid/last hit re-skips earlier siblings, therefore use the index approach instead of this approach if you'll make several calls to find a mid-placed element.


Benchmark Task 6: Sparse find mid element (🏆 indexed 🏆)

Opt-in array side-table (IndexedDocument).

  • APIs

    • jshift: doc.index_array_str("products")?; doc.find(&parse_path("products[N].title"))?
    • serde / gjson / sonic: no jshift-style side-table — still full parse or full scan each time
  • Why: side-table bookmarks every element start (table of contents). Lookup is O(1) jump + tiny local scan. Index build is opt-in (~0.7 to 0.9 s once on 338 MiB).

  • Ratios (medium): vs serde ~1 500 000×; vs gjson ~36 000×; vs sonic ~210 000× (peers re-scan/parse)

  • Timings (peers = same as linear row above)

    Engine ~500 KiB ~10 MiB Large (338 MiB)
    jshift indexed ~100 ns ~100 ns ~110 ns
    serde ~7.0 ms ~154 ms ~3.4 s
    gjson ~173 µs ~3.6 ms ~109 ms
    sonic ~986 µs ~21 ms ~469 ms
  • When to use: random access / multi-query on one snapshot. Build once (IndexedDocument / index_for_plan), then many finds/projects. Indexes go stale after in-place mutate.

Code Example of Sparse find mid element (indexed)

use jshift::{IndexedDocument, parse_path};

let json = br#"{
  "products": [
    {"id":0,"title":"A"},
    {"id":1,"title":"B"},
    {"id":2,"title":"C"}
  ]
}"#;

let mut doc = IndexedDocument::empty(json);
doc.index_array_str("products").unwrap(); // pay once

let v = doc.find(&parse_path("products[2].title")).unwrap();
assert_eq!(v, br#""C""#);

The jshift concept: index = “table of contents” for array starts; then mid/last is a jump, not a marathon to skip each sibling.


Benchmark Task 7: Sparse project first card

Keep-list / schema card for products[0] (id / title / handle).

  • APIs

    • jshift: project(json, &ProjectPlan::from_paths(&["products[0].id", "products[0].title", "products[0].handle"])?)?
    • serde: parse → build map → to_vec
    • gjson: 3× get + string rebuild
    • sonic-rs: 3× pointer + string rebuild
  • Why: jshift emits a schema-shaped mini document without parsing the catalog (P0 open-ended descent). sonic/gjson win “hand me three fragments”; jshift wins “hand me JSON I can forward.”

  • Ratios (medium): vs serde ~140 000×; vs gjson ~0.85× (gjson slightly faster on synthetic medium); vs sonic ~0.59× (sonic faster on pure fragments)

  • Timings

    Engine ~500 KiB ~10 MiB Large (338 MiB)
    jshift ~1.1 µs ~1.1 µs ~7.7 µs
    serde ~6.4 ms ~156 ms ~3.2 s
    gjson ~962 ns ~962 ns ~11.7 µs
    sonic ~691 ns ~672 ns ~991 ns
  • When to use: admin sample cards, support tools. Prefer jshift when output must be valid nested JSON / keep-list schema; prefer sonic when you only need raw field strings.

Code Example of Sparse project first card

use jshift::{project, ProjectPlan};

let json = br#"{
  "products": [
    {"id":1,"title":"Hat","handle":"hat","blob":[1,2,3]},
    {"id":2,"title":"Mug","handle":"mug","blob":[]}
  ]
}"#;

let plan = ProjectPlan::from_paths(&[
    "products[0].id",
    "products[0].title",
    "products[0].handle",
]).unwrap();

let out = project(json, &plan).unwrap();
assert_eq!(
    out,
    br#"{"products":{"id":1,"title":"Hat","handle":"hat"}}"#
);
// output:
// {"products":{"id":1,"title":"Hat","handle":"hat"}}

The jshift concept: schema-shaped card you can forward; blob never enters the output. (JMES products[0].{id: id, title: title, handle: handle} yields a flat object instead of nesting under products—different shape, same “first card” job.)


Benchmark Task 8: Full-catalog thin cards

When you want a thinned full-catalog of any array of objects, for example removing all but 2 keys from the full catalog → {id, title}.

  • APIs

    • jshift: index_for_plan + project_indexed / project_object_fields("products", &["id","title"])
    • serde: parse all → map each → to_vec
    • gjson: each product + rebuild array string
    • sonic-rs: Value map + serialize
  • Why: one-pass multi-select + streaming list emit + array side-table. Avoids a full DOM of every product. Medium synthetic is dense/small (scanner-friendly); large real catalog is where jshift’s stack pulls ahead clearly.

  • Ratios: medium synthetic vs serde ~6.3×, vs gjson ~0.73×, vs sonic ~3.0×; 338 MiB real vs serde ~63×, vs gjson ~8.8×, vs sonic ~10×

  • Timings

    Engine ~500 KiB ~10 MiB Large (338 MiB)
    jshift ~1.6 ms ~32 ms ~50 ms
    serde ~8.3 ms ~203 ms ~3.2 s
    gjson ~1.2 ms ~24 ms ~444 ms
    sonic ~2.9 ms ~95 ms ~503 ms
  • When to use: domain-agnostic thin cards (any array path). For NDJSON / one-card-at-a-time use project_jsonl_write / project_each. Use project_indexed_auto when plans may be CPU-heavy.

Code Example of Full-catalog thin cards

The standard example:

use jshift::project_object_fields;

let json = br#"{
  "products": [
    {"id":1,"title":"Hat","price":9.99,"images":[1,2]},
    {"id":2,"title":"Mug","price":5.00,"images":[]}
  ]
}"#;

let out = project_object_fields(json, "products", &["id", "title"]).unwrap();
assert_eq!(
    out,
    br#"[{"id":1,"title":"Hat"},{"id":2,"title":"Mug"}]"#
);
// output:
// [{"id":1,"title":"Hat"},{"id":2,"title":"Mug"}]

JMES example with optional index:

use jshift::{project_indexed, IndexedDocument, ProjectPlan};

let plan = ProjectPlan::from_jmespath("products[*].{id: id, title: title}").unwrap();
let mut doc = IndexedDocument::empty(json);
doc.index_for_plan(&plan).unwrap();

let out = project_indexed(&doc, &plan).unwrap();
// same: [{"id":1,"title":"Hat"},{"id":2,"title":"Mug"}]

The jshift concept: one small card per element; fat fields stay in the source and never get a DOM.


A few more shapes

JSONL lines

use jshift::{find_value, json_lines, parse_path, project_paths};

let buf = br#"{"topic":"rust neural networks","messages":[{"role":"user","content":"…huge…"}],"record_id":"abc"}
"#;

for line in json_lines(buf) {
    let topic = find_value(line, &parse_path("topic")).unwrap();
    // topic == "rust neural networks" (with quotes in the slice)

    let all_topic_records = project_paths(line, &["topic", "record_id"]).unwrap();
    // all_topic_records: {"topic":"rust neural networks","record_id":"abc"}
}

The jshift concept: You've got gigabytes, maybe terabytes, of JSONL training data for LoRA tuning your favorite open-source LLM. You don’t want to fully deserialize every record just to keep a couple of fields, filter by topic, or rewrite a system prompt. jshift lets you do path-based reads and mutations directly on the raw bytes at blazing speeds and with minimal allocation, so your data prep finishes faster and your time goes toward actual training instead of waiting on the data pipeline.

NDJSON stream of cards (without a giant [...] in memory)

use jshift::{project_object_fields_jsonl_write, ProjectPlan};
// or project_jsonl_write with a list-projection plan

let json = br#"{
  "messages": [
    {"id":1,"title":"How to debug borrow checker","difficulty":9,"related_topics":["rust","rust lifetimes",...], ...},
    {"id":2,"title":"Best crates for JSON","difficulty":3,"related_topics":["jshift","serde",...], ...}
  ]
}"#;

let mut out = Vec::new();
project_object_fields_jsonl_write(json, "messages", &["id", "title"], &mut out).unwrap();
// out as text:
// [{"id":1,"title":"How to debug borrow checker"}, {"id":2,"title":"Best crates for JSON"}, ...]

The jshift concept: Efficiently output a new smaller JSON document or NDJSON lines.


Key Features

  • Path-selective scans: Walk only the path you need on raw &[u8] / Vec<u8>.
  • In-place mutations: Upsize and downsize with safe slice rotations (compiles to memmove-class moves).
  • 100% Safe Rust: #![forbid(unsafe_code)], no get_unchecked in the hot path.
  • Zero-copy reads: find_value returns a subslice of the input buffer.
  • JsonView trait: one protocol surface for “this Rust type is a projection of JSON bytes” (read_from / read_from_indexed / write_into), generic pipelines without ad-hoc methods.
  • Schema derive: #[derive(JsonView)] or JsonMutatorSchema, typed readers/mutators, FIELD_PATHS, schema-guided INDEXED_ARRAY_PATHS / prepare().
  • Open projections: fields you don’t name are unread and byte-preserved on write (API evolution as a feature).
  • Shared documents: SharedDocument (Arc<[u8]>) for cheap clone + many concurrent readers.
  • JSONL helpers: json_lines / read_jsonl -- index per line, not one giant merge; array→NDJSON cards via project_jsonl_write / project_object_fields_jsonl_write.
  • Field projection: project / project_paths / project_jmespath / project_write / project_indexed / ProjectPlan (keep-list + byte-oriented JMESPath subset on raw spans, not a DOM port of jmespath.rs; Compact / PreserveSource / Pretty).
  • Streaming cards: project_each / project_object_fields_each — one callback per list element (no giant output array); peak RAM ≈ one card.
  • Parallel auto-pick: plan_prefers_parallel + project_indexed_auto / project_parallel_auto (Rayon only when the plan is likely CPU-bound; thin cards stay sequential).
  • Projection estimates: estimate_projected_len / projected_len (planning vs exact).
  • Transforms: Transform / TransformPipeline (KeepPaths, Jmes, Rename, Drop, Inject, Style).
  • Derive JMES: #[json(jmes = "...")] + FIELD_JMES multi-select project plan.
  • Object & array CRUD: Update, upsert, delete keys; append, index, delete elements; nested upsert_at_path.
  • Correct string encoding: ToJsonBytes and key upserts escape ", \, and control characters.
  • Owned + pointer paths: Path, try_parse_path, JSON Pointer (Path::from_json_pointer).
  • Option / null: first-class for training JSONL and partial records.
  • Structural indexes (opt-in): [IndexedDocument] side-tables so mid/last products[i].field jumps instead of scanning every sibling.

In other words, the feature set is three jobs that share one engine:

Job Everyday phrase Typical API
Peek “What’s at this path?” find_value, JsonView::read_from
Patch “Change this field, keep the rest of the bytes.” mutate_value, derive mutator
Projection “Emit a smaller JSON with only the fields I need.” project, project_object_fields, JMES

jshift is a non-validating path engine. It assumes mostly well-formed JSON along the path you traverse. Callers must supply complete JSON value bytes for raw mutations (or use ToJsonBytes / mutate_value_checked).


Why jshift exists

Typical stack today:

bytes → serde_json::from_slice → Value tree → change a field → to_vec → bytes

That is correct and ergonomic. It is also expensive when:

  • documents are large (MBs of payload, one hot field),
  • you process millions of JSONL lines,
  • you run many concurrent workers on the same shape of traffic,
  • you only care about one path (or a small schema of paths).

In other words: the cost is not “JSON is text.” The cost is materializing a second representation of everything so you can touch a tiny slice of it. That is the right default for application models; it is the wrong default for gateways, cleaners, and data pipeline projections.

jshift flips the model:

bytes → path scan to byte offsets → splice / shift bytes in place → same Vec

No tree. No second full document serialize. Reads return zero-copy slices into the original buffer.

Analogy: think of a paper book versus photocopying the entire library to highlight one sentence. Serde often photocopies (parse → tree → re-print). jshift walks the aisle, opens the right volume, and edits the margin... still carefully, still with bounds and Results, but without reprinting the shelves.

Real-world example: API JSON ingestion (what IT teams do today)

A common pattern in platform / integration teams looks like this:

  1. Pull a large JSON payload from a partner or SaaS API (catalog, events, orders, transaction dumps... often multi‑MB or hundreds of MB).
  2. Ingest into an internal service: validate a couple of fields, stamp metadata (ingested_at, source, status), maybe drop or rewrite a status flag, then forward the body to a queue, object store, or downstream microservice.
  3. Today’s default stack is almost always: HTTP body → serde_json::from_slice (or equivalent) → walk a full in-memory tree → change one or two fields → to_vec / re-serialize → publish. That is simple to write and easy to reason about but on a 300 MB json catalog you still allocate and walk the entire tree, then allocate and write another ~300 MB of output, even if you only needed products[0].title and a top-level status. Under bursty multi-worker ingestion, that becomes CPU, memory, and GC (or allocator) pressure, not “JSON is slow because HTTP is slow.”
  4. With jshift, the same job is: keep the body as Vec<u8> → path-scan only the fields you need → splice stamps or flags in place with safe byte rotations → hand the same buffer downstream. Peers that only need a header field never pay for the giant products array. Teams that must stay on safe Rust (no unsafe hot loops) get selective R/W without becoming a second full parser.

When this is impactful: multi-worker ingestion of multi‑MB bodies, JSONL cleaners at millions of lines/day, feature-flag or status rewrites in a gateway, bronze-to-silver “keep ten fields, drop the rest.” When it is not: a request handler that already deserializes into a rich domain struct and never ships the raw bytes again --> this ought to stay on serde.


When to use jshift vs serde_json

You should use… When…
jshift You touch few fields on large or high-volume JSON; you want in-place updates; you control or trust path shape; latency / throughput matter.
serde_json You need a full typed model, validation of the whole document, arbitrary transforms, or you already pay for a complete parse.
Both Parse selectively with jshift for hot paths; use serde when a request actually needs the full document.

Rule of thumb

  • “Filter, add tags, or rewrite status on every JSONL line” → jshift.
  • “Deserialize into struct Request { … } with dozens of fields and nested untagged enums” → serde (or jshift + hand paths until more enum encodings land).
  • “Gateway: inspect headers.x-request-id, maybe set status, forward body” → jshift.
  • “Partial view struct (ListingCard { id, title }) over a fat catalog object” → JsonView.
  • “Axum handler that only needs a few fields of the body” → JshiftJson<T: JsonView> (feature axum).
  • “Must plug into a serde-only API for a while” → AsJshiftView<T> (feature serde).

In other words: choose by how much of the document becomes your domain model. If the answer is “almost all of it, with types and invariants,” serde wins. If the answer is “a few paths, then forward or slim the bytes,” jshift wins. Many production systems can benefit from both: jshift on the wire edge, serde inside the application core.


How it works under the hood (byte shifts, not magic)

You do not need a systems-programming background to use jshift, but understanding the mechanism explains the speed — and when not to expect miracles.

1. Find = locate byte offsets, not build a tree

find_value(json, path) walks the raw bytes along a path like user.score or tags[0]:

  1. Skip whitespace and nested structures you do not need (including large arrays/objects on the way).
  2. Match object keys (as on-wire bytes between quotes) or array indexes.
  3. Return (start, end) into the original buffer or just the slice json[start..end].

So for:

{"data":[ … megabytes … ],"target":123456}

a path "target" means: skip the entire data array as one bulk value, then read the number.
No heap tree of every object in data.

In other words: skipping is not “ignoring data forever”; it is refusing to build furniture for rooms you will never enter. The bytes stay in the buffer; they just never become Value::Object nodes.

2. Mutate = splice into the same Vec<u8>

When you replace a value:

Case What jshift does
Same length (e.g. 123456 → 999999) Overwrite bytes in place.
Longer Vec::resize, then tail.rotate_right(delta) to open a gap, write the new value.
Shorter tail.rotate_left(delta) to close the gap, then truncate.

rotate_left / rotate_right on a slice are safe APIs. LLVM typically lowers them to the same class of bulk memory moves as memmove. You get high performance without writing unsafe pointer arithmetic.

Deletes work the same way: compute a span (including commas), expand over adjacent whitespace for tidy output (“pretty delete”), then shift the tail left.

In other words: the “hard” part of editing a JSON string in place is not finding the field — it is keeping commas and braces honest when the replacement changes length. jshift’s job is that bookkeeping, with the tail of the document treated like a sliding window of bytes.

3. Why that is faster than serde for selective work

serde’s mutation happy path is:

  1. Parse every token into a Value (or a typed struct).
  2. Mutate the tree.
  3. Walk the tree and write a new document.

jshift’s mutation happy path is:

  1. Scan until the path of interest.
  2. Move only the tail after the edit.

If the document is 10MB and you change six bytes near the end, serde still rebuilds ~10MB of output. jshift moves a tail and rewrites the field.

4. Safe vs “unchecked” path engines (gjson)

gjson is an excellent, battle-tested read engine. Its hottest skip loops use techniques like continuous bulk scans, and in the Rust port, unchecked indexing (get_unchecked) in the inner loop. That can buy speed on “skip a huge array to a trailing key.”

jshift deliberately does not.

Choice jshift Typical max-speed finder (e.g. gjson hot path)
unsafe Forbidden (forbid(unsafe_code)) Used in hot loops for fewer bounds checks
Primary goal Find and mutate with byte offsets Find / query
Failure mode on bad input Result / syntax errors along the path Often best-effort on malformed JSON
Dependency / trust surface One crate, auditable safe code Speed via unsafe assumptions

We do absorb a few ideas that transfer cleanly to safe Rust:

  • bulk “squash” of nested containers,
  • tight string skipping,
  • unrolled scans over non-structural bytes,

…and left the remaining gap on pure “key-last + 10MB array” finds as a conscious trade for memory safety and mutator correctness. If your only requirement is the absolute fastest read and you accept unsafe, evaluate gjson. If you need safe in-place mutation, jshift is the fit.

In other words: speed without safety is a different product. jshift optimizes under the constraint forbid(unsafe_code) because many teams cannot take “we get_unchecked in a skip loop” into a security review — even when that loop is carefully written.


Capabilities (full API map)

Public surface as of 0.6.0 (#![forbid(unsafe_code)]). Convenience map for agentic workflows and humans reading crates.io / GitHub.

Typed documents (0.5–0.6 — center of gravity)

API Role
TypedDoc / TypedDocRef Owned / borrowed buffer; exclusive mutate()
JsonDoc Shared read trait: get / get_opt / get_nullable / get_str / contains / presence / view_at / each_* / each_get / …
TypedMutator Exclusive splice: set / upsert / delete / rename_key / merge_shallow / merge_patch / apply_ops
ViewList / ValueList Array cursors (len / get / iter / collect_owned)
IndexedViewList / IndexedValueList list.index() → O(1) get(i) after one walk
NestedView / MapView / IndexedMapView Subtree cursors; string-key maps; O(1) multi-get
CollectPolicy / project_view_each Stream|owned|projected; typed array stream
ArrayElems / ObjectEntries / ObjectEntry Raw element / DynObject key-value cursors
RootKind / Presence Root type + missing vs null vs value
RawJson Dynamic pocket (owned subtree bytes)
from_jshift_bytes / project_as_view / to_schema_bytes View decode; project keep-list; schema emit
TypedDoc::from_view Emit from JsonView onto {}
merge_patch / Limits / check_document RFC 7396; DoS depth/size guards

Build without Value (0.5–0.6)

API Role
ObjectBuilder / ArrayBuilder Fluent emit; in-place nesting (single buffer)
JsonWriter Imperative encoder (key / value / begin_* / auto-close finish / pretty)
object_from_iter / array_from_iter / build_object / build_array Helpers

Batch mutate & validate (0.5–0.6)

API Role
MutateOp / BatchPlan / apply_ops Ordered multi-op plans (set/upsert/delete/rename/merge/merge_patch)
rename_key / merge_object_shallow Key rename; shallow object merge
require_paths / require_paths_non_null Required fields present
deny_unknown_keys / validate_open / validate_closed Open vs closed (no DOM)
type_at / require_type / JsonType Span type sniff

Find (zero-copy path scan)

API Role
find_value Locate a path → &[u8] subslice into the buffer
parse_path / try_parse_path Dot/bracket paths → PathSegments
Path / Path::parse / Path::from_json_pointer Owned paths; JSON Pointer support
OwnedPathSegment / PathSegment Path AST types

Mutate (in-place on Vec<u8>)

API Role
mutate_value Overwrite value at path (caller supplies JSON bytes)
mutate_value_checked Same + validates the replacement span
upsert_object_key Insert or update a key under an object path
upsert_at_path Upsert leaf; create missing object parents as {}
delete_key Remove key/value; fix commas / whitespace. Early keys memmove the buffer tail — prefer trailing keys / sparse open-view edits on large docs
append_to_array Append element with comma injection
prepend_to_array Insert at index 0
insert_array_element Insert at index (0 = prepend, len = append)
delete_index Remove array element; fix commas. Mid/front deletes shift the array tail
array_len Count array elements without allocating a Vec of them
rename_key / merge_object_shallow See batch/validate section

Project (emit smaller JSON)

API Role
project / project_into Run a ProjectPlan → new buffer
project_paths Keep-list convenience
project_jmespath JMESPath subset on raw spans (not a DOM port)
project_write Stream projected document into any Write
projected_len / estimate_projected_len / estimate_values_len Exact / ballpark sizes
ProjectPlan / ProjectStyle / MissingPolicy Compact / Pretty / PreserveSource; missing policy
SelectExpr / ArraySelect / ObjectSelect / HashField / CmpOp Selection AST
parse_jmespath_expr / parse_project_path / select_from_project_path Parsers
WriteSink / CountingSink Emit sinks
Transform / TransformPipeline KeepPaths, Jmes, Rename, Drop, Inject, Style

Thin cards & streaming (no giant output array)

API Role
project_object_fields / plan_object_fields Array path + field list → card array (any schema)
project_each / project_each_indexed Callback per list-projection element: [*], filter [?…], and slice; one-card peak RAM
project_object_fields_each / _indexed Thin-field each-callback
project_jsonl_write / project_jsonl_write_indexed NDJSON lines to Write (inherits filter/slice peel from project_each)
project_object_fields_jsonl_write Thin-field NDJSON

Index-wired project & parallel

API Role
project_indexed Project with prebuilt IndexedDocument
project_indexed_prepare / project_auto_indexed Index plan paths then project
plan_prefers_parallel Heuristic: heavy filters/calls vs thin pure fields
project_indexed_auto / project_parallel_auto Seq vs Rayon by heuristic
project_parallel / project_indexed_parallel / project_object_fields_parallel Feature parallel (Rayon)

Structural indexes (opt-in)

API Role
IndexedDocument Side-tables for arrays/objects; find, for_each_element, index_for_plan
index_array / index_array_str / index_object / index_structural Build tables (via IndexedDocument methods)
build_array_index / build_object_key_index / build_structural_index Free functions
ArrayIndex / ObjectKeyIndex / StructuralIndex Table types
static_array_prefixes_from_path Derive helper for schema array prefixes

Never forced on default find_value / project / read_from_json.

JSONL / multi-document

API Role
json_lines / JsonLines Zero-copy iterator over non-empty lines
jsonl_docs / jsonl_docs_owned TypedDocRef / TypedDoc per line
read_jsonl Map each line through JsonView
read_jsonl_indexed / read_line_indexed Per-line index + view
write_jsonl_line / write_jsonl_views / write_jsonl_docs NDJSON out without Value
for_each_jsonl_line / for_each_jsonl_doc Stream callbacks

Views, documents, derive

API Role
JsonView Trait: read_from / read_from_indexed / write_into / project helpers
read_view / write_view / from_jshift_bytes / project_as_view Free helpers
SharedDocument Arc<[u8]> shared buffer; typed_ref / to_typed_doc
#[derive(JsonView)] / JsonMutatorSchema Feature derive (default): FIELD_PATHS, mutators (set_*, array insert/delete), project_json, #[json(jmes)], #[json(default)] (0.5)
Error Path / syntax / JMES / type / Decode / MissingField / UnknownField (non_exhaustive)

Convert / escape

API Role
FromJsonSlice / ToJsonBytes Numbers, bool, String, Vec, Option, …; write_json_bytes (stack itoa for integers)
from_json_string / escape_json_string / escape_json_key String helpers
write_json_string / write_json_string_content Append escaped strings into a buffer

Cargo features

Feature Default Enables
derive yes Proc-macro schemas
parallel no Rayon list project APIs
dhat-heap no Allocator profiling example only

Related crates

Crate Role Prefer when…
serde_json Full AST / typed serde Complete models, validation, general transforms
gjson Fast path reads (may use unsafe in hot loops) Read-only queries; absolute max skip speed
sonic-rs / simd-json High-performance parse Full document parse with speed focus
jshift Path scan + in-place mutate + schema derive + safe Selective R/W, JSONL, gateways, safe-only codebases

In other words: these crates are not enemies on a single leaderboard — they are tools for different jobs. Serde owns the typed application core. gjson/sonic own pure read races (sometimes with different safety trade-offs). jshift owns safe selective read+write and projection on raw buffers. Mature systems often use more than one.


Development

cargo test
cargo clippy --workspace --all-targets -- -D warnings
cargo bench --bench json_benchmark -- "Compete Find"

See CONTRIBUTING.md and CHANGELOG.md.


License

Licensed under either of:

at your option.