jshift 0.7.0

Schema-guided typed JSON on raw bytes: path get/mutate, open views, project — without a Value tree.
Documentation
//! # API guide — when to use what
//!
//! jshift exposes several surfaces that look similar (`JsonView`, path gets,
//! project, multi-page streams, optional serde/axum bridges). **Picking the
//! wrong one for the job** is the main way to get surprising benches (e.g.
//! serde beating a `JsonView` card on multi-field ingest).
//!
//! This guide is the product map. Prefer it over inventing a parallel style.
//!
//! # One-line product
//!
//! **Schema-guided JSON on raw bytes:** sparse path read, open in-place mutate,
//! project/stream — without a `serde_json::Value` tree as the spine.
//!
//! # When to use which API
//!
//! | Job | Prefer | Avoid for this job |
//! | --- | --- | --- |
//! | **Hot path:** 1–few fields (`id`, `seq`, flag) | [`JsonDoc::get`] / [`each_get`](JsonDoc::each_get) / [`find_value`] | Full [`JsonView`] card, `Value` |
//! | **Ingest / stage tables** (cols + optional `raw_json`) | [`each_with`](JsonDoc::each_with) + [`find_value`] (static [`PathSegment`]s); raw = element span | `JsonView` multi-string card as the hot loop |
//! | **Domain object** (handler, business logic, open mutate) | `#[derive(JsonView)]` + [`read_from`](JsonView::read_from) / [`write_into`](JsonView::write_into) | Hand-rolled `find_value` soup for 20 fields |
//! | **Thin JSON out** (drop variants/images) | [`project`] / [`project_jmespath`] / [`ProjectPlan`] | Deserialize full tree then re-serialize |
//! | **Archive bronze** (exact wire bytes) | Write the buffer; hold [`TypedDoc`] if you need paths later | Parsing into Arrow “for storage” if you only need raw |
//! | **Paginated / multi-file / JSONL** | [`for_each_page_items`], [`for_each_jsonl_multi`], [`JsonlDecoder`] | Concatenate pages into one mega-buffer |
//! | **HTTP body → view** (axum) | feature `axum`: [`JshiftJson`](crate::JshiftJson) / [`JshiftDoc`](crate::JshiftDoc) | `Json<Value>` then convert |
//! | **Must satisfy `Deserialize`** | feature `serde`: [`AsJshiftView`](crate::AsJshiftView) — see [serde bridge guide](crate::serde_bridge_guide) | Implementing `Deserialize` on every jshift type by default |
//! | **Externally tagged enums** | `#[derive(JsonView)]` default, or [`read_external_*`](crate::read_external) | `Value` match on tags |
//! | **Internally / adjacent / untagged enums** | `#[json(tag = …)]` / `content` / `untagged` (0.7+) — see [`crate::enums`] | Assuming only external exists |
//!
//! [`JsonDoc::get`]: crate::JsonDoc::get
//! [`find_value`]: crate::find_value
//! [`PathSegment`]: crate::PathSegment
//! [`project`]: crate::project
//! [`project_jmespath`]: crate::project_jmespath
//! [`ProjectPlan`]: crate::ProjectPlan
//! [`TypedDoc`]: crate::TypedDoc
//! [`for_each_page_items`]: crate::for_each_page_items
//! [`for_each_jsonl_multi`]: crate::for_each_jsonl_multi
//! [`JsonlDecoder`]: crate::JsonlDecoder
//!
//! # Ingest pattern (Arrow / Parquet hand-off)
//!
//! jshift does **not** build Arrow rows. Your loop does:
//!
//! ```rust,ignore
//! use jshift::{find_value, FromJsonSlice, JsonDoc, PathSegment, TypedDocRef};
//!
//! const P_ID: &[PathSegment] = &[PathSegment::Key("id")];
//! const P_TITLE: &[PathSegment] = &[PathSegment::Key("title")];
//!
//! let doc = TypedDocRef::from_slice(page_json);
//! doc.each_with("products", |elem| {
//!     let id = u64::from_json_slice(find_value(elem, P_ID)?)?;
//!     let title = String::from_json_slice(find_value(elem, P_TITLE)?)?;
//!     let raw_json = std::str::from_utf8(elem)?; // full product span
//!     // id_col.append_value(id);
//!     // title_col.append_value(&title);
//!     // raw_col.append_value(raw_json);
//!     let _ = (id, title, raw_json);
//!     Ok(())
//! })?;
//! // finish builders → RecordBatch → Parquet (arrow/parquet crates, not jshift)
//! ```
//!
//! | Layer | Owner |
//! | --- | --- |
//! | Nested JSON on the wire | Network / files |
//! | Sparse fields + raw spans | **jshift** |
//! | Column appends / `RecordBatch` | **Your Rust** (or a thin adapter) |
//! | Parquet / IPC / Iceberg | **arrow / parquet / lakehouse** |
//!
//! **Arrow does not force serde.** Builders work without it. Serde is optional for
//! JSON decode if you choose that stack instead of jshift.
//!
//! # Why `JsonView` can lose to serde on multi-field ingest
//!
//! `#[derive(JsonView)]` is optimized for **correct open documents and ergonomics**,
//! not for beating serde’s single-pass tokenizer on “decode 5 strings from every
//! fat object.” Each path field is a selective scan. For stage throughput, use
//! **`each_with` + `find_value`** (and accept that multi-key single-pass extract is
//! a future API). See `examples/json_parquet_stage_bench.rs` and
//! `examples/bronze_land_bench.rs`.
//!
//! # Edge decode vs true raw land
//!
//! | Prefer **edge decode** when… | Prefer **raw land** when… |
//! | --- | --- |
//! | Hot path needs fields before land finishes | Exact wire archive is the only goal |
//! | Stage is schema-on-write (Parquet rows) at the collector | Silver will reprocess and has CPU |
//! | You must drop PII / fat tails before storage | Schema is unstable |
//! | Routing / dedupe needs `id` / `account` now | Feed is already binary / Arrow |
//!
//! # Open vs closed documents
//!
//! | Mode | Meaning | Typical API |
//! | --- | --- | --- |
//! | **Open** | Unmentioned fields unread / preserved on write | path get, `write_into`, open nested write |
//! | **Closed** | Schema-only card | `to_schema_bytes`, `write_view_at_closed`, project keep-list |
//!
//! Open is the product default for API evolution (unknowns stay).
//!
//! # Enums (tagging styles)
//!
//! | Style | Wire | Derive |
//! | --- | --- | --- |
//! | **External** (default) | `"Ready"` / `{"Click":{…}}` | `#[derive(JsonView)]` |
//! | **Internal** | `{"type":"Click","x":1}` | `#[json(tag = "type")]` |
//! | **Adjacent** | `{"t":"Click","c":{…}}` | `#[json(tag = "t", content = "c")]` |
//! | **Untagged** | try variants in order | `#[json(untagged)]` |
//!
//! Free functions: [`read_external`](crate::read_external),
//! [`read_internal`](crate::read_internal), [`read_adjacent`](crate::read_adjacent),
//! [`read_untagged`](crate::read_untagged). Module overview: [`crate::enums`].
//!
//! # Feature flags (keep core light)
//!
//! | Feature | When |
//! | --- | --- |
//! | default `derive` | Almost always |
//! | `serde` | Framework requires `Deserialize` / gradual migration |
//! | `axum` | Axum extractors |
//! | `parallel` | Huge indexed array project |
//!
//! Core never depends on Arrow/Parquet/Iceberg.
//!
//! # Examples to run
//!
//! ```text
//! cargo run --release --example teefury_ingest_bench --features derive
//! cargo run --release --example bronze_land_bench --features derive
//! cargo run --release --example json_parquet_stage_bench --features derive
//! cargo run --release --example multi_json_bench --features derive
//! ```
//!
//! # Stability note (pre-1.0)
//!
//! The surfaces above are the intended long-term map. Multi-key single-pass
//! extract and further enum polish may land before 1.0 without changing this
//! “which API for which job” table.