# Qubit JSON
[](https://github.com/qubit-ltd/rs-json/actions/workflows/ci.yml)
[](https://qubit-ltd.github.io/rs-json/coverage/)
[](https://crates.io/crates/qubit-json)
[](https://www.rust-lang.org)
[](LICENSE)
[](README.zh_CN.md)
Lenient JSON decoder for Rust, designed for non-fully-trusted text inputs.
## Overview
Qubit JSON provides a small and predictable decoding layer on top of
`serde_json`. Its core type, `LenientJsonDecoder`, normalizes a limited set of
common input issues before parsing and deserializing JSON values.
The crate is intended for cases where JSON text may come from sources such as:
- Markdown-wrapped text
- Markdown code blocks using backtick or tilde fences
- copied snippets
- CLI output streams
- other text channels that may wrap otherwise valid JSON
It is intentionally narrow. The crate does not try to be a general JSON repair
engine, and it does not attempt to guess missing quotes, commas, or braces.
## Design Goals
- **Lenient but predictable**: only handle a small set of well-defined input
problems
- **Object-oriented API**: use a reusable `LenientJsonDecoder` instance instead
of a loose bag of helper functions
- **Serde-first**: delegate actual parsing and deserialization to `serde_json`
- **Privacy-aware errors**: report stable, redacted diagnostics by default and
allow detailed serde diagnostics only by explicit configuration
- **Low overhead**: avoid unnecessary allocation when normalization can borrow
the original input
## Features
### `LenientJsonDecoder`
- Reusable decoder object that holds immutable decoding options
- `decode<T>()`: decodes any JSON top-level value into `T`
- `decode_slice<T>()`: validates UTF-8 bytes and decodes them into `T`
- `decode_value()`: decodes into `serde_json::Value`
- `decode_object<T>()`: requires a top-level JSON object and deserializes `T`
directly from normalized text
- `decode_array<T>()`: requires a top-level JSON array and deserializes its
elements directly from normalized text
### `JsonDecodeOptions`
- Immutable presets, getters, and value-style builders for every option
- Presets: `lenient()` and `strict()`; strict mode disables text rewriting but
retains empty-input classification, optional size limits, privacy handling,
and stable error mapping
- `trim_whitespace`: trims leading and trailing whitespace
- `strip_utf8_bom`: strips a leading UTF-8 BOM
- `markdown_fence_policy`: selects disabled, any-language, or JSON-only fence
stripping, together with an optional or required closing fence
- The default accepts only empty, `json`, and `jsonc` fence labels.
Any-language stripping requires an explicit `MarkdownFencePolicy::Any`.
- `jsonc` is accepted only as a Markdown fence label; fenced content is still
parsed as standard JSON, so comments and trailing commas remain invalid
- `escape_control_chars_in_strings`: escapes ASCII control characters inside
JSON string literals
- `max_input_bytes`: optional byte-size limit applied before normalization
- `max_normalized_bytes`: optional byte-size limit applied to normalized JSON
before control-character repair allocates text
- `error_privacy_policy`: selects safe redacted errors (the default) or
explicitly requested detailed serde diagnostics
### Explicit Error Model
- `InputTooLarge`: raw or normalized input size exceeds its configured limit
- `EmptyInput`: input becomes empty after normalization
- `InvalidUtf8`: raw byte input is not valid UTF-8
- `InvalidJson`: normalized text is not valid JSON syntax
- `UnexpectedTopLevel`: top-level JSON kind does not match the requested method
- `Deserialize`: JSON is valid but cannot be deserialized into the target type
- `JsonDecodeError` exposes immutable accessors for the failure kind, stage,
message, top-level context, raw and normalized byte sizes, and both input
limits
- parser line and column accessors refer to normalized JSON text
- invalid UTF-8 errors expose the safe byte offset and, when known, invalid
sequence length through `utf8_valid_up_to()` and `utf8_error_len()`
- `privacy_policy()` records the policy applied to every returned error
- under the default `Redacted` policy, parser/deserializer messages do not
contain serde-provided input fragments and `Error::source()` is `None`
- `Detailed` preserves the complete UTF-8 or serde source and may therefore
expose input-derived diagnostics; use it only in controlled environments
## Installation
Add this to your `Cargo.toml`:
```toml
[dependencies]
qubit-json = "0.6"
serde = { version = "1.0", features = ["derive"] }
```
The direct `serde` dependency is only needed when deriving `Deserialize` for
typed decoding, as shown in the first quick-start example below.
If your code names `serde_json::Value` or uses `serde_json` macros, add
`serde_json` as a direct dependency. This crate intentionally does not
re-export it.
## Quick Start
### Decode a JSON Object from a Markdown Code Fence
```rust
use serde::Deserialize;
use qubit_json::LenientJsonDecoder;
#[derive(Debug, Deserialize)]
struct User {
name: String,
age: u8,
}
fn main() {
let decoder = LenientJsonDecoder::default();
let user: User = decoder
.decode_object("```json\n{\"name\":\"Alice\",\"age\":30}\n```")
.expect("decoder should extract and decode the fenced JSON object");
assert_eq!(user.name, "Alice");
assert_eq!(user.age, 30);
}
```
### Decode JSON Containing Raw Control Characters in Strings
```rust
use qubit_json::LenientJsonDecoder;
fn main() {
let decoder = LenientJsonDecoder::default();
let value = decoder
.decode_value("{\"text\":\"line 1\nline 2\"}")
.expect("decoder should escape raw control characters inside strings");
assert_eq!(value["text"], "line 1\nline 2");
}
```
### Customize Decoder Options
```rust
use qubit_json::{LenientJsonDecoder, JsonDecodeOptions};
fn main() {
let decoder = LenientJsonDecoder::new(
JsonDecodeOptions::lenient()
.with_max_input_bytes(Some(1024)),
);
let value = decoder
.decode_value("{\"ok\":true}")
.expect("plain JSON should still decode with custom options");
assert_eq!(value["ok"], true);
}
```
### Set an Input Limit for Untrusted Sources
`JsonDecodeOptions::default()` deliberately leaves `max_input_bytes` and
`max_normalized_bytes` unset so the crate does not impose application-specific
limits. When inputs cross a trust boundary, configure limits appropriate to the
caller's memory and latency budget.
`max_input_bytes` applies to raw input. `max_normalized_bytes` applies after
trimming and fence removal, and is checked before control-character repair
allocates text. Escaping one raw ASCII control byte as `\\u00XX` can expand
content from one byte to six bytes.
```rust
use qubit_json::{JsonDecodeOptions, LenientJsonDecoder};
let decoder = LenientJsonDecoder::new(
JsonDecodeOptions::default()
.with_max_input_bytes(Some(1_048_576))
.with_max_normalized_bytes(Some(6_291_456)),
);
let value = decoder.decode_value("{\"ok\":true}")?;
assert_eq!(value["ok"], true);
# Ok::<(), qubit_json::JsonDecodeError>(())
```
### Opt In to Detailed Error Diagnostics
Detailed serde diagnostics may include values from the input. Enable them only
when the diagnostic sink and its readers are trusted.
```rust
use qubit_json::{
ErrorPrivacyPolicy,
JsonDecodeOptions,
LenientJsonDecoder,
};
fn main() {
let options = JsonDecodeOptions::default()
.with_error_privacy_policy(ErrorPrivacyPolicy::Detailed);
let decoder = LenientJsonDecoder::new(options);
let error = decoder
.decode::<u64>(r#""not a number""#)
.expect_err("the JSON string cannot deserialize into u64");
assert_eq!(error.privacy_policy(), ErrorPrivacyPolicy::Detailed);
assert!(std::error::Error::source(&error).is_some());
}
```
## Normalization Rules
When enabled, the decoder applies the following pipeline before parsing:
1. enforce the optional raw input byte-size limit
2. validate that the input is not empty
3. trim surrounding whitespace
4. strip a leading UTF-8 BOM
5. trim surrounding whitespace again
6. strip one outer backtick or tilde Markdown code fence
7. trim surrounding whitespace again
8. enforce the optional normalized JSON byte-size limit before allocation
9. escape ASCII control characters inside JSON string literals
The decoder does not:
- add missing quotes
- add missing commas
- add missing braces or brackets
- rewrite arbitrary malformed JSON into guessed valid JSON
## When to Use
Qubit JSON is a good fit when:
- you need a reusable, configurable JSON decoder object
- your inputs are mostly valid JSON but may be wrapped or slightly noisy
- you want stable and safe-by-default error categories around `serde_json`
It is not a good fit when:
- you need aggressive repair for heavily malformed JSON
- your inputs are not actually JSON
- a plain `serde_json::from_str()` call is already sufficient
## Alignment Notes
This README reflects the current object model:
- `LenientJsonDecoder` owns an internal `LenientJsonNormalizer`.
- Public decoding APIs are `decode`, `decode_object`, `decode_array`,
`decode_value`, and `decode_slice`.
- Normalization and error handling are implemented in
`src/internal/lenient_json_normalizer.rs` and `src/error/json_decode_error.rs`,
which are covered by tests in `tests/`.
- Product requirements and implementation behavior are aligned with
`doc/json_prd.zh_CN.md` and `doc/json_design.zh_CN.md`.
## Development Validation
Run the repository checks with `./align-ci.sh` followed by `./ci-check.sh`.
Criterion benchmarks cover small public-entry comparisons, HTTP-style strict
byte decoding (with both reused and per-call decoder construction), LLM-style
lenient typed decoding up to 1 MiB, normalization density, and representative
failure paths. Compile them with:
```bash
cargo bench --bench decoder_bench --no-run
```
The optional fuzz target is development tooling and is not a runtime
dependency. It exercises the default, strict, JSON-only, and required-closing
decoder policies. A bounded run is scheduled by `.github/workflows/fuzz.yml`;
failures retain their reproduction artifacts. Install `cargo-fuzz` to build or
run the same target locally from the repository root:
```bash
rustup toolchain install nightly-2026-06-05 --profile minimal
cargo install cargo-fuzz --version 0.13.2 --locked
(cd fuzz && cargo +nightly-2026-06-05 fuzz build decoder)
(cd fuzz && cargo +nightly-2026-06-05 fuzz run decoder -- -max_len=4096)
```
## Testing
```bash
# Run tests with the default feature set
cargo test
# Run tests with all declared features
cargo test --all-features
# Project CI checks
./ci-check.sh
# Check code coverage
./coverage.sh
```
## License
Copyright (c) 2025 - 2026. Haixing Hu. All rights reserved.
Licensed under the Apache License, Version 2.0. See [LICENSE](LICENSE) for the
full license text.
## Contributing
Contributions are welcome. Please follow the Rust API guidelines, keep public
API documentation and tests current, and run `./align-ci.sh` to format code and
`./ci-check.sh` to satisfy CI requirements before submitting a pull request.
## Author
**Haixing Hu** - *Qubit Co. Ltd.*
Repository: [https://github.com/qubit-ltd/rs-json](https://github.com/qubit-ltd/rs-json)