Module incremental_json 

Incremental scanning of concatenated / partial JSON.

When consuming a streamed body you often receive JSON in arbitrary byte chunks and need to know where one top-level JSON value ends and the next begins — without pulling in a full serde dependency. JsonSplitter is a dependency-free, byte-level scanner that tracks structural depth, string state and escapes, and emits the byte ranges of complete top-level values as they arrive.

It does not validate the full JSON grammar (e.g. it will not reject {,}); it tracks just enough state — brackets, braces, strings, escapes — to find correct value boundaries, which is exactly what streaming framing needs. Whitespace and separators (commas / newlines) between top-level values are skipped, so it works for both NDJSON and bare concatenation.

Because the default splitter is non-validating, malformed structure is not flagged: a stray leading close bracket (e.g. } or ] at depth 0, as in }{"a":1}) is emitted as a one-byte “value” rather than rejected — the depth counter saturates at zero rather than going negative. Feed it well-formed framing (which is what every LLM streaming API actually sends) and this never arises.

If you must guard against adversarial framing, either validate each emitted value with a real JSON parser, or construct the splitter in strict mode with JsonSplitter::strict. In strict mode a structural violation (a close bracket with no matching open at depth 0) is recorded as a MalformedJson error, surfaced from the next JsonSplitter::feed return value and from JsonSplitter::finish, and the offending byte is dropped instead of being emitted as a bogus value.

Example

use stream_rs::incremental_json::JsonSplitter;

let mut s = JsonSplitter::new();
let mut out = Vec::new();
s.feed(br#"{"a":1}{"b":"#, &mut out);
assert_eq!(out, vec![r#"{"a":1}"#.to_string()]);

out.clear();
s.feed(br#"[1,2]}"#, &mut out); // completes the second object
assert_eq!(out, vec![r#"{"b":[1,2]}"#.to_string()]);

When the stream ends, call JsonSplitter::finish: it flushes a trailing bare scalar that no separator terminated (e.g. a final 42 or true with no newline), and returns a FinishError if structural data was left dangling inside an object, array or string, or — in strict mode — if a framing violation was seen earlier in the stream.

Structs

JsonSplitter

Splits a byte stream into complete top-level JSON values.

Enums

FinishError

Error returned by JsonSplitter::finish.