Struct Lexer 

pub struct Lexer<'input, const MAX_DEPTH: usize = DEFAULT_MAX_DEPTH> { /* private fields */ }

Stateless JSON lexer over a borrowed byte slice.

Each read_* method consumes one syntactic token from the input. The caller is responsible for invoking the right method at the right time — there is no state machine that says “after reading a key, you must call expect_byte(b':') next.” Typed consumers (FromJson impls) drive the lexer directly; streaming consumers go through Parser instead.

Implementations

impl<'input, const MAX_DEPTH: usize> Lexer<'input, MAX_DEPTH>

pub const fn new(input: &'input [u8]) -> Self

Construct a lexer over input.

Panics

Panics if input.len() > MAX_INPUT_LEN (~2 GB). The packed offset representation in JsonStr/JsonNum reserves the top bit of a u32 for has_escapes, so positions are limited to 31 bits. Real-world JSON documents are far smaller than this; consumers needing larger streams should chunk and parse incrementally.

pub const fn position(&self) -> Position

pub const fn input(&self) -> &'input [u8]

The input slice the lexer was constructed with. Consumers use this to materialize &str/&[u8] from JsonStr and JsonNum, which store offsets rather than fat pointers.

pub const fn offset(&self) -> usize

pub const fn checkpoint(&self) -> Checkpoint

Snapshot the current cursor and nesting depth. Pair with restore to roll the lexer back after a speculative parse — typically used by #[bourne(untagged)] enum dispatch to try variants in order.

The returned Checkpoint is opaque: do not construct one yourself or mix checkpoints across different Lexer instances.

pub const fn restore(&mut self, cp: Checkpoint)

Roll the lexer back to a previously taken Checkpoint.

Restores both the byte cursor and the container-frame stack depth. Intended for the speculative-retry pattern: take a checkpoint, attempt a parse, on Err call restore and try a different shape.

The checkpoint must have been produced by self.checkpoint(). If the checkpoint is from a different lexer or from after a restore to a deeper depth, behavior is logically incoherent (the truncate no-ops if asked to grow), though never memory-unsafe.

pub fn peek_value_kind(&mut self) -> Result<ValueKind, Error>

Skip whitespace then peek at the next byte to determine the kind of value that begins there. Does not consume the byte. Returns UnexpectedEof if there is no next byte.

pub fn read_value(&mut self) -> Result<Event, Error>

Skip whitespace then read one full JSON value, returning the matching Event. For containers, opens the container (consuming [ or {) and pushes a frame onto the nesting stack — the caller is responsible for matching ]/} later via read_array_continue / read_object_continue (or by going through Parser).

pub fn read_string(&mut self) -> Result<JsonStr, Error>

Read a JSON string token. Cursor must be at the opening ". Returns a JsonStr covering the bytes between the quotes (exclusive).

When the body contains escape sequences, the deferred validate_escapes pass runs before returning — every escape is checked for syntactic validity (escape kind, hex digits, surrogate pairing). The contract for stream/Event consumers: a returned JsonStr with has_escapes() == true means the escapes are well-formed.

pub fn read_string_no_validate(&mut self) -> Result<JsonStr, Error>

Like read_string, but skip the deferred validate_escapes pass. The caller commits to performing equivalent validation as part of decoding (the typed String / Cow<str> impls do exactly this).

This exists because validate_escapes and an eager decoder do overlapping work: the deferred validation walks the body checking every escape; the decoder walks the body to actually emit the decoded form, and naturally has to inspect every escape anyway. On profile, the redundant validate_escapes walk was 42% of total time on the mixed_length_strings_with_escapes corpus when going to Vec<String>. Skipping it here gives the typed path a faster route without weakening the stream-consumer contract.

pub fn read_number(&mut self) -> Result<JsonNum, Error>

Read a JSON number token. Cursor must be at - or a digit. Returns a JsonNum covering the literal.

pub fn parse_i64_value(&mut self) -> Result<i64, Error>

Parse a JSON integer directly into i64, fusing lex and conversion.

Caller must position the lexer at the first byte of the value (after any whitespace). Returns the parsed i64 and leaves the cursor at the byte after the number. Rejects fractional and exponent forms — those are not integers.

pub fn parse_i128_value(&mut self) -> Result<i128, Error>

Parse a JSON integer directly into i128, fusing lex and conversion.

Same shape as parse_i64_value but scaled to 128-bit. On profile, routing Vec<i128> through JsonNum::as_i128 (which calls str::parse::<i128>) was 60% of the workload; the generic str::parse path uses checked arithmetic on every digit. The fast path here skips overflow checks for the first 38 digits (which always fit a u128) and only pays them on the 39-digit boundary case.

Caller must position the lexer at the first byte of the value. Rejects fractional and exponent forms.

pub fn parse_u128_value(&mut self) -> Result<u128, Error>

Parse a JSON unsigned integer directly into u128, fusing lex and conversion. Rejects negative literals, fractional, and exponent forms.

Caller must position the lexer at the first byte of the value.

pub fn parse_f64_value(&mut self) -> Result<f64, Error>

Parse a JSON number directly into f64, fusing lex and decode.

Caller must position the lexer at the first byte of the value (after any whitespace). Skips the JsonNum middle layer that read_number() + JsonNum::as_f64 would build — saves one Option-wrapped slice and the per-element struct construction. On profile, Vec<f64> was ~50% slower than Vec<i64> largely because of this missing fast path.

Rejects non-finite results (out-of-range literals like 1e400 decode to ±inf from str::parse::<f64>, which JSON disallows).

pub fn parse_str_value(&mut self) -> Result<&'input str, Error>

Read a JSON string and return it as a borrowed &'input str. Errors if the string contains escape sequences — those require a caller-owned decode buffer, which json-bourne does not allocate.

Caller must position the lexer at the opening ". On return the cursor is past the closing ". The returned slice points into the original input — zero copy.

pub fn array_continue(&mut self, end_byte: u8) -> Result<bool, Error>

Skip whitespace then expect , or the array-end byte. Returns true if at end (caller should stop), false to continue with another element.

On ] this also pops the matching frame from the nesting stack so a subsequent operation resumes correctly in the enclosing context.

pub fn object_first_key(&mut self) -> Result<Option<&'input str>, Error>

After a StartObject, return the next key as a borrowed &'input str, or None if the object closes immediately.

pub fn object_first_key_lex(&mut self) -> Result<Option<JsonStr>, Error>

Like object_first_key, but returns the key as a raw JsonStr span.

pub fn object_next_key(&mut self) -> Result<Option<&'input str>, Error>

After a field’s value, advance to the next key or close the object.

pub fn object_next_key_lex(&mut self) -> Result<Option<JsonStr>, Error>

Like object_next_key, but returns the key as a raw JsonStr span.

pub fn array_start(&mut self) -> Result<bool, Error>

Expect the byte that opens an array ([), advance past it, push a nesting frame, and skip whitespace to the first element (or ]). Returns true if the array is empty (the closing ] has just been consumed and the frame has been popped).

pub fn object_start(&mut self) -> Result<(), Error>

Expect the byte that opens an object ({), advance past it, push a nesting frame, and skip whitespace. Does not consume any keys.

pub fn finish(&mut self) -> Result<(), Error>

Require that no further (non-whitespace) data follows. Used by the typed entry point to reject "1 2" and similar.

pub fn skip_value(&mut self) -> Result<(), Error>

Consume one complete JSON value and discard it.

Walks whatever value sits at the cursor — primitive, string, number, array, or object — and advances past it. For composite values, every nested element is also skipped. The structural frame stack stays balanced: this method pushes and pops the same frames read_value would.

Validation is the same as read_value: malformed input (control char in string, lone surrogate, malformed number, etc.) still raises an Error. The “skip” here means “throw away the value”, not “throw away the parse”. A lenient deny_unknown_fields = false consumer wants the cursor advanced, but it still wants the surrounding object to parse correctly afterward — which requires lexing the skipped value to find its end.

Used by #[derive(FromJson)] when a struct opts into #[bourne(deny_unknown_fields = false)] to consume the value associated with an unrecognized key.

Trait Implementations

impl<'input, const MAX_DEPTH: usize> Debug for Lexer<'input, MAX_DEPTH>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.