Skip to main content

Module cursor

Module cursor 

Source
Expand description

The lexer’s scan cursor. This is the one place the port uses unsafe (decision “B”): a raw *const u8 cursor over the source buffer for parity with the C++ lexer’s pointer arithmetic. The buffer is held as an Rc<SourceBuffer> (stable heap address; kept alive for the cursor’s life), and every public method converts to/from a byte offset, so nothing unsafe escapes this module. The buffer is NUL-terminated, so peek_at one past the last real byte reads the terminating 0 (in-bounds).

§Safety invariants

  • start, cur, and end all point into the single contiguous allocation owned by buffer (the Rc<SourceBuffer> keeps it alive for self’s life, and SourceBuffer’s storage is a Vec<u8> whose data pointer is stable while the buffer is alive — it is never mutated).
  • start is the first byte, end points at the trailing NUL (index bytes().len()), so [start, end] is in-bounds and end itself is a valid, readable byte (the NUL).
  • Byte offsets are u32 (matching the front end’s SMLoc), so source buffers are assumed to be smaller than 4 GiB.
  • cur is always kept within [start, end] by the public methods. The C++ lexer dereferences *curCharPtr_ at end (reading the NUL) and uses bounded lookahead (curCharPtr_[1], …) only after seeing a non-NUL byte, which is also in-bounds because of the terminator. We preserve that contract: peek_at(n) is only used by callers respecting the same lookahead invariant.

Structs§

Cursor
A raw-pointer scan cursor over a NUL-terminated source buffer.