Struct Scanner 

pub struct Scanner<'a> { /* private fields */ }

A stateful lexical cursor over one immutable source text.

A caller may drive it token by token with Scanner::next_token and, at a grammatical decision point, request an explicit reinterpretation with one of the rescan_*/scan_jsx_* operations.

Implementations

impl<'a> Scanner<'a>

pub fn new( source_id: SourceId, script_kind: ScriptKind, source: &'a SourceText, ) -> Self

Creates a scanner positioned at the start of source.

pub const fn script_kind(&self) -> ScriptKind

Returns the syntax this scanner lexes.

pub const fn position(&self) -> Utf16Pos

Returns the current UTF-16 cursor position.

pub fn is_at_end(&self) -> bool

Returns whether the whole source has been consumed.

pub fn diagnostics(&self) -> &[Diagnostic]

Returns the diagnostics recorded so far, in emission order.

pub fn into_diagnostics(self) -> Vec<Diagnostic>

Consumes the scanner and returns its recorded diagnostics.

pub fn next_token(&mut self) -> Token

Scans the next token, or an end-of-file token at the end of the source.

pub fn rescan_regex(&mut self) -> Token

Reinterprets the most recent ///= token as a regular-expression literal starting at the same position, advancing past its body and flags.

This is the explicit division-versus-regex decision. The caller supplies the grammatical context; the scanner never guesses it.

pub fn rescan_greater_than(&mut self) -> Token

Reinterprets the most recent >-family token as a single >, advancing exactly one code unit past the last token’s start.

A caller closing type arguments or a JSX element uses this to split a greedily formed shift/compound operator.

pub fn rescan_template_continuation(&mut self) -> Token

Reinterprets the most recent } token as a template continuation, producing TokenKind::TemplateMiddle or TokenKind::TemplateTail.

This serves a caller that drives the scanner without relying on the single-pass brace tracking used by scan.

pub fn scan_jsx_text(&mut self) -> Token

Scans a run of JSX character data up to the next < or {.

The token kind is TokenKind::StringLiteral: the fixed token space has no dedicated JSX kind, and JSX text is uninterpreted character content whose lexeme the caller reads directly. The run may be empty when the caller is already positioned at a < or {.

pub fn scan_jsx_identifier(&mut self) -> Token

Scans a JSX name, which unlike an ECMAScript identifier admits interior hyphens (for example data-role).

pub fn scan_jsx_attribute_string(&mut self) -> Token

Scans a JSX attribute string, which is delimited by matching quotes, performs no escape processing, and may span line terminators.