Crate nova_lexer 

Lexer for the Nova programming language.

Converts Nova source text into a Vec<Spanned<Token>>. Uses logos for the core token recognition and a hand-written state machine wrapper for string interpolation ("Hello, ${name}!").

An interpolation hole is lexed inline — its tokens go straight into the one stream, bracketed by InterpOpen/InterpClose — rather than being captured as a span and re-lexed. The hole therefore ends at the } that matches its ${, which the lexer tracks with a per-hole brace counter so that "${f(R { v: 1 })}" works.

Usage

use nova_diagnostics::{FileDb, FileId};
use nova_lexer::lex;

let mut db = FileDb::new();
let file = db.add("<test>", r#"fn main() { println("hi") }"#);
let (tokens, errors) = lex(r#"fn main() { println("hi") }"#, file);
assert!(errors.is_empty());

Structs

Comment

A comment the lexer skipped.

Lexer

Stateful lexer that wraps the logos-generated token recognizer and adds string-interpolation tracking.

Enums

CommentKind

Which kind of comment.

LexError

A non-fatal error produced by the lexer.

Token

Every token produced by the Nova lexer.

Constants

KEYWORDS

Every keyword, in the order the lexer declares its tokens: the words completion offers, and the extension’s grammar colours (spec docs/superpowers/specs/2026-10-08-phase-3-2-lsp-core-design.md §4.3, §7.4). tests/keywords.rs keeps it equal to the lexer’s alphabetic tokens.

Functions

lex

Lex source and return all tokens and any errors encountered.

lex_with_comments

lex, plus every comment it skipped, in source order: what the formatter needs to keep them (spec docs/superpowers/specs/2026-10-07-phase-3-1-formatter-design.md §3.1). The tokens and errors are exactly lex’s.