Module reader 

The reader: VLA.bas’s Tokenize and ParseAll/ParseForm, with the source lines they remember and the two refusals they raise.

The shape is the VBA’s. Tokenize walks the text once, by character: a line feed counts a line; a space, a tab or a carriage return is skipped; ( and ) are tokens; ; runs a comment to the end of its line; " opens a string literal whose only escapes are \" and \\ (any other backslash stands); everything else runs a symbol to the next delimiter. A string token carries its content behind a leading quote mark, the VBA’s own marker, until the parser makes a Form::Str of it. An unterminated string ends at the end of the text, as the VBA’s does.

Lines: every token remembers the raw line it began on, and a list remembers the line of its ( less the caller’s offset, stored as 0 when that is below 1 (the prelude’s territory; S3.1’s explicit zero). A refusal’s location is spelled by SrcLineTag and OpenedAtTag: (vla line N) and for the list opened at vla line N, or nothing at all when the line is the prelude’s. The reference also maps lines spliced in by (include ...); nothing here splices, so every line is the main file’s.

Structs

Token

One token: its text, with a string literal’s content behind a leading quote mark, and the raw line it began on (1-based).

Constants

FORM_DEPTH_LIMIT

The deepest a list may nest, the top-level list being 1 (VLA.1): the reader refuses a list opened deeper, and so does the expander’s walk, so that no stage after them recurses past it. Six times the deepest real program (the corpus golden nests 10 deep, english.vla 9, the prelude 6); every stage of the language holds well past it, and the core’s emitter, the shallowest, holds 80 in a debug build.

Functions

count_lf

VBA CountLf: how many line feeds a text holds.

opened_at_tag

VBA OpenedAtTag: for the list opened at vla line N, or nothing.

parse_all

VBA ParseAll: every top-level form of a token list. line_offset is the number of raw lines before the user’s line 1 (mLineOffset): 0 for a bare read, the prelude’s line count plus one for a compile.

read_forms

VBA VlaReadForms: the forms of a text on its own, every list tagged with its line.

read_forms_with_lines

VBA VlaReadFormsWithLines: the forms, each with its own source line (0 for an atom at top level, which carries none).

src_line_tag

VBA SrcLineTag: (vla line N), or nothing for the prelude.

tokenize

VBA Tokenize.