Skip to main content

Module lexer

Module lexer 

Source
Expand description

Shell lexer for the $fy translator.

Produces a flat, lossless token stream: every character of the input ends up in exactly one token’s text, so the formalizer can rebuild the original source from the links network.

Structs§

Token
One lexed token.

Enums§

TokenType
Token kinds emitted by tokenize.

Functions§

tokenize
Tokenizes a complete shell script. The stream always ends with an EOF token.