Lexical analyzer — port of llex.c + llex.h.
Provides the Lua 5.4 lexer: character-by-character scanning of a [ZIO]
input stream into [Token] values, with one-token lookahead. The
llex.h header is merged here per PORTING.md §1.
C source files
reference/lua-5.4.7/src/llex.c(581 lines, 24 functions)reference/lua-5.4.7/src/llex.h(91 lines; merged here)
Design notes
LexState.L(back-pointer tolua_State) is removed. All functions that needLuaStatereceive it asstate: &mut LuaState.Token.tokenisi32in Phase A (matching the Cint tokenfield). Single-byte tokens are their ASCII values; reserved-word tokens start atFIRST_RESERVED(257). A properTokenKindenum is deferred to Phase B.save/save_and_nextare now fallible (Result<(), LuaError>); the?operator replaces the C noreturnlexerrorcall on buffer overflow.- The
goto read_save / only_save / no_savepattern inread_stringis translated via the localEscapeResultenum.