Skip to main content

Module utf8_lib

Module utf8_lib 

Source
Expand description

The utf8 standard library: char, codepoint, codes, len, offset, and charpattern. Present from Lua 5.3.

GRADUATED (Phase-2 idiomatization). The line-by-line C correspondence to lutf8lib.c has been removed; what guards this module now is the behavioral net, in two layers:

  • the official 5.4 suite (reference/lua-5.4.7-tests/utf8.lua, run via the harness) and the cross-crate multiversion_oracle, and
  • crates/lua-stdlib/tests/utf8_strengthen.rs, which pins the version seam the 5.4-only suite cannot see: the decode/encode regime differs between 5.3 and 5.4+.

The version seam is a single source of truth, [DecodeMode], resolved once per call by [decode_mode_for]:

  • 5.3 — cap at MAX_UNICODE, at most a 4-byte sequence, surrogates ALWAYS accepted, the lax argument ignored (5.3’s utf8_decode has no strict/lax parameter). charpattern stops the lead byte at \xF4.
  • 5.4+ strict (default) — cap at MAX_UTF while decoding, then reject surrogates and values above MAX_UNICODE.
  • 5.4+ lax — cap at MAX_UTF, accept surrogates. charpattern widens the lead byte to \xFD.

LOAD-BEARING (do not reshape — only the validity ceiling is version-split): the continuation-byte decode loop in [utf8_decode] and the backward-fill encode in [encode_utf8_codepoint]. Those carry the bit-exact arithmetic and are documented in place.

Constants§

FUNCS
Function registration table for the utf8 library.

Functions§

open_utf8
Open the utf8 library.