Skip to main content

Module tables

Module tables 

Source
Expand description

Unicode tables pinned to the oracle’s Python (3.11, Unicode 14.0), so the core’s string rules do not drift with Rust’s newer Unicode.

  • ND_ZEROS: the zero of every run of Unicode Nd digits. Python’s re \d and int() accept any Nd digit, so UnityVersion.from_str does (its port is version/parse.rs; plan section 3.4).
  • LOWER_EXCEPTIONS: code points whose str.lower() under Unicode 14.0 differs from Rust’s char::to_lowercase, with Python’s result. Every other code point lowers as Rust lowers it (crate::pystr::py_lower, plan risk R7).
  • SIGMA_CASED and SIGMA_CASE_IGNORABLE: the two classes of code point that decide a capital sigma’s lower-case form under Unicode 14.0 (Final_Sigma: a cased character before it and none after it, skipping case-ignorable ones), read from Python 3.11’s own str.lower(). A case-ignorable character’s casedness is never read, so the cased table lists only cased characters that are not case-ignorable (crate::pystr::py_lower; D request 5).

Owner: package A. The tables are generated by tools/gen/gen_tables.py in the oracle image (plan section 2.6), each under a header naming the generator, its Python and, for LOWER_EXCEPTIONS, the rustc whose char::to_lowercase it was compared with; gen_tables.py --check regenerates and compares. A toolchain whose Unicode differs from that rustc’s fails tests::the_exceptions_were_generated_for_this_toolchain, so the table moves with the toolchain (plan risk R7).

Statics§

LOWER_EXCEPTIONS
(code point, Python's lower-case string) pairs, sorted by code point.
ND_ZEROS
The first code point of each Nd run (each run is ten digits, 0-9), sorted.
SIGMA_CASED
Cased code points that are not case-ignorable, as inclusive ranges, sorted.
SIGMA_CASE_IGNORABLE
Case-ignorable code points, as inclusive ranges, sorted.

Functions§

is_case_ignorable
Whether c is case-ignorable, as Python 3.11’s final-sigma context sees it.
is_sigma_cased
Whether c is cased and not case-ignorable, as Python 3.11’s final-sigma context sees it.
lower_exception
Python’s lower-case form of c when it differs from Rust’s.
nd_digit
The digit value of c if it is a Unicode Nd digit (as Python 3.11 sees it).