Skip to main content

Module unicode

Module unicode 

Source
Expand description

Unicode punctuation (general categories P and S), as CommonMark’s flanking rules define it.

micromark tests /\p{P}|\p{S}/u, i.e. the JS runtime’s Unicode tables; this table is that set pinned at Unicode 17.0 (Node 26). Regenerate when the runtime moves:

const re = /\p{P}|\p{S}/u, cps = [];
for (let c = 0x80; c <= 0x10ffff; c++)
  if ((c < 0xd800 || c > 0xdfff) && re.test(String.fromCodePoint(c))) cps.push(c);
// then collapse `cps` into inclusive ranges

Astral characters included: micromark tests UTF-16 code units, so it sees them as surrogates and never as punctuation. That is a documented divergence (DIVERGENCES.md, “Astral punctuation”), not parity.

Functions§

is_punctuation
Whether c is a Unicode punctuation character (general category P or S).
is_whitespace
micromark’s unicodeWhitespace (JavaScript \s = Unicode White_Space minus U+0085, plus U+FEFF).