Expand description
Byte-level scanning helpers ported from markmend/core/src/preprocess/utils.ts
and pattern.ts. All positions are byte offsets into the original string.
ASCII needles are searched with memchr: in UTF-8, ASCII bytes never occur inside multi-byte chars, so the returned offsets are always char boundaries — safe to slice with.
Functions§
- append_
before_ trailing_ whitespace - Append
suffixbefore trailing whitespace. - count_
of - memchr 版 count(ASCII needle:偏移天然边界,SIMD 加速)。
- find_
closed_ code_ block_ ranges /```[\s\S]*?```/g— all closed fenced code ranges[start, end).- find_
inline_ code_ ranges /[^\n]+/-style inline code ranges[start, end)` outside code blocks.- is_
escaped_ character - Odd number of preceding backslashes.
- is_
inside_ unclosed_ code_ block - is_
position_ in_ ranges - is_
range_ overlapping_ ranges - Whether
[start, end)overlaps any range. - is_
underscore_ inside_ word - is_
within_ code_ block - is_
within_ html_ tag - Whether
positionis inside an unclosed HTML tag (<...>). - is_
within_ link_ or_ image_ url - Whether
positionis inside](...)of a link/image. - is_
within_ math_ block - Whether
positionsits inside a$$math span (or$inline math when enabled). - is_ws
- JS
\s(without unicode flag): space, tab, newline, CR, FF, VT. - join_
multiline_ math - Fold every
$$…$$span that crosses a line break onto one line, leaving fenced code blocks alone. - js_trim
- JS
String.prototype.trim-ish (ASCII + unicode whitespace). - last_
non_ empty_ line_ index - Find last non-empty line index (JS
findLastNonEmptyLineIndex). - last_
paragraph - last_
paragraph_ range - Last paragraph = content after the last blank line.
Returns
(start_line_index, byte_offset_of_start). - mask_
inline_ code_ markdown_ markers - Mask
* _ ~inside inline code with spaces (offsets preserved). - mask_
invalid_ underscore_ markers - Mask escaped / intraword underscore runs with spaces.
- remove_
math_ blocks_ from_ text - Remove
$$...$$block math (and optionally$...$inline math) spans. - remove_
urls_ from_ text - Remove code blocks, HTML tags, and link/image URL bodies from text, so markdown markers inside URLs don’t get counted.
- should_
ignore_ underscore_ marker - trailing_
ws_ offset - Trailing whitespace run (
/\s+$/), returns byte offset of the run start.