tsv_svelte 0.8.0

//! ## The crate's whitespace discipline — read this before adding a whitespace test
//!
//! `tsv_svelte` has exactly **two** standing whitespace questions, one class each, and which
//! one a site wants follows from the question it is answering — never from the language the
//! code is written in. A read that answers neither is an exception and has to argue for itself;
//! the three that do are enumerated at the bottom of this doc.
//!
//! | question | class | where |
//! | --- | --- | --- |
//! | does this END A TOKEN / separate a keyword? (parse) | [`is_svelte_ws`] — JS `\s` | parser, lexer, and any printer read that MIRRORS one |
//! | may a formatter add, drop or respell this without changing what RENDERS? | [`is_collapsible_ws`](crate::ast::is_collapsible_ws) — `[ \t\n\r]` | printer text/fill |
//!
//! A third question is the DIFFERENCE of the two: which characters does the parse delete at the
//! very end of the document (`template.trimEnd()`, JS `\s`) that the compiler renders
//! everywhere else? [`is_end_trimmed_content`] answers it, for the one printer read that asks —
//! the template text the section reorder leaves at the end of the output.
//!
//! ⚠️ **Rust's own whitespace is neither, and reaching for it is the recurring bug.**
//! `str::trim*`, `str::split_whitespace` and `char::is_whitespace` are Unicode `White_Space`,
//! which disagrees with JS `\s` in BOTH directions — it **lacks U+FEFF** and **adds U+0085
//! NEL** — while `u8::is_ascii_whitespace` and a hand-written `b' ' | b'\t' | …` byte match are
//! narrower still, missing every non-ASCII member.
//!
//! ⚠️ **The trap is not this crate's**, and treating it as this crate's is how the rest of it
//! stayed open after the sites below were fixed: it has been found in FOUR crates, and the
//! workspace-wide index — every class, which oracle each answers to, and the sites in
//! `tsv_lang` / `tsv_ts` / `tsv_css` — is the module doc on `tsv_lang`'s `whitespace.rs`,
//! beside the shared predicate. The five sites HERE, each reached only by a spelling no
//! fixture and no real repo contains, and each with its own symptom:
//!
//! - an unquoted attribute value's terminator (a BYTE match) absorbed the character, turning
//!   an expression attribute into a quoted string the Svelte compiler rejects;
//! - a `{:…}` continuation run cut at a LITERAL SPACE, so `{:else⏎if x}` over-rejected and
//!   `{:else⇥junk}` over-accepted;
//! - the document's trailing-text `trim_end` dropped a NEL canonical keeps and kept a U+FEFF
//!   canonical drops;
//! - the printer's `lang` read trimmed with `str::trim`, so `lang="<ZWNBSP>css"` froze a body
//!   prettier formats. ⚠️ The only one of the five whose answer is **not** [`is_svelte_ws`]:
//!   that class alone fails the mirror-image case (`lang="<NEL>css"`), so the trim took the
//!   union instead — see the last bullet below;
//! - `scan_past_trailing_comments` (the wire's mimic of acorn's post-expression token scan)
//!   ended early, so `{expr<NBSP>/* c */}` lost the comment's `trailingComments` attachment
//!   AND its root `comments` entry.
//!
//! Three whitespace reads here are deliberately NOT [`is_svelte_ws`], and each carries its own
//! argument. None of them is "Rust's class, and safe" — one is *narrower* than either class
//! above, and naming it Rust's is what would send the next sweep at a site that is already
//! right:
//!
//! - `is_horizontal_ws` (printer/text.rs) is a BYTE match, `[ \t\r]` — narrower than
//!   everything above, and answering to Svelte's own `regex_not_whitespace` (`/[^ \t\r\n]/`).
//!   It asks whether a byte lets a NEWLINE RUN continue; a form feed is rendered content
//!   there, so it must END the run, which every wider class would get wrong.
//! - `is_separator_like_text` (printer/nodes/fragment_doc.rs) is the one read that really is
//!   Rust's `char::is_whitespace`, and deliberately **wider** than either class above: it asks
//!   "is this text node a separator rather than a word for the fill to pack", where an NBSP is
//!   a separator and a `[ \t\n\r]` reading would call it prose.
//! - `narrow_lang_value`'s trim (ast/mod.rs) is wider still — the UNION of JS `\s` and
//!   Rust's `White_Space` — and mirrors no oracle at all: tsv's formattable-`lang` list is an
//!   ALLOWLIST that the trim feeds, so widening can only route a name toward formatting, while
//!   either class alone gives one witness away. Argued in full at that function.
//!
//! ⚠️ Every OTHER `str::trim*` still in this crate is **unclassified, not sanctioned** — the
//! list above is a record of the reads that have been argued, not a clean bill of health for
//! the rest. Ask a new one which oracle it mirrors before assuming Rust's class is fine.

/// Whether `c` is whitespace to **Svelte's parser** — the class that separates tokens,
/// ends a tag-name or attribute-name run, and satisfies a block keyword's required space.
///
/// This is exactly JS's `\s` ([`tsv_lang::is_js_whitespace`], which owns the set and
/// its exhaustive test), because every place Svelte asks the question is spelled in
/// JS: `is_whitespace(cc)` (`1-parse/index.js`, backing `allow_whitespace` /
/// `require_whitespace`) enumerates these code points by hand, and the name-run regexes match
/// the same set through `\s` — `regex_whitespace_or_slash_or_closing_tag = /(\s|\/|>)/` for
/// tag names and `regex_token_ending_character = /[\s=/>"']/` for attribute and directive
/// names (both in `1-parse/state/element.js`), as do the raw-text closes `/<\/script\s*>/`,
/// `/<\/style\s*>/` and the RCDATA close `/<\/textarea(\s[^>]*)?>/iy`.
///
/// It keeps its own name here because the *question* is Svelte-specific — "does this end a
/// token in this parser" — while the shared definition answers only "is this JS `\s`". The
/// two ⚠️ traps (`char::is_whitespace()` differs in both directions; U+FEFF and U+0085 are
/// the witnesses) live with the definition.
///
/// ⚠️ Not [`is_collapsible_ws`](crate::ast::is_collapsible_ws), the narrower render
/// class (`[ \t\n\r]`). That one answers "may a formatter add, drop or respell this without
/// changing what renders"; this one answers "does this end a token". A form feed is
/// whitespace here and rendered content there.
///
/// ⚠️ And **not** `tsv_ts`'s `is_es_whitespace`, which is the `WhiteSpace` production
/// *alone*: a JS lexer matches `LineTerminator` separately because a newline drives ASI,
/// so that predicate deliberately omits `<LF>`/`<CR>`/`<LS>`/`<PS>`. This class is the
/// union of both productions (that is what `\s` means), so it is a strict superset.
pub(crate) use tsv_lang::is_js_whitespace as is_svelte_ws;

/// The character at byte offset `i` in `source` and its UTF-8 width, or `None` at or past
/// the end. `i` must be a character boundary.
///
/// The shared scanning primitive behind the name-run and tag-close scans. It dispatches on
/// the raw byte and **decodes only on the non-ASCII branch** (an ASCII byte *is* its own
/// `char`), the way the lexer's own cursor does — these scans run per character of every
/// name in the document, where an unconditional decode is a measurable regression.
///
/// Returning the width is what keeps a caller on character boundaries: a multi-byte
/// character has to be stepped over whole, because the next call slices `source[i..]`.
/// Handing back the `char` rather than a bool lets each scan ask its own question of it, so
/// the byte-vs-char dispatch exists once instead of once per predicate.
#[inline]
pub(crate) fn char_at(source: &str, i: usize) -> Option<(char, usize)> {
    let b = *source.as_bytes().get(i)?;
    if b.is_ascii() {
        return Some((b as char, 1));
    }
    let c = source[i..].chars().next()?;
    Some((c, c.len_utf8()))
}

/// Byte offset of the first character at/after `start` that `is_terminator` accepts, or the
/// end of `source` when none does. The name is `source[start..end]`.
///
/// The one scan behind both of Svelte's `read_tag` name runs — the tag-name class
/// (`parser/element.rs`) and the attribute/directive class (`parser/attribute.rs`) ask the
/// same question of the same cursor and differ **only** in that predicate, so the loop lives
/// here once and each class stays its own named predicate
/// (`is_tag_name_terminator` / `is_attr_name_terminator` — the two sets are genuinely
/// different, and collapsing them into one predicate would be the bug).
///
/// Generic over the predicate rather than taking a `fn` pointer so each caller monomorphizes
/// to its own inlined class test: this runs per character of every name in the document.
#[inline]
pub(crate) fn name_run_end(
    source: &str,
    start: usize,
    is_terminator: impl Fn(char) -> bool,
) -> usize {
    let mut end = start;
    while let Some((c, width)) = char_at(source, end) {
        if is_terminator(c) {
            break;
        }
        end += width;
    }
    end
}

/// The UTF-8 width of the Svelte-whitespace character at byte offset `i` in `source`, or
/// `None` when `i` is at/past the end or the character there is not whitespace.
///
/// The scanning form of [`is_svelte_ws`], for the tag-close scans.
#[inline]
pub(crate) fn svelte_ws_width_at(source: &str, i: usize) -> Option<usize> {
    let (c, width) = char_at(source, i)?;
    is_svelte_ws(c).then_some(width)
}

/// Byte offset of the first non-whitespace character at/after `start`, or the end of `source`
/// when none follows.
///
/// Svelte's `allow_whitespace()` (`1-parse/index.js`) over a raw offset rather than the parser
/// cursor — for the scans that own their own cursor and only resync the lexer once at the end.
#[inline]
pub(crate) fn skip_svelte_ws(source: &str, start: usize) -> usize {
    let mut i = start;
    while let Some(width) = svelte_ws_width_at(source, i) {
        i += width;
    }
    i
}

/// Byte offset where the interior of the `{` at `brace_pos` begins — Svelte's `eat('{')`
/// followed by `allow_whitespace()`.
///
/// ⚠️ **The one question every brace-led reader must ask, and the one place to ask it.**
/// The gap between a brace and what follows it is any width, so a reader that reaches its
/// content by a fixed `brace_pos + 2` has baked in width **zero** — and both bugs that
/// produced this helper were exactly that: `parse_attach_tag` read the author's space as the
/// `attach` keyword's first byte, and the sequence placement guard missed a marker one space
/// over, then rejected the glued form its own printer produced from it. A named helper is
/// what makes a fourth reader that forgets visible; three independent `skip_svelte_ws(source,
/// pos + 1)` calls agreeing was invisible.
///
/// ⚠️ **A fixed offset is not the only spelling of the mistake** — resuming from a lexer
/// TOKEN's end is the same assumption wearing a different hat, since the marker braces are one
/// token ACROSS the gap. That is a question about token boundaries rather than about byte
/// arithmetic, so it is not this helper's to answer; it is answered at
/// `parser::attribute::SvelteParser::attribute_name_run_end`, which is where a static
/// `<script>` head's `{ #a}` folded the author's whitespace into an attribute name. A sweep
/// for this class has to read both spellings.
///
/// `brace_pos + 1` is always a char boundary — `{` is one byte.
#[inline]
pub(crate) fn brace_interior_start(source: &str, brace_pos: usize) -> usize {
    skip_svelte_ws(source, brace_pos + 1)
}

/// Whether `c` is template **content** that Svelte's `parse` deletes when it ends the document.
///
/// The parser reads `template.trimEnd()`, and JavaScript's trim class is ECMAScript `WhiteSpace`
/// plus `LineTerminator` ([`is_svelte_ws`], JS `\s`). The compiler's own whitespace, the one
/// that decides what renders, is only `[ \t\r\n]`
/// ([`is_collapsible_ws_char`](crate::ast::is_collapsible_ws_char)) — so the
/// difference is a set of characters that render as text anywhere but at the very end of the
/// source: `<VT>`, `<FF>`, U+00A0, U+FEFF, the other `Zs` spaces, U+2028 and U+2029. Not Rust's
/// `char::is_whitespace`, which lacks U+FEFF and adds U+0085; and nothing in the C1 range, which
/// Svelte's character-reference decoder remaps through windows-1252, is ever a member.
pub(crate) fn is_end_trimmed_content(c: char) -> bool {
    is_svelte_ws(c) && !crate::ast::is_collapsible_ws_char(c)
}

#[cfg(test)]
mod tests {
    use super::*;

    /// Every ECMAScript `WhiteSpace` + `LineTerminator` code point (the class `trimEnd()`
    /// strips), split by whether Svelte's compiler renders it.
    #[test]
    fn end_trimmed_content_is_the_trim_class_minus_collapsible_whitespace() {
        let collapsible = ['\t', '\n', '\r', ' '];
        let content = [
            '\u{b}', '\u{c}', '\u{a0}', '\u{1680}', '\u{2000}', '\u{2001}', '\u{2002}', '\u{2003}',
            '\u{2004}', '\u{2005}', '\u{2006}', '\u{2007}', '\u{2008}', '\u{2009}', '\u{200a}',
            '\u{2028}', '\u{2029}', '\u{202f}', '\u{205f}', '\u{3000}', '\u{feff}',
        ];
        assert_eq!(collapsible.len() + content.len(), 25);
        for c in collapsible {
            assert!(!is_end_trimmed_content(c), "{c:?}");
        }
        for c in content {
            assert!(is_end_trimmed_content(c), "{c:?}");
        }
        // Rust's `White_Space` adds NEL; U+180E left `Zs` in Unicode 6.3; U+200B is `Cf`.
        for c in ['\u{85}', '\u{180e}', '\u{200b}', 'a', ';'] {
            assert!(!is_end_trimmed_content(c), "{c:?}");
        }
        // Exhaustive: exactly the 21 content characters, and no C1 control among them.
        let members: Vec<char> = (0..=0x10_ffff_u32)
            .filter_map(char::from_u32)
            .filter(|&c| is_end_trimmed_content(c))
            .collect();
        assert_eq!(members, content);
    }

    /// `char_at`'s two branches must agree with a plain decode at every code point — the
    /// ASCII branch skips the decoder entirely, so a divergence there would mis-scan silently.
    #[test]
    fn char_at_matches_a_plain_decode_everywhere() {
        for cp in 0..=0x10ffff_u32 {
            let Some(c) = char::from_u32(cp) else {
                continue;
            };
            let s = c.to_string();
            assert_eq!(char_at(&s, 0), Some((c, c.len_utf8())), "U+{cp:04X}");
        }
        // a mid-string offset, and the two out-of-range forms
        assert_eq!(char_at("aé", 1), Some(('é', 2)));
        assert_eq!(char_at("a", 1), None);
        assert_eq!(char_at("", 0), None);
    }

    #[test]
    fn width_at_decodes_every_member_and_stops_on_non_whitespace() {
        for cp in 0..=0x10ffff_u32 {
            let Some(c) = char::from_u32(cp) else {
                continue;
            };
            let s = c.to_string();
            assert_eq!(
                svelte_ws_width_at(&s, 0),
                is_svelte_ws(c).then(|| c.len_utf8()),
                "U+{cp:04X}"
            );
        }
        assert_eq!(svelte_ws_width_at("a", 0), None);
        assert_eq!(svelte_ws_width_at("", 0), None);
        assert_eq!(svelte_ws_width_at("a", 9), None);
    }
}