ingraph 0.1.0

A declaration-driven indexing framework: records and nodes are written as Rust types, and the storage rows, the dataflow between them and the GraphQL query surface are emitted from the declaration.
Documentation
//! [`WritingSystem`] — the ISO 15924 vocabulary, its roster, and the door a word arrives through.

use core::{convert::Infallible, fmt, str::FromStr};

use smol_bytes::Utf8Bytes;

use crate::name::{folded, folds_to};

#[cfg(test)]
mod tests;

/// **A WRITING SYSTEM**, spelled the way ISO 15924 spells one.
///
/// The roster is the set this crate carries a CODEPOINT TABLE for — the scripts a routing story
/// distinguishes, because each of them wants a different analyzer — plus the standard's own three
/// answers for the characters that belong to no single script. Everything outside the roster is
/// [`Zzzz`](Self::Zzzz) when it comes from a codepoint and [`Other`](Self::Other) when it comes from
/// a word.
///
/// # THE TWO ESCAPES ARE DIFFERENT QUESTIONS, and conflating them would lose a fact
///
/// [`Zzzz`](Self::Zzzz) means *this crate's table does not name this character's script* — the
/// answer [`of`](Self::of) gives, and the bucket every unrouted run lands in. [`Other`](Self::Other)
/// means *somebody wrote a script code this roster does not carry* — the answer [`new`](Self::new)
/// gives, and it carries the word so the thing they wrote is not lost.
///
/// A segmenter can only ever produce the first: a table that does not know a character cannot know
/// what to call it. So `Other("cher")` is a word a declaration wrote, never a run's script, and the
/// two never have to be told apart by guessing.
///
/// # It is an OPEN vocabulary, and the door FOLDS
///
/// ISO 15924 writes its codes in title case (`Latn`) and BCP-47 carries them that way; a
/// declaration in this framework writes lowercase (`latn`); a producer somewhere writes `LATN`.
/// Those are one word in three spellings, so the door reads every word through this crate's own
/// [`folds_to`] — the ONE fold, workspace-wide — and the canonical word each variant answers to is
/// its code, lowercased.
///
/// That is also the whole of what makes this ISO 15924 rather than a pile of four-letter words: the
/// standard's own spelling of every member parses, and a downstream reading a BCP-47 script subtag
/// hands it straight to [`new`](Self::new).
///
/// # `#[non_exhaustive]` is NOT written here
///
/// The roster grows when a script's analyzer story does, and this crate is unpublished — so a new
/// member is an ordinary edit rather than a compatibility event, and the standing rule against
/// compatibility ceremony in an unpublished crate applies. What a downstream gets instead is an
/// EXHAUSTIVE match: a router that must send every run somewhere is the whole consumer of this
/// type, and a wildcard arm is exactly the thing that would let a new script be silently dropped
/// into a fallback nobody chose.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum WritingSystem {
  /// **Arabic** — `arab`. Arabic, Persian, Urdu.
  Arab,
  /// **Armenian** — `armn`.
  ///
  /// Its own bicameral casing and its own punctuation set, so a folding analyzer needs Armenian's case pairs rather than Latin's.
  Armn,
  /// **Bengali** — `beng`. Bengali, Assamese.
  ///
  /// A Brahmic abugida whose conjuncts are formed with an explicit virama, so a token is a grapheme cluster rather than a codepoint run.
  Beng,
  /// **Bopomofo** — `bopo`. Chinese phonetic annotation.
  ///
  /// Written without spaces beside Han and never inside a Han word, so it needs Han's dictionary segmentation but must not be fed to a Han dictionary.
  Bopo,
  /// **Canadian Aboriginal Syllabics** — `cans`. Inuktitut, Cree, Ojibwe.
  ///
  /// A syllabary whose finals attach to the preceding syllable, so breaking on codepoints splits words a reader would not.
  Cans,
  /// **Ethiopic** — `ethi`. Amharic, Tigrinya, Ge'ez.
  ///
  /// A syllabary with its OWN word separator (`\u{1361}`) beside the space, so a whitespace tokenizer silently keeps whole phrases as one term.
  Ethi,
  /// **Georgian** — `geor`.
  ///
  /// Its casing is unlike every other bicameral script — Mtavruli maps only in one direction — so Latin's case folding is wrong for it in both.
  Geor,
  /// **Gujarati** — `gujr`.
  ///
  /// Brahmic, and its own vowel-sign repertoire, so cluster boundaries differ from Devanagari's despite the shared ancestry.
  Gujr,
  /// **Gurmukhi** — `guru`. Punjabi.
  ///
  /// Brahmic, with subjoined consonants that a codepoint-wise break would separate from their base.
  Guru,
  /// **Khmer** — `khmr`.
  ///
  /// Written with NO spaces between words at all — spaces mark phrases — so it needs dictionary segmentation exactly as Thai does.
  Khmr,
  /// **Kannada** — `knda`.
  ///
  /// Brahmic, with a distinctive consonant-conjunct form that a cluster-unaware analyzer breaks mid-word.
  Knda,
  /// **Lao** — `laoo`.
  ///
  /// No inter-word spaces, the same segmentation class as Thai and needing its own dictionary.
  Laoo,
  /// **Malayalam** — `mlym`.
  ///
  /// The heaviest conjunct load of the Brahmic family, plus chillu letters, so cluster handling is not optional here.
  Mlym,
  /// **Mongolian** — `mong`. Traditional Mongolian.
  ///
  /// Vertical, cursive-joining, and it carries its own free-variation selectors, so both the joining and the direction differ from every other roster member.
  Mong,
  /// **Myanmar** — `mymr`. Burmese, Shan, Mon.
  ///
  /// No inter-word spaces, and syllable clusters that reorder visually, so segmentation is both dictionary-driven and cluster-aware.
  Mymr,
  /// **Odia** — `orya`.
  ///
  /// Brahmic, with its own virama behaviour, so its clusters do not match Devanagari's rules.
  Orya,
  /// **Sinhala** — `sinh`.
  ///
  /// Brahmic, with two distinct vowel-sign series, so a cluster boundary depends on which series is in play.
  Sinh,
  /// **Syriac** — `syrc`.
  ///
  /// Right-to-left and cursive-joining like Arabic, but with its own diacritic set and its own abbreviation mark.
  Syrc,
  /// **Tamil** — `taml`.
  ///
  /// Brahmic, and the one that does NOT form conjuncts — so an analyzer tuned to Devanagari's clusters over-merges Tamil.
  Taml,
  /// **Telugu** — `telu`.
  ///
  /// Brahmic, with below-base conjuncts that a naive break detaches from their base.
  Telu,
  /// **Thaana** — `thaa`. Dhivehi.
  ///
  /// Right-to-left, and an abugida whose vowels are obligatory diacritics — so stripping marks, as a Latin folder would, destroys the word.
  Thaa,
  /// **Tibetan** — `tibt`. Tibetan, Dzongkha.
  ///
  /// The tsheg separates SYLLABLES rather than words, so a delimiter-driven tokenizer produces syllables and never terms.
  Tibt,
  /// **Cyrillic** — `cyrl`. Russian, Ukrainian, Bulgarian, Serbian.
  Cyrl,
  /// **Devanagari** — `deva`. Hindi, Marathi, Nepali.
  Deva,
  /// **Greek** — `grek`.
  Grek,
  /// **Hangul** — `hang`. Korean.
  Hang,
  /// **Han** — `hani`. Chinese, and the kanji half of Japanese.
  ///
  /// UNIFIED: simplified and traditional share these codepoints, so this is the only answer a Han
  /// character can give and the distinction lives in a language tag one layer away.
  Hani,
  /// **Hebrew** — `hebr`.
  Hebr,
  /// **Hiragana** — `hira`. The Japanese syllabary a sentence's grammar is written in.
  Hira,
  /// **Katakana** — `kana`. The Japanese syllabary loanwords are written in.
  Kana,
  /// **Latin** — `latn`. English, French, German, Spanish, Vietnamese, Indonesian, and most of the
  /// rest.
  Latn,
  /// **Thai** — `thai`.
  Thai,
  /// **INHERITED** — `zinh`, ISO 15924's *code for inherited script*.
  ///
  /// A combining mark takes the script of whatever it is attached to, so it has none of its own.
  /// [`is_contextual`](Self::is_contextual) is true here, and [`script_runs`](super::script_runs) is
  /// where the taking-from is done.
  Zinh,
  /// **COMMON** — `zyyy`, ISO 15924's *code for undetermined script*.
  ///
  /// Spaces, digits, punctuation and most symbols: characters every writing system uses, which
  /// therefore belong to none. [`is_contextual`](Self::is_contextual) is true here too.
  Zyyy,
  /// **UNKNOWN** — `zzzz`, ISO 15924's *code for uncoded script*, and this crate's FALLBACK BUCKET.
  ///
  /// What [`of`](Self::of) answers for a character the roster's table does not name, and what a run
  /// made of nothing but [`Zyyy`](Self::Zyyy) and [`Zinh`](Self::Zinh) wears. It is a legitimate
  /// destination rather than an error: a router sends these somewhere, and *somewhere* is a
  /// deployment's choice.
  Zzzz,
  /// A SCRIPT CODE SOMEBODY WROTE that this roster does not carry — the word's escape, folded.
  ///
  /// Never produced by [`of`](Self::of). It is what [`new`](Self::new) answers for `"cher"`,
  /// `"Runr"` or anything else outside the roster, and it carries the word so a refusal further
  /// down the line can name what it was handed.
  Other(Utf8Bytes),
}

impl WritingSystem {
  /// **THE ROSTER**, escape EXCLUDED — every script this crate carries a codepoint table for, plus
  /// the standard's three non-script answers.
  ///
  /// The escape is not a member, on the open-vocabulary device's own law: this is the list a reader
  /// consults to learn what may be NAMED, and `Other` is what happens when something outside it is.
  pub const VARIANTS: &'static [Self] = &[
    Self::Arab,
    Self::Armn,
    Self::Beng,
    Self::Bopo,
    Self::Cans,
    Self::Cyrl,
    Self::Deva,
    Self::Ethi,
    Self::Geor,
    Self::Grek,
    Self::Gujr,
    Self::Guru,
    Self::Hang,
    Self::Hani,
    Self::Hebr,
    Self::Hira,
    Self::Kana,
    Self::Khmr,
    Self::Knda,
    Self::Laoo,
    Self::Latn,
    Self::Mlym,
    Self::Mong,
    Self::Mymr,
    Self::Orya,
    Self::Sinh,
    Self::Syrc,
    Self::Taml,
    Self::Telu,
    Self::Thaa,
    Self::Thai,
    Self::Tibt,
    Self::Zinh,
    Self::Zyyy,
    Self::Zzzz,
  ];

  /// **THE WORD'S DOOR**, and it cannot refuse.
  ///
  /// `word` is read through this crate's [`folds_to`] — ASCII case and the three separators — so
  /// `Latn`, `latn` and `LATN` are one value, and the standard's own title-case spelling parses
  /// beside the lowercase one a declaration writes. Anything the roster does not name becomes
  /// [`Other`](Self::Other) carrying the FOLDED word, which is what makes two spellings of one
  /// unrostered code compare and hash alike.
  #[must_use]
  pub fn new(word: &str) -> Self {
    for member in Self::VARIANTS {
      if folds_to(word, member.as_str()) {
        return member.clone();
      }
    }

    Self::Other(Utf8Bytes::from(folded(word)))
  }

  /// **WHICH SCRIPT A CHARACTER IS WRITTEN IN** — UAX #24's question, answered off this crate's own
  /// range table.
  ///
  /// [`Zyyy`](Self::Zyyy) and [`Zinh`](Self::Zinh) for the characters that belong to every script
  /// and to none, and [`Zzzz`](Self::Zzzz) for everything the table does not name — which is a
  /// deliberate spread rather than a gap, and the `script` module's note says what the table is FOR.
  ///
  /// Never [`Other`](Self::Other): a table that does not know a character cannot know its name.
  #[must_use]
  pub fn of(character: char) -> Self {
    super::table::of(character)
  }

  /// This script's canonical word — its ISO 15924 code, lowercased.
  ///
  /// The escape answers the folded word it carries, so what [`Display`](fmt::Display) writes is what
  /// [`new`] reads back for every value including that one.
  ///
  /// [`new`]: Self::new
  #[must_use]
  pub fn as_str(&self) -> &str {
    match self {
      Self::Arab => "arab",
      Self::Armn => "armn",
      Self::Beng => "beng",
      Self::Bopo => "bopo",
      Self::Cans => "cans",
      Self::Cyrl => "cyrl",
      Self::Deva => "deva",
      Self::Ethi => "ethi",
      Self::Geor => "geor",
      Self::Grek => "grek",
      Self::Gujr => "gujr",
      Self::Guru => "guru",
      Self::Hang => "hang",
      Self::Hani => "hani",
      Self::Hebr => "hebr",
      Self::Hira => "hira",
      Self::Kana => "kana",
      Self::Khmr => "khmr",
      Self::Knda => "knda",
      Self::Laoo => "laoo",
      Self::Latn => "latn",
      Self::Mlym => "mlym",
      Self::Mong => "mong",
      Self::Mymr => "mymr",
      Self::Orya => "orya",
      Self::Sinh => "sinh",
      Self::Syrc => "syrc",
      Self::Taml => "taml",
      Self::Telu => "telu",
      Self::Thaa => "thaa",
      Self::Thai => "thai",
      Self::Tibt => "tibt",
      Self::Zinh => "zinh",
      Self::Zyyy => "zyyy",
      Self::Zzzz => "zzzz",
      Self::Other(word) => word.as_str(),
    }
  }

  /// **WHETHER THIS CHARACTER TAKES ITS SCRIPT FROM ITS NEIGHBOURS** — true for
  /// [`Zyyy`](Self::Zyyy) and [`Zinh`](Self::Zinh), and for nothing else.
  ///
  /// UAX #24's own division, and the one predicate [`script_runs`](super::script_runs)' resolution
  /// is written in: a space between two Han characters is part of the Han run, and an acute accent
  /// over an `e` is part of the Latin one. A character that is contextual mints no run of its own.
  ///
  /// [`Zzzz`](Self::Zzzz) is NOT contextual, and that is load-bearing. A character from a script the
  /// table does not name is a character a Han or Latin analyzer would mangle, so it forms its own
  /// run and lands in the fallback bucket — where a deployment can see it — rather than being
  /// swallowed by whatever happened to precede it.
  #[must_use]
  pub const fn is_contextual(&self) -> bool {
    matches!(self, Self::Zyyy | Self::Zinh)
  }

  /// Whether this is the WORD's escape — a code somebody wrote that the roster does not carry.
  #[must_use]
  pub const fn is_other(&self) -> bool {
    matches!(self, Self::Other(_))
  }
}

/// What [`WritingSystem::as_str`] answers, so a value written out is a value [`WritingSystem::new`] reads back.
impl fmt::Display for WritingSystem {
  #[inline]
  fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
    f.write_str(self.as_str())
  }
}

/// **AN OPEN DOOR CANNOT REFUSE**, and the type says so: every word names a value, so the error is
/// [`Infallible`] and a caller writing `?` has nothing to handle.
impl FromStr for WritingSystem {
  type Err = Infallible;

  #[inline]
  fn from_str(word: &str) -> Result<Self, Self::Err> {
    Ok(Self::new(word))
  }
}

impl From<&str> for WritingSystem {
  #[inline]
  fn from(word: &str) -> Self {
    Self::new(word)
  }
}

impl AsRef<str> for WritingSystem {
  #[inline]
  fn as_ref(&self) -> &str {
    self.as_str()
  }
}