ingraph 0.1.0

A declaration-driven indexing framework: records and nodes are written as Rust types, and the storage rows, the dataflow between them and the GraphQL query surface are emitted from the declaration.
Documentation
//! [`LanguageTag`] — RFC 5646's SYNTAX, and the structural reduction the lane gate's first step is.

use crate::{WritingSystem, name::folds_to};

#[cfg(test)]
mod tests;

/// **A LANGUAGE TAG, REDUCED TO WHAT A LANE CAN BE KEYED ON** — a primary subtag and an optional
/// script, with everything a lane does not partition on dropped.
///
/// ```
/// use ingraph::{LanguageTag, WritingSystem};
///
/// let reduced = LanguageTag::parse("zh-Hant-TW").expect("a well-formed tag");
///
/// assert_eq!(reduced.language(), "zh");
/// assert_eq!(reduced.script_subtag(), Some("Hant"));
/// // The region went, because no lane is keyed on where a speaker lives.
/// assert_eq!(LanguageTag::parse("en-Latn-US-posix").unwrap().language(), "en");
/// ```
///
/// # SYNTAX ONLY, and the registry deliberately absent
///
/// RFC 5646 has two halves: a grammar, and the IANA subtag registry that says which words the
/// grammar's slots may hold. This reads the GRAMMAR and carries no registry at all, which is the
/// ruling's own division — `sw`, `zzz` and `qaa` all parse here, and whether any of them names a
/// lane is the roster's question rather than a registry's.
///
/// The alternative was a table of nine thousand subtags that goes stale on IANA's schedule, to
/// answer a question the roster answers better: a tag naming a language nobody declared a lane for
/// is refused either way, and refused with a more useful sentence by the party that knows the
/// roster.
///
/// # WHAT IT DROPS, and why each
///
/// | subtag | kept | why |
/// |---|---|---|
/// | primary language | **yes** | it is what a lane is keyed on |
/// | extlang | no | `zh-cmn` is a variety of `zh`, and the lane is `zh` |
/// | script | **yes** | `zh-Hans` and `zh-Hant` may be two lanes or one, and only the roster decides |
/// | region | no | `en-US` and `en-GB` are one analyzer's problem |
/// | variant, extension, private use | no | none of them changes which tokenizer a text wants |
///
/// The script is kept and nothing else optional is, and that asymmetry is the whole reason the
/// reduction is not just *take the first subtag*: a deployment that genuinely needs simplified and
/// traditional in separate lanes has to be able to say so, and a deployment that does not folds them
/// with an alias.
///
/// # IT BORROWS, and the case is the FOLD's business
///
/// The subtags are slices of the tag handed in, spelled however its author spelled them, because
/// materialising a canonical form would be a second authority on what two spellings mean. What
/// decides that is this crate's ONE fold, spent at [`matches`](Self::matches) — so `zh-Hant`,
/// `zh_hant` and `ZH-HANT` are one tag without any of them being rewritten.
///
/// `_` is accepted as a separator beside `-`, which is the *broad in* half of the gate meeting
/// reality: BCP-47 writes `zh-Hant`, POSIX and Java write `zh_Hant`, and a declaration writing a
/// lane's aliases in Rust-ish spelling writes the second.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
pub struct LanguageTag<'a> {
  /// The primary language subtag, as written.
  language: &'a str,
  /// The script subtag, as written, where the tag carried one.
  script: Option<&'a str>,
}

impl<'a> LanguageTag<'a> {
  /// **REDUCE `tag`** — [`None`] where it is not a language tag this grammar recognises.
  ///
  /// [`None`] rather than a refusal, because the party that can word the refusal usefully is the
  /// gate: it holds the roster, and *`sw` is not a lane here, the lanes are `en`, `zh`, `ja`* is a
  /// sentence this function has no way to write.
  ///
  /// What is refused is a tag with no primary subtag this grammar can read — an empty string, a
  /// private-use tag (`x-whatever`), a grandfathered irregular one (`i-klingon`), or a first subtag
  /// that is not two to eight ASCII letters.
  #[must_use]
  pub fn parse(tag: &'a str) -> Option<Self> {
    let mut subtags = tag.split(['-', '_']);
    let language = subtags.next()?;
    if !alpha(language, 2, 8) {
      return None;
    }

    let mut rest = subtags.peekable();
    // EXTLANG, skipped: three letters after a two- or three-letter primary can only be one, since a
    // region is two letters or three DIGITS. RFC 5646 allows up to three, and every one of them
    // names a variety of the primary — which is the lane the reduction is heading for anyway.
    if language.len() <= 3 {
      for _ in 0..3 {
        match rest.peek() {
          Some(subtag) if alpha(subtag, 3, 3) => {
            rest.next();
          }
          _ => break,
        }
      }
    }

    let script = rest.peek().filter(|subtag| alpha(subtag, 4, 4)).copied();

    Some(Self { language, script })
  }

  /// The primary language subtag, as written.
  #[inline]
  #[must_use]
  pub const fn language(&self) -> &'a str {
    self.language
  }

  /// The script subtag, as written, where the tag carried one.
  #[inline]
  #[must_use]
  pub const fn script_subtag(&self) -> Option<&'a str> {
    self.script
  }

  /// The script this tag NAMES, read through the segmenter's own vocabulary.
  ///
  /// It is what the tag SAYS rather than what its text is written in — a claim by whoever produced
  /// the tag, where [`WritingSystem::of`](crate::WritingSystem::of) is an observation about a codepoint. The two
  /// answer different questions and a deployment may hold both.
  ///
  /// A subtag outside the segmenter's roster lands on that vocabulary's own word escape
  /// ([`WritingSystem::Other`](crate::WritingSystem::Other)) rather than being lost — `zh-Hant` answers
  /// `Other("hant")`, because Han is unified and no codepoint could ever say `Hant`.
  #[inline]
  #[must_use]
  pub fn script(&self) -> Option<WritingSystem> {
    self.script.map(WritingSystem::new)
  }

  /// **WHETHER THIS TAG NAMES `candidate`** — the comparison the gate is built out of.
  ///
  /// `candidate` is reduced by the same rules and the two are compared PART BY PART through this
  /// crate's one fold, so `zh-Hant` matches a `zh_hant` written in a declaration and `EN-us`
  /// matches `en`. A candidate this grammar cannot read matches nothing.
  ///
  /// # A SCRIPT IS PART OF THE KEY, and absence is not a wildcard
  ///
  /// `zh-Hant` does NOT match the lane keyed `zh`, and that is what makes the alias table mean
  /// something: a deployment holding one Chinese lane writes `zh` accepting `zh-Hant`, and one
  /// holding two writes both keys. A match that ignored an absent script would make the second
  /// unsayable, because every traditional tag would already have been taken by the first lane it
  /// was compared against.
  /// **WHETHER THIS TAG'S LANGUAGE NAMES `candidate`'s** — the comparison the gate's DEGRADATION
  /// steps are built out of, and the only place a script subtag is deliberately ignored.
  ///
  /// `zh-Hant` answers `true` against `zh`, and `sr-Latn` against `sr`. It reads the same primary
  /// subtag [`language`](Self::language) already exposes, through the same fold — there is no second
  /// parse and no second notion of what a language is.
  ///
  /// # It is NOT a looser [`matches`](Self::matches), and the gate is why
  ///
  /// The two answer different questions and the gate asks them in order: an EXACT match is a lane
  /// that was declared for this tag, and a language-only one is the nearest home a tag has when
  /// nobody declared its script. Collapsing them would delete that distinction, which is the whole
  /// of *declared outranks derived*.
  #[must_use]
  pub fn matches_language(&self, candidate: &str) -> bool {
    LanguageTag::parse(candidate)
      .is_some_and(|candidate| folds_to(self.language, candidate.language))
  }

  #[must_use]
  pub fn matches(&self, candidate: &str) -> bool {
    let Some(candidate) = LanguageTag::parse(candidate) else {
      return false;
    };

    folds_to(self.language, candidate.language)
      && match (self.script, candidate.script) {
        (None, None) => true,
        (Some(mine), Some(theirs)) => folds_to(mine, theirs),
        _ => false,
      }
  }
}

/// Whether `subtag` is `low` to `high` ASCII letters and nothing else.
///
/// The one shape test the grammar needs, and it is BYTE-WISE because RFC 5646's alphabet is: a
/// subtag is ASCII by the standard's own definition, so a multi-byte character in one is not a
/// subtag whose case might fold — it is not a subtag.
fn alpha(subtag: &str, low: usize, high: usize) -> bool {
  (low..=high).contains(&subtag.len()) && subtag.bytes().all(|byte| byte.is_ascii_alphabetic())
}