Expand description
Sentence boundaries — the granularity between a word and a block.
TextDocument::sentence_at and SelectionType::SentenceUnderCursor rest on this module.
Like find_word_boundaries and SelectionType::BlockUnderCursor, a sentence is
block-scoped: a paragraph break always ends a sentence, so the search never leaves the
caret’s block.
Both queries here are pure functions over &str — no document, no store, no threads — for
the same reason document_search::matching is: a host app measuring sentence
length across a whole manuscript cannot afford to build a document per scene just to ask
how long its sentences are, and an app that rolled its own splitter would disagree with
this crate’s caret navigation about where a sentence ends.
sentence_bounds answers “which sentence contains this offset” (the caret’s question);
sentences answers “what are all the sentences” (the statistics question). Both are
built on the same boundary pass, so they cannot drift apart.
§Why UAX #29 alone is not enough
unicode-segmentation’s split_sentence_bound_indices implements UAX #29, which already
gets far more right than it is usually given credit for. Its rule SB8 suppresses a break
after a period when a lower-case word follows, which covers the bulk of real
abbreviations for free — German z.B. gestern, Russian т. д. и т. п., Swedish
bl.a. igår and Finnish esim. eilen all stay in one sentence with no help from us. Digits
behave the same way, so Nr. 5, ca. 1920 and S. 42 are fine too.
Exactly one failure mode survives: an abbreviation followed by a capitalised word. In
prose that is nearly always a title before a name — Mr. Smith, M. Dupont, Dr. Ayşe,
Sr. García, Sig. Rossi, prof. Nowak, κ. Παπαδόπουλος, د. أحمد — plus a short tail
of reference abbreviations that precede a capitalised noun (Vgl. Abb., Kap. Zwei).
Profile::abbreviations is that list and nothing more, which is what keeps it reviewable:
a term that can legitimately end a sentence must never appear in it. etc. is the
cautionary example — “…pears, etc. Then he left.” is two sentences, so suppressing etc.
would silently weld them together.
Two smaller corrections are punctuation, not vocabulary.
Spaced closing marks. UAX #29 keeps a closing quotation mark with the sentence it closes
(rules SB9/SB10 admit Close* after a terminator), so English ?", German ?« and Polish
?” all need no help. French is the exception, because it writes a space before the closing
guillemet: « Vraiment ? » strands the » at the head of the next sentence.
Profile::spaced_closers is that narrow repair, and it is deliberately per-language — "
opens as often as it closes, so a general rule here would weld He left. "Come," she said.
into one sentence.
Extra terminators. Greek asks questions with ; — an ordinary ASCII semicolon, which UAX
#29 quite correctly does not treat as a sentence ending. Profile::extra_terminators adds
it back for Greek only. (Its · is the Greek semicolon and rightly keeps not terminating.)
A language with no profile falls back to plain UAX #29. That is a real fallback rather than a stub — it mis-splits only at title + Name, and every other rule above still applies.
Hebrew needs no abbreviation list at all, and its empty profile is deliberate: Hebrew
abbreviations end in geresh (׳) or gershayim (״), not a full stop, so UAX #29 never
splits them in the first place.
Structs§
- Sentence
- One sentence of a text: the slice itself, and where it sits as char offsets.
Functions§
- sentence_
bounds - The sentence of
textcontainingchar_offset, as block-relative char offsets, with trailing whitespace trimmed off the end so a highlight stops at the terminator rather than trailing into the gap before the next sentence. - sentences
- Every sentence of
text, in order, with the same boundary rulessentence_boundsapplies — trailing whitespace trimmed, whitespace-only segments dropped.