Skip to main content

Module sentence

Module sentence 

Source
Expand description

Sentence boundaries — the granularity between a word and a block.

TextDocument::sentence_at and SelectionType::SentenceUnderCursor rest on this module. Like find_word_boundaries and SelectionType::BlockUnderCursor, a sentence is block-scoped: a paragraph break always ends a sentence, so the search never leaves the caret’s block.

Both queries here are pure functions over &str — no document, no store, no threads — for the same reason document_search::matching is: a host app measuring sentence length across a whole manuscript cannot afford to build a document per scene just to ask how long its sentences are, and an app that rolled its own splitter would disagree with this crate’s caret navigation about where a sentence ends.

sentence_bounds answers “which sentence contains this offset” (the caret’s question); sentences answers “what are all the sentences” (the statistics question). Both are built on the same boundary pass, so they cannot drift apart.

§Why UAX #29 alone is not enough

unicode-segmentation’s split_sentence_bound_indices implements UAX #29, which already gets far more right than it is usually given credit for. Its rule SB8 suppresses a break after a period when a lower-case word follows, which covers the bulk of real abbreviations for free — German z.B. gestern, Russian т. д. и т. п., Swedish bl.a. igår and Finnish esim. eilen all stay in one sentence with no help from us. Digits behave the same way, so Nr. 5, ca. 1920 and S. 42 are fine too.

Exactly one failure mode survives: an abbreviation followed by a capitalised word. In prose that is nearly always a title before a name — Mr. Smith, M. Dupont, Dr. Ayşe, Sr. García, Sig. Rossi, prof. Nowak, κ. Παπαδόπουλος, د. أحمد — plus a short tail of reference abbreviations that precede a capitalised noun (Vgl. Abb., Kap. Zwei). Profile::abbreviations is that list and nothing more, which is what keeps it reviewable: a term that can legitimately end a sentence must never appear in it. etc. is the cautionary example — “…pears, etc. Then he left.” is two sentences, so suppressing etc. would silently weld them together.

Two smaller corrections are punctuation, not vocabulary.

Spaced closing marks. UAX #29 keeps a closing quotation mark with the sentence it closes (rules SB9/SB10 admit Close* after a terminator), so English ?", German and Polish ?” all need no help. French is the exception, because it writes a space before the closing guillemet: « Vraiment ? » strands the » at the head of the next sentence. Profile::spaced_closers is that narrow repair, and it is deliberately per-language — " opens as often as it closes, so a general rule here would weld He left. "Come," she said. into one sentence.

Extra terminators. Greek asks questions with ; — an ordinary ASCII semicolon, which UAX #29 quite correctly does not treat as a sentence ending. Profile::extra_terminators adds it back for Greek only. (Its · is the Greek semicolon and rightly keeps not terminating.)

A language with no profile falls back to plain UAX #29. That is a real fallback rather than a stub — it mis-splits only at title + Name, and every other rule above still applies.

Hebrew needs no abbreviation list at all, and its empty profile is deliberate: Hebrew abbreviations end in geresh (׳) or gershayim (״), not a full stop, so UAX #29 never splits them in the first place.

Structs§

Sentence
One sentence of a text: the slice itself, and where it sits as char offsets.

Functions§

sentence_bounds
The sentence of text containing char_offset, as block-relative char offsets, with trailing whitespace trimmed off the end so a highlight stops at the terminator rather than trailing into the gap before the next sentence.
sentences
Every sentence of text, in order, with the same boundary rules sentence_bounds applies — trailing whitespace trimmed, whitespace-only segments dropped.