1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
//! THE SEGMENTER: a text in, a sequence of script runs out — UAX #24's resolution, stated once.
use FusedIterator;
use WritingSystem;
/// **CUT `text` INTO SCRIPT RUNS** — the one implementation both sides of a fan call.
///
/// ```
/// use ingraph::{WritingSystem, script_runs};
///
/// let runs: Vec<_> = script_runs("Rust 2024 年鉴").collect();
///
/// assert_eq!(runs.len(), 2);
/// assert_eq!(runs[0].script(), &WritingSystem::Latn);
/// assert_eq!(runs[0].text(), "Rust 2024 ");
/// assert_eq!(runs[1].script(), &WritingSystem::Hani);
/// assert_eq!(runs[1].text(), "年鉴");
/// ```
///
/// # THE RESOLUTION, in the three sentences it takes
///
/// 1. **A contextual character continues the run it is in.** Spaces, digits, punctuation and
/// combining marks belong to every script and to none
/// ([`WritingSystem::is_contextual`]), so they extend whatever preceded them rather than cutting it.
/// The space and the digits in the example above are Latin because the word before them was.
/// 2. **At the head, a contextual character joins what FOLLOWS.** There is nothing before it to
/// continue, so the leading run of them is attached to the first concrete script that appears —
/// `" 年鉴"` is one Han run and not a fallback run plus a Han one.
/// 3. **A text with no concrete script at all is one run in the fallback bucket.** `"123 !!!"` is
/// entirely contextual, so there is nothing for rules 1 and 2 to attach it to, and it lands in
/// [`WritingSystem::Zzzz`] beside every character the table does not name.
///
/// **PRECEDING WINS OVER FOLLOWING**, and rule 2 is the exception rather than a second rule. It is
/// UAX #24's own order and it is the one that matches how text is written: a trailing space, comma
/// or full stop is part of the sentence it ends, and a script change begins at the character that
/// changed it. The alternative — attaching to the following — would put the space in `"日本 word"`
/// into the Latin run, which is a boundary no writer of either script would draw.
///
/// # WHAT IT PROMISES, exactly
///
/// - **Total and non-overlapping**: the runs are contiguous, in order, and cover every byte of
/// `text` exactly once. Concatenating their texts reproduces the input.
/// - **Non-empty**: no run is ever zero bytes, so an empty input yields no runs at all rather than
/// one empty one.
/// - **Deterministic**: the same text always cuts the same way. Nothing here reads a locale, a
/// clock or an environment.
/// - **Zero-allocation**: it borrows `text` and yields slices of it, which is what lets a write path
/// run it per delivery.
///
/// # WHAT IT DOES NOT DO
///
/// It does not tokenise, lowercase, normalise or fold anything, and it has no opinion about
/// language — see the `script` module's note for where each of those lives. It also does not merge
/// runs of different scripts that a deployment considers one lane: a Japanese sentence cuts into Han,
/// Hiragana and Katakana runs, and putting them back together is a router's policy over an answer
/// this function gives faithfully.
/// ONE RUN — a maximal stretch of `text` written in one script.
///
/// # It carries the SLICE and the OFFSET, and both are spent
///
/// The slice is what a fan feeds to the analyzer that holds this script, and the offset is where it
/// came from — which is what lets a hit found in a per-script index be pointed back at a position in
/// the transcript that produced it. Neither can be derived from the other without the original text,
/// so both are here.
///
/// # There is no `with_` / `set_` pair, and the absence is the invariant
///
/// The house four-piece is for a value a holder may narrow after construction. These three seats are
/// ONE FACT — this stretch, at this place, in this script — and a run whose text was replaced
/// without its offset would be a lie the type had helped tell. A caller that wants a different run
/// builds one.
/// The runs of one text, cut lazily — [`script_runs`]' own iterator.
///
/// Lazy rather than collected, because a write path runs this per delivery and a query path runs it
/// per question: the runs are slices of the text already in hand, so there is nothing to allocate
/// unless a caller asks for a `Vec` by collecting one.
/// Once the cursor has reached the end it stays there, so a spent iterator keeps answering [`None`].