1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
//! **The format string** — which `{…}` and `%…` of a literal is what.
//!
//! ```cpp
//! std::println("值 = {:.3f},宽度 {:{}}", x, w); the field, its conversion, the dynamic width
//! printf("%-8.2f|%*d|%1$s|%%", y, w, s); the conversion, two argument references, the escape
//! scanf("%3d %[a-z] %n", &n, name, &used); the scanset, and `%n` which reads nothing
//! ```
//!
//! # Why this is a scanner and not a parse
//!
//! A format string is a **language inside a string**, written in the syntax of whatever function receives it —
//! `std::format`'s is Python's, `printf`'s is the C standard's, `scanf`'s is the same with no flags and a scanset.
//! The C++ parser has no business reading it: the literal is one token to the grammar, and turning it into fields
//! would be a second grammar threaded through the first one. So this module is a **pure function over a `&str`**
//! with no session, no index and no LSP in it, and everything it knows about a format string is in this file.
//!
//! Two things ask it: the **colour** (`crates/cpp_ls`'s `semantic_token` pass), and later the *check* — "this format
//! string wants three arguments and the call passes two" — which is the same walk over the same pieces. That is why
//! the pieces are a data structure rather than a list of ranges to paint.
//!
//! # The rule that shapes the output: ascending, non-overlapping, merged
//!
//! The protocol this feeds requires one token per range with **no two ranges overlapping**, and the builder that
//! encodes it keeps the first claim on an offset. A `{…}` containing a `{…}` (the dynamic width `{:{}}`) would
//! naturally produce a piece *inside* a piece, and a `%-8.2f` would naturally be "one specifier, whose last
//! character is a conversion" — both of which overlap. So the walk here emits **runs of one kind**, into a
//! container that has no children:
//!
//! ```text
//! {:{}} Specifier "{:" DynamicWidth "{}" Specifier "}"
//! %-8.2f Specifier "%-8.2" Conversion "f"
//! {} Specifier "{}" (the two merge: one kind, contiguous)
//! ```
//!
//! The colour a reader sees is the same — the field's own syntax is one colour and its conversion
//! another — and every client can draw it, including the ones that do not say they can draw
//! overlapping tokens.
//!
//! # What it must never do
//!
//! A malformed format string is *legal C++*: `printf("%")` compiles with a warning, `std::format("{")` throws at
//! runtime, and a file being typed has both for as long as the keystroke lasts. So this walk **never panics, never
//! runs past the literal, and never reports a piece it is not sure of**: an unclosed `{`, a `%` at the end, a `%[`
//! with no `]` all stop at the literal's last byte and keep what was decided before that point.
//!
//! # The files, and why there are four
//!
//! ```text
//! mod.rs the vocabulary — what a piece is, what a literal is, which functions are read
//! scanner.rs the three grammars, scanned into runs (a pure walk over one literal's text)
//! locator.rs which literal of a file is a format string, and in which grammar (the function table)
//! describe.rs what a piece means, for the popup
//! complete.rs what may be typed at a position, for the completion list
//! ```
//!
//! The split is the feature's three consumers — colour, popup, completion — meeting one scanner and one locator.
//! They were one file while there were two consumers; the third is what made it worth saying which part answers
//! which question.
//!
use SourceRange;
/// **Which language a format string is written in** — decided by the function that receives it, not by its text.
///
/// The three are not dialects of one thing: `std::format`'s grammar is Python's (`{:>8.2f}`, `{{`), the C library's
/// two share a shape (`%-8.2f`) and differ in what a conversion means. A string is read as exactly one of them, and
/// a caller that cannot tell which does not read it at all — that is the whole reason [`FormatKind`] is an argument
/// rather than something the scanner guesses.
/// **What one run of bytes inside a literal is.** The vocabulary the colour maps to token types.
/// One run of a literal: what it is, and **where it is inside the literal token** (not in the file).
/// **Which functions' literals to read, and with what names** — the project's own answer, not the scanner's.
///
/// The scanner does not know this type exists; the locator ([`literals_in`]) does. Defaults are **on**: a project
/// that says nothing gets the feature, and `false` is how it is turned off. See `[format]` in `.cppls.toml` — the
/// file's shape and the reasons for it are in `crate::project::config::FormatSection`.
/// One of a project's own format functions — see [`FormatConfig::functions`].
/// **One literal that is a format string**, with the pieces already read.
pub use ;
pub use describe;
pub use ;
pub use pieces_in;