Skip to main content

oneyaml_syntax/parser/
mod.rs

1//! The YAML 1.2.2 parser: source text to the lossless event stream.
2//!
3//! # Architecture
4//!
5//! * `scan` — the lexical layer: character classes (§5) and single-line
6//!   scanners for indicators, properties, and plain scalar lines.
7//! * `scalar` — multi-line flow scalars (plain, single- and double-quoted)
8//!   with folding and escapes.
9//! * `stream` — stream and document level: BOMs, directives, document
10//!   markers (§9).
11//! * `block`, `head` — block collections and node heads (§8.2, §6.9).
12//! * `block_scalar` — literal and folded block scalars (§8.1).
13//! * `flow`, `flow_node`, `flow_block` — flow collections (§7.4), parsed
14//!   by a sub-machine with its own explicit stack into a local buffer so a
15//!   single-line flow collection can still become a block implicit key.
16//!
17//! The parser is an **explicit-stack state machine**: `State` says
18//! what the machine expects next and `Frame`s record the open documents and
19//! block collections. Nothing recurses, so arbitrarily deep input cannot
20//! overflow the call stack.
21//!
22//! **Placement (event contract C4).** Trivia (white space, comments, line
23//! breaks) is scanned into a *pending* buffer. Zero-width closing events
24//! (collection ends, implicit document ends, empty nodes) are pushed before
25//! the buffer is flushed, so they "end early"; zero-width openings are pushed
26//! after the flush, immediately before their first child, so they "start
27//! late".
28//!
29//! **Errors (C6).** A syntax error emits an [`EventKind::Error`] covering the
30//! rest of the offending line, completes the open structure with zero-width
31//! events (R2), and then skips whole lines as [`TriviaKind::Skipped`] until
32//! the next document marker, where parsing resumes (R4).
33//!
34//! # Streaming and buffering
35//!
36//! [`events_with`] is the parser; [`parse_with`] only collects it.
37//! What the parser holds between two pulls, for a consumer that drops
38//! each event (the bound is derived here; `tests/streaming.rs` checks it
39//! with `Events::retained_capacity`, hidden test instrumentation):
40//!
41//! * the flow hold (`flow.rs`): one line and at most the implicit-key
42//!   window of productions 154-155, 1024 characters or `W` = 4096 bytes,
43//!   whatever the collection's size. A byte yields at most 3 held items:
44//!   an entry of `k` bytes plus its `,` yields at most `k + 1` items that
45//!   cover bytes and 4 zero-width ones (a flow pair's start and end, an
46//!   empty key and value), and `(k + 5) / (k + 1) <= 3`, reached by
47//!   `[:,:,…]` and `[?,?,…]`. So the hold holds at most `3W` = 12,288
48//!   items, and its open pair reservations at most one per entry, `W / 2`;
49//! * the output queue: a batch of 64 items plus the zero-width ends one
50//!   token confirms (bounded by depth), **or**, once, the whole flow hold
51//!   when the root is decided (or an error ends it) and the hold drains
52//!   into it: at most `3W` items plus the batch and the depth-bounded
53//!   ends. Both buffers keep their capacity, so for a single-line flow
54//!   collection longer than the window both reach 12,288 items, which
55//!   `std`'s power-of-two `VecDeque` rounds to a capacity of 16,384 each;
56//! * frame stacks, `deferred` ends and empty nodes, `flow_repair`, and the
57//!   head buffer: bounded by depth;
58//! * trivia runs cost one queue item each and are lexed as they are
59//!   pulled, including the white lines before a block scalar error;
60//!   scalar values are decoded only by `Sourced::value`;
61//! * **not** bounded by depth or the key window: the `%TAG` handles of the
62//!   current and the next document (one entry per directive).
63//!
64//! So, `%TAG` handles aside, the parser holds `O(depth)` plus at most
65//! `2 × 3W + W / 2` items, independent of the input's length; with
66//! amortised growth (capacity below twice the peak length) that is below
67//! `4 × 3W + W` = 53,248 slots plus `O(depth)`. Measured peak: ≈32,784.
68//!
69//! # Prefix timing conventions (not contract requirements)
70//!
71//! The event contract fixes *where* each event goes (C1-C6), not *when*,
72//! relative to a later error, the parser commits to it. yaml-test-suite
73//! records, for every invalid input, exactly the events its reference
74//! parser emitted before failing, and our harness compares that prefix
75//! byte for byte. The rules below reproduce that timing. They are
76//! conventions for matching the suite, justified by R5 ("the events before
77//! the first `Error` are what the parser was sure of"); they are **not**
78//! part of the contract, `check_contract` does not test them, and another
79//! choice would be equally valid structurally. Code comments cite them as
80//! T1-T5.
81//!
82//! * **T1: deferred implicit document start.** An implicit document's
83//!   `DocumentStart` is emitted when its root node's content token is
84//!   recognised, not when its properties are, so an error in the first
85//!   head (e.g. `&a - b`) yields no document (`doc_start_deferred`).
86//! * **T2: deferred ends and empty nodes.** A block collection end caused
87//!   by a strict dedent, and an empty node with properties, are emitted at
88//!   once. A sequence end caused by a same-column non-`-` line, and an
89//!   empty node without properties, are deferred until the next token is
90//!   accepted; if it is not, they follow the `Error` as its repair
91//!   (`deferred`).
92//! * **T3: key errors.** A node that turns out to be a multi-line implicit
93//!   key never gets its completing event: a multi-line scalar key is
94//!   rejected before it is emitted, and a multi-line flow collection key
95//!   is rejected at its closing bracket (its contents are emitted, its end
96//!   is not). A complete single-line quoted scalar or flow collection
97//!   followed by a `:` that cannot start a mapping is emitted first, and
98//!   the `:` is reported after it. A multi-line flow collection in a block
99//!   key slot that no `:` follows (`'k' : ---\n[\n  'x' : value]`) is
100//!   emitted whole; the missing `:` (`Y0018`) is reported where the scan
101//!   stopped, which may be lines later or the end of input, so its
102//!   diagnostic carries a secondary label covering the key.
103//! * **T4: the document stays open at content after the root.** A document
104//!   ends only at its end marker, at the next document start, or at the end
105//!   of the stream, so content after a complete root (whatever its style) is
106//!   reported before the implicit `DocumentEnd`, which comes with the repair.
107//! * **T5: flow prefixes.** A flow collection with an error commits every
108//!   event before the error, then the repairs.
109//!
110//! # Settings and limits
111//!
112//! [`parse_with`] takes the [`Settings`]. The parser enforces four of the
113//! [`Limits`](oneyaml_core::Limits); each is checked **before** the
114//! offending event is emitted, so the stream stays contract-valid and the
115//! error is repaired like any other (C6):
116//!
117//! * `max_depth` (`Y2000`): block and flow collections counted together.
118//!   Checked when a collection opens, where a block collection that may
119//!   not start there would be reported; for a flow collection that turns
120//!   out to be an implicit key (whose depth grows by the mapping or flow
121//!   pair it starts), when the key's `:` is seen.
122//! * `max_scalar_len` (`Y2001`): a scalar's value in bytes, checked before
123//!   the scalar is emitted.
124//! * `max_document_size` (`Y2002`): the input length in bytes, checked
125//!   first; the whole input is then skipped.
126//! * `max_documents` (`Y2003`): checked when a document would start; the
127//!   rest of the input is then skipped.
128//!
129//! The help text of each limit diagnostic names the setting and its current
130//! value. The one leniency, `lenient_inline_colons`, is
131//! described in `head.rs`.
132
133mod block;
134mod block_scalar;
135mod flow;
136mod flow_block;
137mod flow_node;
138mod head;
139mod polish;
140mod read;
141mod run;
142mod scalar;
143mod scan;
144mod stream;
145
146/// How far past an implicit key's start the parser looks for its `:`
147/// (bounds how far a validation window reads past a peeked
148/// element's start).
149pub(crate) use flow::KEY_WINDOW;
150
151use std::collections::{BTreeMap, VecDeque};
152
153use oneyaml_core::code::{
154    DEPTH_LIMIT_EXCEEDED, DOCUMENT_TOO_LARGE, INTERNAL_ERROR, INVALID_COMMENT_CHARACTER,
155    INVALID_ESCAPE, MISPLACED_BYTE_ORDER_MARK, MISSING_MAPPING_COLON, SCALAR_TOO_LONG,
156    TAB_INDENTATION, TOO_MANY_DOCUMENTS, UNTERMINATED_FLOW_COLLECTION, UNTERMINATED_QUOTED_SCALAR,
157};
158use oneyaml_core::{Code, Diagnostic, Label, Settings, Span};
159
160use crate::event::{Event, EventKind, EventList, NodeInfo, Properties, Sourced, TriviaKind};
161
162pub(crate) use read::{
163    anchor_at, decoded_suffix, directive_at, directive_prefix, resolve_tag, scalar_value,
164    tag_source_at,
165};
166
167use scan::{break_len, byte, line_end, skip_white};
168
169/// Parses `source` into the lossless event stream with the default
170/// [`Settings`] (including [`Limits::default()`](oneyaml_core::Limits)).
171///
172/// Collects [`events`] into an [`EventList`]: the same parser, the same
173/// events. Never fails and never panics: problems are reported in-band as
174/// [`EventKind::Error`] and [`EventKind::Warning`] events, and the stream
175/// always satisfies the event contract (C1–C6). Runs in O(n) time with no
176/// recursion.
177#[must_use]
178pub fn parse(source: &str) -> EventList<'_> {
179    parse_with(source, &Settings::new())
180}
181
182/// Parses `source` into the lossless event stream under `settings`,
183/// collected into an [`EventList`] that keeps the source for reading (see
184/// [`events_with`]).
185///
186/// The parser reads the limits `max_depth`, `max_scalar_len`,
187/// `max_document_size`, and `max_documents`, the `lenient_inline_colons`
188/// leniency, and `honor_version_directive` (for the wording of the `%YAML
189/// 1.0` and `%YAML 1.1` warnings); every other setting belongs to a later layer. Same
190/// guarantees as [`parse`].
191#[must_use]
192pub fn parse_with<'a>(source: &'a str, settings: &Settings) -> EventList<'a> {
193    let events = events_with(source, settings)
194        .map(Sourced::into_event)
195        .collect();
196    EventList::new(source, events)
197}
198
199/// The lossless event stream of `source` under the default [`Settings`],
200/// produced incrementally (see [`Events`]).
201#[must_use]
202pub fn events(source: &str) -> Events<'_> {
203    events_with(source, &Settings::new())
204}
205
206/// The lossless event stream of `source` under `settings`, produced
207/// incrementally (see [`Events`]). The settings are read once; the
208/// iterator does not borrow them.
209#[must_use]
210pub fn events_with<'a>(source: &'a str, settings: &Settings) -> Events<'a> {
211    Events {
212        parser: Parser::new(source, settings),
213    }
214}
215
216/// A pull parser: an iterator over the lossless event stream of one source.
217///
218/// Each call to `next` runs the state machine only until the next event is
219/// known, so a consumer that drops events as it reads them (the serde
220/// layer) holds memory bounded by the nesting depth plus the grammar's
221/// lookahead, independent of the input's length. See the module
222/// docs for the bounds.
223///
224/// The stream is the one [`parse`] collects, and satisfies the event
225/// contract (C1–C6). Each event comes paired with the source as a
226/// [`Sourced`], so names and values are read from the right text by
227/// construction. The iterator is fused: after `StreamEnd` it returns
228/// `None` forever.
229#[derive(Debug)]
230pub struct Events<'a> {
231    parser: Parser<'a>,
232}
233
234impl<'a> Iterator for Events<'a> {
235    type Item = Sourced<'a>;
236
237    fn next(&mut self) -> Option<Sourced<'a>> {
238        let p = &mut self.parser;
239        if let Some(event) = p.out.pop(p.src) {
240            return Some(event);
241        }
242        p.fill();
243        p.out.pop(p.src)
244    }
245}
246
247impl core::iter::FusedIterator for Events<'_> {}
248
249impl<'a> Events<'a> {
250    /// The source text being parsed.
251    #[must_use]
252    pub const fn source(&self) -> &'a str {
253        self.parser.src
254    }
255
256    /// Test instrumentation, not part of the API: the total size, in
257    /// elements, of what the parser holds right now: the capacity of the
258    /// output queue, the flow-collection hold, and the stacks and owed
259    /// events that grow with nesting depth, plus the number of `%TAG`
260    /// handles held for the current and the next document.
261    ///
262    /// For a consumer that drops each event, everything but the `%TAG`
263    /// handles is bounded by the nesting depth plus the implicit-key
264    /// lookahead (see *Streaming and buffering* in the module docs), never
265    /// by the input's length. The handles grow with the number of `%TAG`
266    /// directives of one document, which the input chooses; they are
267    /// counted so that tests see them, and the bound is stated without
268    /// them. Hidden from the docs: `tests/streaming.rs` checks the bound
269    /// through it.
270    #[doc(hidden)]
271    #[must_use]
272    pub fn retained_capacity(&self) -> usize {
273        let p = &self.parser;
274        p.out
275            .q
276            .capacity()
277            .saturating_add(p.deferred.capacity())
278            .saturating_add(p.stack.capacity())
279            .saturating_add(p.flow_repair.capacity())
280            .saturating_add(p.scratch.capacity())
281            .saturating_add(p.flow.capacity())
282            .saturating_add(p.tags.len())
283            .saturating_add(p.pending_tags.len())
284    }
285}
286
287/// How many items the parser queues ahead before handing events out: a
288/// constant, so the queue stays bounded, and large enough that the
289/// state machine runs in batches rather than once per event.
290const BATCH: usize = 64;
291
292/// An entry of the output queue: an event, or a run of trivia (white
293/// space, line breaks, comments, byte order marks) that is lexed into
294/// events only as the consumer pulls them, so any amount of trivia costs
295/// O(1) memory.
296#[derive(Clone, Debug)]
297pub(super) enum Item<'a> {
298    Event(Event<'a>),
299    /// Trivia covering `start..end`.
300    Trivia(usize, usize),
301}
302
303/// The output queue, in stream order.
304#[derive(Debug, Default)]
305pub(super) struct Out<'a> {
306    q: VecDeque<Item<'a>>,
307    /// What is left of the trivia run being lexed (empty when `.0 >= .1`);
308    /// it comes before everything in `q`. Held here, not pushed back on the
309    /// queue, so a run of several pieces (a line break and the next line's
310    /// indentation, a comment) costs no queue traffic between them.
311    run: (usize, usize),
312}
313
314impl<'a> Out<'a> {
315    pub(super) fn event(&mut self, event: Event<'a>) {
316        self.q.push_back(Item::Event(event));
317    }
318
319    /// Queues the trivia covering `start..end` (nothing when empty).
320    pub(super) fn trivia(&mut self, start: usize, end: usize) {
321        if start < end {
322            self.q.push_back(Item::Trivia(start, end));
323        }
324    }
325
326    pub(super) fn item(&mut self, item: Item<'a>) {
327        match item {
328            Item::Event(event) => self.event(event),
329            Item::Trivia(start, end) => self.trivia(start, end),
330        }
331    }
332
333    /// Enough is queued to hand out (see [`BATCH`]).
334    fn is_full(&self) -> bool {
335        self.q.len() >= BATCH
336    }
337
338    /// The next event, lexing the front trivia run if needed.
339    #[inline]
340    fn pop(&mut self, src: &'a str) -> Option<Sourced<'a>> {
341        // The `Sourced` is built at each return site, so the item is
342        // written once, straight into the caller's slot.
343        let (from, to) = if self.run.0 < self.run.1 {
344            self.run
345        } else {
346            match self.q.pop_front()? {
347                Item::Event(event) => return Some(Sourced::new(event, src)),
348                Item::Trivia(from, to) => (from, to),
349            }
350        };
351        match scan::lex_trivia(src, from, to) {
352            Some((kind, stop)) => {
353                self.run = (stop, to);
354                let span = Span::new(from, stop);
355                Some(Sourced::new(Event::new(EventKind::Trivia(kind), span), src))
356            }
357            None => {
358                self.run = (to, to);
359                Some(Sourced::new(broken_trivia_run(from, to), src))
360            }
361        }
362    }
363}
364
365/// Only trivia is ever queued as a run. In release the rest of the
366/// run becomes an in-band Y9000 error, never a silently mislabelled trivia
367/// event.
368#[cold]
369#[inline(never)]
370fn broken_trivia_run<'a>(from: usize, to: usize) -> Event<'a> {
371    debug_assert!(false, "non-trivia byte in a trivia run at {from}");
372    let span = Span::new(from, to);
373    Event::new(
374        EventKind::Error(internal("non-trivia bytes in a trivia run", span)),
375        span,
376    )
377}
378
379/// Where a block mapping is within its current entry.
380#[derive(Clone, Copy, Debug, PartialEq, Eq)]
381enum MapPhase {
382    /// Expecting a key or the end of the mapping: `ns-l-block-map-entry`
383    /// (production 188) via `?` (190), `:` after an empty key, or an
384    /// implicit key (192-194).
385    Key,
386    /// The key after `?` is being parsed (`c-l-block-map-explicit-key`,
387    /// production 190).
388    ExplicitKey,
389    /// An explicit key is complete; expecting `l-block-map-explicit-value`
390    /// (production 191) or the next entry.
391    AfterExplicitKey,
392    /// The value after `:` is being parsed (productions 191 and 194).
393    Value,
394}
395
396/// An open document or block collection.
397#[derive(Clone, Copy, Debug)]
398enum Frame {
399    /// `l-any-document` (production 210); the root node is complete.
400    Document { root_done: bool },
401    /// `l+block-sequence(n)` (production 183) with entries at `indent`.
402    Seq { indent: usize },
403    /// `l+block-mapping(n)` (production 187) with keys at `indent`.
404    Map { indent: usize, phase: MapPhase },
405}
406
407/// Context for parsing one node (`s-l+block-node(n,c)`, production 196).
408#[derive(Clone, Copy, Debug)]
409struct NodeCtx {
410    /// Indentation `n` of the parent collection; `None` is the document
411    /// level (indentation -1).
412    parent: Option<usize>,
413    /// A compact block collection may start on the current line
414    /// (`s-l+block-indented`, production 185: after `-`, `?`, or an explicit
415    /// `:`).
416    compact: bool,
417    /// A block sequence may sit at the parent's own indentation
418    /// (`seq-space(n, block-out)`, production 201).
419    seq_at_parent: bool,
420}
421
422/// What the machine does next.
423#[derive(Clone, Copy, Debug)]
424enum State {
425    /// Between documents (`l-yaml-stream`, production 211).
426    Stream,
427    /// Expecting a node in the context given.
428    Node(NodeCtx),
429    /// A node or indicator is done; the innermost frame decides: another
430    /// entry (productions 183-184, 187-194), the collection's end, or the
431    /// document's end (`l-document-suffix`, production 205).
432    Next,
433    /// Inside a flow collection opened by a block node head (§7.4); the
434    /// flow sub-machine (`flow.rs`) holds its own explicit stack.
435    Flow,
436    /// Resynchronising after an error (C6 R4): whole lines are skipped up
437    /// to the next `c-directives-end`/`c-document-end` (203-204).
438    Skip,
439    /// `StreamEnd` emitted.
440    Done,
441}
442
443#[derive(Debug)]
444struct Parser<'a> {
445    src: &'a str,
446    /// Events ready for the consumer (bounded by one step's output).
447    out: Out<'a>,
448    /// End of the last emitted event.
449    emit_pos: usize,
450    /// Zero-width events owed at `emit_pos` (block collection ends and
451    /// empty nodes), emitted lazily: before the pending trivia once the next
452    /// token is accepted, or after the `Error` if it is not. Timing
453    /// convention T2 (module docs), not a contract rule. Bounded by depth.
454    deferred: Vec<EventKind>,
455    /// Scan position. The bytes `emit_pos..pos` are trivia scanned but not
456    /// yet emitted ("pending"); they are queued as one lazily lexed run.
457    pos: usize,
458    /// Start of the current line.
459    line_start: usize,
460    /// Leading spaces of the current line.
461    line_indent: usize,
462    /// No content has been consumed on the current line yet.
463    fresh: bool,
464    /// A tab occurred in white space since the last line break (or since the
465    /// last consumed token on this line).
466    tab_seen: bool,
467    stack: Vec<Frame>,
468    state: State,
469    /// Properties emitted as trivia but not yet attached to a node.
470    props: Properties,
471    /// An implicit document was entered but its `DocumentStart` is deferred
472    /// until its first node event (so an error in the first node's head
473    /// yields no document at all). Timing convention T1 (module docs).
474    doc_start_deferred: bool,
475    /// Directives seen since the last document.
476    directives_pending: bool,
477    /// A `%YAML` directive was seen for the next document.
478    yaml_seen: bool,
479    /// `%TAG` handles declared for the next document, with their prefixes
480    /// (read once).
481    pending_tags: BTreeMap<&'a str, scan::TagPrefix>,
482    /// `%TAG` handles of the current document.
483    tags: BTreeMap<&'a str, scan::TagPrefix>,
484    /// A `...` was just consumed; the rest of its line must be trivia.
485    doc_end_line: bool,
486    /// A document was ended by a byte order mark (`prefix_mark_at`), not
487    /// by `...`: until the next `---`, only a document prefix may follow
488    /// (production 211's second branch).
489    /// Where that mark starts (for the diagnostic).
490    prefix_without_end: Option<usize>,
491    /// A byte order mark that caused the error about to be reported
492    /// (`mark_then_fail`): its `MISPLACED_BYTE_ORDER_MARK` is reported with
493    /// the error, in the order of their labels. Consumed
494    /// by `fail_with`.
495    mark_before_error: Option<usize>,
496    /// Where the first character that cannot belong to any valid document
497    /// is, when that is later than the scan position the failing construct
498    /// started at: the error is reported there (consumed by `fail_with`).
499    /// A cell, so that scanners holding only `&self` can name it.
500    blame: core::cell::Cell<Option<usize>>,
501    /// Zero-width events completing an erroneous flow collection, emitted
502    /// right after its `Error` (C6 R2).
503    flow_repair: Vec<Owed>,
504    /// Reused buffer for a node head's events (no allocation per
505    /// node on the hot path). At most a few events: one line's properties.
506    scratch: Vec<Event<'a>>,
507    /// The open flow collection, if any (`State::Flow`).
508    flow: flow::FlowState<'a>,
509    /// `Limits::max_depth`.
510    max_depth: usize,
511    /// `Limits::max_scalar_len`.
512    max_scalar_len: usize,
513    /// `Limits::max_document_size`.
514    max_document_size: usize,
515    /// `Limits::max_documents`.
516    max_documents: usize,
517    /// `Settings::lenient_inline_colons`.
518    lenient_inline_colons: bool,
519    /// `Settings::honor_version_directive`, for the `%YAML 1.0`
520    /// and `%YAML 1.1` warnings' wording only.
521    honor_version_directive: bool,
522    /// Documents started so far.
523    documents: usize,
524    /// A stream-wide limit was exceeded: skip to the end of the input, not
525    /// just to the next document marker.
526    halted: bool,
527    /// Start and end of the last quoted scalar emitted as a block node, so
528    /// content glued to its closing quote (`'it's'`) gets a targeted
529    /// diagnostic.
530    last_quoted: (usize, usize),
531}
532
533/// A zero-width event owed after an error (C6 R2), packed once its position
534/// is known.
535#[derive(Clone, Debug)]
536pub(super) enum Owed {
537    /// A collection end.
538    Kind(EventKind),
539    /// An empty plain scalar with these properties.
540    Empty(Properties),
541}
542
543/// `kind` with its node information set (for a node start or scalar).
544pub(super) fn with_info(mut kind: EventKind, new: NodeInfo) -> EventKind {
545    if let EventKind::Scalar { info, .. }
546    | EventKind::MappingStart { info, .. }
547    | EventKind::SequenceStart { info, .. } = &mut kind
548    {
549        *info = new;
550    }
551    kind
552}
553
554/// The `Y9000` diagnostic for a broken internal invariant: the release-build
555/// outcome of a `debug_assert!`.
556pub(super) fn internal(what: &str, span: Span) -> Box<Diagnostic> {
557    Box::new(
558        Diagnostic::error(
559            INTERNAL_ERROR,
560            format!("internal error: {what}"),
561            Label::new(span, ""),
562        )
563        .with_help("this is a bug in oneyaml; please report it with this input"),
564    )
565}
566
567/// The help every diagnostic with `code` gets unless its producer
568/// refines it (the diagnostic polish list: what to write instead).
569fn default_help(code: Code) -> Option<&'static str> {
570    Some(match code {
571        TAB_INDENTATION => {
572            "YAML indents with spaces only; replace the tabs used for indentation here with spaces"
573        }
574        MISSING_MAPPING_COLON => {
575            "each entry of a block mapping is written `key: value`, with a space after the `:`"
576        }
577        UNTERMINATED_QUOTED_SCALAR => {
578            "add the closing quote; inside '...' write '' for a literal ', inside \"...\" write \\\" for a literal \""
579        }
580        INVALID_ESCAPE => {
581            "in double quotes, `\\` starts an escape such as `\\n`, `\\t`, `\\\"` or `\\\\`; write `\\\\` for a literal backslash, or use single quotes, where `\\` has no special meaning"
582        }
583        UNTERMINATED_FLOW_COLLECTION => "close every `[` with `]` and every `{` with `}`",
584        _ => return None,
585    })
586}
587
588/// The `MISPLACED_BYTE_ORDER_MARK` diagnostic for the mark at `start..end`
589/// (§5.2, production 202).
590pub(super) fn misplaced_bom_diagnostic(start: usize, end: usize) -> Box<Diagnostic> {
591    Box::new(
592        Diagnostic::error(
593            MISPLACED_BYTE_ORDER_MARK,
594            "a byte order mark may appear only at the start of a document prefix",
595            Label::new(Span::new(start, end), ""),
596        )
597        .with_help(
598            "remove this byte order mark (U+FEFF); a mark is allowed only at the very start of the stream, or at the start of a line before a document",
599        ),
600    )
601}
602
603/// Whether `bytes` may hold a character that is not printable: false only
604/// for bytes that are all ASCII, none a control character (a tab counts as
605/// one here) or DEL. Eight bytes at a time.
606#[inline(never)]
607fn may_hold_unprintable(bytes: &[u8]) -> bool {
608    const ONES: u64 = 0x0101_0101_0101_0101;
609    const HIGH: u64 = 0x8080_8080_8080_8080;
610    let mut words = bytes.chunks_exact(8);
611    for chunk in words.by_ref() {
612        let Ok(chunk) = <[u8; 8]>::try_from(chunk) else {
613            return true;
614        };
615        let w = u64::from_le_bytes(chunk);
616        // A byte with the high bit set, below 0x20, or equal to 0x7f.
617        let del = w ^ (ONES * 0x7f);
618        let found = w | (w.wrapping_sub(ONES * 0x20) & !w) | (del.wrapping_sub(ONES) & !del);
619        if found & HIGH != 0 {
620            return true;
621        }
622    }
623    words.remainder().iter().any(|b| *b < 0x20 || *b >= 0x7f)
624}
625
626/// The error for the first character of the comment `start..end` that is
627/// not printable, if any.
628#[inline(never)]
629pub(super) fn comment_char_diagnostic(
630    src: &str,
631    start: usize,
632    end: usize,
633) -> Option<Box<Diagnostic>> {
634    let text = src.get(start..end)?;
635    // Nearly every comment is plain printable ASCII: a cheap pass over the
636    // bytes rules that out before the slower pass over characters.
637    if !may_hold_unprintable(text.as_bytes()) {
638        return None;
639    }
640    let (k, c) = text.char_indices().find(|(_, c)| !scan::is_printable(*c))?;
641    let at = start.saturating_add(k);
642    Some(Box::new(
643        Diagnostic::error(
644            INVALID_COMMENT_CHARACTER,
645            "a comment may hold only printable characters",
646            Label::new(Span::new(at, at.saturating_add(c.len_utf8())), ""),
647        )
648        .with_help("remove this character; control characters (U+0000 to U+001F except tab, U+007F, and U+0080 to U+009F except U+0085) cannot appear in a comment"),
649    ))
650}
651
652fn diagnostic(code: Code, message: &str, span: Span) -> Box<Diagnostic> {
653    Box::new(Diagnostic::error(code, message, Label::new(span, "")))
654}
655
656impl<'a> Parser<'a> {
657    fn new(src: &'a str, settings: &Settings) -> Self {
658        let limits = settings.get_limits();
659        let mut p = Self {
660            src,
661            out: Out::default(),
662            emit_pos: 0,
663            deferred: Vec::new(),
664            pos: 0,
665            line_start: 0,
666            line_indent: 0,
667            fresh: true,
668            tab_seen: false,
669            stack: Vec::new(),
670            state: State::Stream,
671            props: Properties::none(),
672            doc_start_deferred: false,
673            directives_pending: false,
674            yaml_seen: false,
675            pending_tags: BTreeMap::new(),
676            tags: BTreeMap::new(),
677            doc_end_line: false,
678            prefix_without_end: None,
679            mark_before_error: None,
680            blame: core::cell::Cell::new(None),
681            flow_repair: Vec::new(),
682            scratch: Vec::new(),
683            flow: flow::FlowState::default(),
684            max_depth: limits.get_max_depth(),
685            max_scalar_len: limits.get_max_scalar_len(),
686            max_document_size: limits.get_max_document_size(),
687            max_documents: limits.get_max_documents(),
688            lenient_inline_colons: settings.get_lenient_inline_colons(),
689            honor_version_directive: settings.get_honor_version_directive(),
690            documents: 0,
691            halted: false,
692            last_quoted: (0, 0),
693        };
694        p.push(EventKind::StreamStart, 0, 0);
695        if src.len() > p.max_document_size {
696            // Refuse oversized input before parsing any of it.
697            p.halt(DOCUMENT_TOO_LARGE, "the input is too large");
698        }
699        p
700    }
701
702    /// Runs the state machine until [`BATCH`] items are queued or the
703    /// stream is complete. Not inlined, so that `Events::next`, which
704    /// mostly just pops the queue, stays small (per-event overhead).
705    #[inline(never)]
706    fn fill(&mut self) {
707        // One loop over an explicit state; no recursion on input.
708        while !self.out.is_full() && !matches!(self.state, State::Done) {
709            self.step();
710        }
711    }
712
713    /// One transition of the state machine. Each queues a bounded number
714    /// of items: one token or line and the zero-width events it
715    /// confirms, which are bounded by the nesting depth.
716    ///
717    fn step(&mut self) {
718        match self.state {
719            State::Stream => self.stream(),
720            State::Node(ctx) => self.node(ctx),
721            State::Next => self.next(),
722            State::Flow => self.flow_step(),
723            State::Skip => self.skip(),
724            State::Done => {}
725        }
726    }
727
728    // ------------------------------------------------------------ emission
729
730    /// Emits an event covering `start..end` (`start` must be `emit_pos`).
731    fn push(&mut self, kind: EventKind, start: usize, end: usize) {
732        self.push_event(Event::new(kind, Span::new(start, end)));
733    }
734
735    /// Emits a built event (its start must be `emit_pos`).
736    fn push_event(&mut self, event: Event<'a>) {
737        self.emit_pos = event.span().end;
738        self.out.event(event);
739    }
740
741    /// Emits a zero-width event at `emit_pos` (before any pending trivia),
742    /// after any deferred ones.
743    fn zero(&mut self, kind: EventKind) {
744        self.materialize();
745        let at = self.emit_pos;
746        self.push(kind, at, at);
747    }
748
749    /// [`Parser::zero`] for a node event with properties (packed relative
750    /// to its position, `emit_pos`).
751    fn zero_node(&mut self, kind: EventKind, props: Properties) {
752        let info = NodeInfo::new(self.emit_pos, props, None);
753        self.zero(with_info(kind, info));
754    }
755
756    /// Owes a zero-width event at `emit_pos` (see `deferred`).
757    fn defer(&mut self, kind: EventKind) {
758        self.deferred.push(kind);
759    }
760
761    /// Emits the deferred zero-width events.
762    fn materialize(&mut self) {
763        if self.deferred.is_empty() {
764            return;
765        }
766        let at = self.emit_pos;
767        for kind in self.deferred.drain(..) {
768            self.out.event(Event::new(kind, Span::empty(at)));
769        }
770    }
771
772    /// Emits the deferred events, then the pending trivia.
773    fn flush(&mut self) {
774        self.materialize();
775        self.out.trivia(self.emit_pos, self.pos);
776        self.emit_pos = self.pos;
777    }
778
779    /// Flushes and, for a deferred implicit document, emits its start: call
780    /// before the first event of any node.
781    fn begin_content(&mut self) {
782        self.flush();
783        if self.doc_start_deferred {
784            self.doc_start_deferred = false;
785            self.zero(EventKind::document_start(false));
786        }
787    }
788
789    /// Emits an event at the scan position after flushing, and consumes it.
790    fn token(&mut self, kind: EventKind, end: usize) {
791        let start = self.pos;
792        self.push(kind, start, end);
793        self.pos = end;
794        self.fresh = false;
795        self.tab_seen = false;
796    }
797
798    /// The reusable head buffer, empty.
799    fn take_scratch(&mut self) -> Vec<Event<'a>> {
800        let mut events = core::mem::take(&mut self.scratch);
801        events.clear();
802        events
803    }
804
805    /// Commits a head buffer and keeps its capacity for the next head.
806    fn commit_scratch(&mut self, mut events: Vec<Event<'a>>) {
807        self.commit(events.drain(..));
808        self.scratch = events;
809    }
810
811    /// An empty plain scalar at `emit_pos` carrying the pending properties.
812    /// With properties the node is certain and is emitted at once; without,
813    /// it is deferred until the next token confirms it (timing convention
814    /// T2).
815    fn empty(&mut self) {
816        let props = core::mem::take(&mut self.props);
817        if props.is_empty() {
818            self.defer(EventKind::plain());
819        } else {
820            self.zero_node(EventKind::plain(), props);
821        }
822    }
823
824    fn col(&self) -> usize {
825        self.pos.saturating_sub(self.line_start)
826    }
827
828    // ------------------------------------------------------------ limits
829
830    /// The number of open block collections. The stack is a `Document`
831    /// frame followed by the block collections inside it.
832    fn block_depth(&self) -> usize {
833        self.stack.len().saturating_sub(1)
834    }
835
836    /// `Limits::max_depth` allows a collection at `depth`.
837    fn depth_ok(&self, depth: usize) -> bool {
838        depth <= self.max_depth
839    }
840
841    /// `Limits::max_scalar_len` allows a scalar value of `len` bytes.
842    fn scalar_len_ok(&self, len: usize) -> bool {
843        len <= self.max_scalar_len
844    }
845
846    /// The help of a limit diagnostic names the setting to raise and
847    /// its current value.
848    fn limit_help(&self, code: Code) -> Option<String> {
849        let (setting, value) = match code {
850            DEPTH_LIMIT_EXCEEDED => ("max_depth", self.max_depth),
851            SCALAR_TOO_LONG => ("max_scalar_len", self.max_scalar_len),
852            DOCUMENT_TOO_LARGE => ("max_document_size", self.max_document_size),
853            TOO_MANY_DOCUMENTS => ("max_documents", self.max_documents),
854            _ => return None,
855        };
856        Some(format!("raise Limits::{setting} (currently {value})"))
857    }
858
859    /// Reports a stream-wide limit and skips the rest of the input.
860    fn halt(&mut self, code: Code, message: &str) {
861        self.fail(code, message, false);
862        self.halted = true;
863    }
864
865    // ------------------------------------------------------------ trivia
866
867    /// `#` at `i` starts a comment (production 75).
868    fn comment_at(&self, i: usize) -> bool {
869        scan::comment_at(self.src, i)
870    }
871
872    /// Scans white space, comments, and line breaks as pending trivia
873    /// (`s-l-comments`, production 79, and `s-separate`, production 80).
874    ///
875    /// Byte order marks (§5.2, productions 202 and 211): `bom` says the
876    /// scan is between documents, where `l-document-prefix` may start.
877    /// There a mark at the very start of a line (nothing before it on the
878    /// line, no directives pending) is `ByteOrderMark` trivia and the
879    /// line's columns count after it. Inside a document such a mark
880    /// ([`Parser::prefix_mark_at`]) ends the document like `---` does
881    /// too: the scan stops before it, and the structure closes
882    /// around it. Anywhere else a mark is a recoverable
883    /// `MISPLACED_BYTE_ORDER_MARK` error, and the parse goes on
884    /// as if it were absent (`misplaced_bom`). A mark inside a comment
885    /// stays in the comment token and is reported after it
886    /// (`marks_in_token`).
887    fn scan_trivia(&mut self, bom: bool) {
888        loop {
889            let i = self.pos;
890            let kind_end = match byte(self.src, i) {
891                Some(b' ' | b'\t') => {
892                    // Leading spaces of the run, then (rarely) tabs.
893                    let spaces = self
894                        .src
895                        .as_bytes()
896                        .get(i..)
897                        .map_or(0, |r| r.iter().take_while(|b| **b == b' ').count());
898                    let mut end = i.saturating_add(spaces);
899                    if byte(self.src, end) == Some(b'\t') {
900                        self.tab_seen = true;
901                        end = skip_white(self.src, end);
902                    }
903                    // Leading spaces of the line: the run starts where the
904                    // line's spaces so far end, which is the line start
905                    // unless a skipped misplaced mark sits in between (its
906                    // bytes were moved out of the line by `misplaced_bom`).
907                    if i == self.line_start.saturating_add(self.line_indent) {
908                        self.line_indent = self.line_indent.saturating_add(spaces);
909                    }
910                    (TriviaKind::Whitespace, end)
911                }
912                Some(b'\n' | b'\r') => {
913                    let end = i.saturating_add(break_len(self.src, i));
914                    self.line_start = end;
915                    self.line_indent = 0;
916                    self.fresh = true;
917                    self.tab_seen = false;
918                    (TriviaKind::LineBreak, end)
919                }
920                Some(b'#') if self.comment_at(i) => {
921                    // c-nb-comment-text (production 75) is `nb-char`s,
922                    // which exclude the mark (production 27).
923                    let (end, clean) = scan::comment_extent(self.src, i);
924                    self.pos = end;
925                    // Printable ASCII holds no byte order mark and no
926                    // character to report.
927                    if !clean {
928                        let bad = comment_char_diagnostic(self.src, i, end);
929                        self.marks_in_token(i, end, bad);
930                    }
931                    continue;
932                }
933                _ if !bom && self.prefix_mark_at(i) => break,
934                _ if scan::mark_at(self.src, i) => {
935                    let end = i.saturating_add('\u{feff}'.len_utf8());
936                    // Any number of document prefixes may precede a document,
937                    // and each may start with a mark (productions 211, 202),
938                    // so a mark right behind another is a prefix start too.
939                    let prefix_start = bom
940                        && self.fresh
941                        && i == self.line_start
942                        && self.line_indent == 0
943                        && !self.directives_pending;
944                    if !prefix_start {
945                        self.misplaced_bom(i, end);
946                        continue;
947                    }
948                    // §5.2: the mark is no column; the line starts after it.
949                    self.line_start = end;
950                    self.line_indent = 0;
951                    (TriviaKind::ByteOrderMark, end)
952                }
953                _ => break,
954            };
955            // Pending trivia is queued later as one lazy run (`flush`).
956            let (_, end) = kind_end;
957            self.pos = end;
958        }
959    }
960
961    /// A byte order mark at `i` at the very start of a fresh line inside a
962    /// document: it can continue nothing (it is no `ns-char`), so it ends
963    /// the document implicitly and starts an `l-document-prefix`
964    /// (production 211, §9.1.1). What may follow the prefix is
965    /// checked in `stream` (`prefix_without_end`).
966    pub(super) fn prefix_mark_at(&self, i: usize) -> bool {
967        self.fresh
968            && i == self.line_start
969            && self.line_indent == 0
970            && !self.stack.is_empty()
971            && scan::mark_at(self.src, i)
972    }
973
974    /// Reports each byte order mark inside the token `start..end` (a
975    /// comment, a block scalar), which keeps its bytes: one
976    /// zero-width `MISPLACED_BYTE_ORDER_MARK` error per mark right after
977    /// the token, whose label covers the mark. The pending
978    /// trivia up to the scan position is flushed first, so the error
979    /// directly follows the token and nothing lies between them.
980    pub(super) fn marks_in_token(
981        &mut self,
982        start: usize,
983        end: usize,
984        bad: Option<Box<Diagnostic>>,
985    ) {
986        let text = self.src.get(start..end).unwrap_or_default();
987        if bad.is_none() && !text.contains('\u{feff}') {
988            return;
989        }
990        self.flush();
991        // The errors keep the order of the characters they report.
992        let mut bad = bad;
993        for (k, _) in text.match_indices('\u{feff}') {
994            let at = start.saturating_add(k);
995            if bad.as_ref().is_some_and(|d| d.primary.span.start < at) {
996                self.comment_error(bad.take());
997            }
998            let diag = misplaced_bom_diagnostic(at, at.saturating_add('\u{feff}'.len_utf8()));
999            let here = self.emit_pos;
1000            self.push(EventKind::Error(diag), here, here);
1001        }
1002        self.comment_error(bad);
1003    }
1004
1005    /// Pushes the error for a character that cannot be in a comment, if
1006    /// there is one, right after the comment (production 1; comment text
1007    /// is `nb-char`s, productions 75 and 27): one zero-width
1008    /// `INVALID_COMMENT_CHARACTER` error whose label covers the character.
1009    /// The comment itself is kept as written, and the parse goes on.
1010    fn comment_error(&mut self, bad: Option<Box<Diagnostic>>) {
1011        if let Some(diag) = bad {
1012            let here = self.emit_pos;
1013            self.push(EventKind::Error(diag), here, here);
1014        }
1015    }
1016
1017    /// A byte order mark at `start..end` where no document prefix may
1018    /// start (§5.2, production 202): a recoverable
1019    /// `MISPLACED_BYTE_ORDER_MARK` error covering the mark, after the
1020    /// pending trivia (deferred zero-width events go first, at their own
1021    /// position). The parse then continues as if the mark were absent
1022    /// between tokens: the line start moves by the mark's length, so
1023    /// columns and the line's leading spaces are counted without it, and
1024    /// `fresh` is unchanged. (A mark right after a plain scalar's text ends
1025    /// that scalar, as any non-`ns-char` does; what follows is new
1026    /// content. Marks inside comments and block scalars: `marks_in_token`.)
1027    fn misplaced_bom(&mut self, start: usize, end: usize) {
1028        self.flush();
1029        self.push(
1030            EventKind::Error(misplaced_bom_diagnostic(start, end)),
1031            start,
1032            end,
1033        );
1034        self.pos = end;
1035        self.line_start = self
1036            .line_start
1037            .saturating_add(end.saturating_sub(start))
1038            .min(end);
1039    }
1040
1041    // ------------------------------------------------------------ structure
1042
1043    /// A child node of the innermost frame is complete.
1044    fn child_done(&mut self) {
1045        match self.stack.last_mut() {
1046            Some(Frame::Document { root_done }) => *root_done = true,
1047            Some(Frame::Map { phase, .. }) => {
1048                *phase = match *phase {
1049                    MapPhase::ExplicitKey => MapPhase::AfterExplicitKey,
1050                    MapPhase::AfterExplicitKey | MapPhase::Value | MapPhase::Key => MapPhase::Key,
1051                };
1052            }
1053            Some(Frame::Seq { .. }) | None => {}
1054        }
1055        self.state = State::Next;
1056    }
1057
1058    /// Closes the innermost block collection with a zero-width end. `certain`:
1059    /// the end is confirmed (a strict dedent or the end of the document) and
1060    /// is emitted at once; otherwise it is deferred until the next token is
1061    /// accepted by an outer collection (timing convention T2).
1062    fn close_block(&mut self, certain: bool) {
1063        let end = match self.stack.pop() {
1064            Some(Frame::Map { .. }) => EventKind::MappingEnd,
1065            Some(Frame::Seq { .. }) => EventKind::SequenceEnd,
1066            Some(frame @ Frame::Document { .. }) => {
1067                self.stack.push(frame);
1068                self.child_done();
1069                return;
1070            }
1071            None => {
1072                self.child_done();
1073                return;
1074            }
1075        };
1076        if certain {
1077            self.zero(end);
1078        } else {
1079            self.defer(end);
1080        }
1081        self.child_done();
1082    }
1083
1084    // ------------------------------------------------------------ errors
1085
1086    /// Reports a syntax error at the scan position covering the rest of the
1087    /// line, repairs the structure (C6 R2), and starts resynchronising (R4).
1088    /// `slot_open`: a node is owed to the innermost frame.
1089    fn fail(&mut self, code: Code, message: &str, slot_open: bool) {
1090        self.fail_with(code, message, slot_open, |d| d);
1091    }
1092
1093    /// [`Parser::fail`] with a diagnostic refined by `refine` (secondary
1094    /// labels, help, suggestions). Limit help is added automatically.
1095    fn fail_with(
1096        &mut self,
1097        code: Code,
1098        message: &str,
1099        slot_open: bool,
1100        refine: impl FnOnce(Diagnostic) -> Diagnostic,
1101    ) {
1102        let mark = self.mark_before_error.take();
1103        let blame = self.blame.take();
1104        if matches!(self.state, State::Skip) {
1105            // Already failed in this step: one error per problem, and never
1106            // a second error that could swallow the next line (e.g. `---`).
1107            // Emit anything scanned since, so the stream keeps tiling (C1)
1108            // when `skip` resumes at `pos`.
1109            self.flush();
1110            if self.emit_pos != self.pos || !self.deferred.is_empty() {
1111                // `flush` guarantees this; never a panic: a Y9000 event in
1112                // release.
1113                debug_assert!(false, "flush left owed events or pending trivia");
1114                self.internal_error("`flush` left owed events or pending trivia");
1115            }
1116            return;
1117        }
1118        // Deferred events were not confirmed by an accepted token: they go
1119        // after the error, as part of its repair (R2, R5).
1120        let deferred = core::mem::take(&mut self.deferred);
1121        self.flush();
1122        self.deferred = deferred;
1123        let start = self.pos;
1124        // Without a mark named by the failing construct, a mark
1125        // on the rest of the offending line that no quote precedes (so it
1126        // is not inside a quoted scalar, whose `nb-json` allows it) is
1127        // reported too. Limit and internal errors are about something
1128        // else.
1129        let mark = mark.or_else(|| {
1130            let about_syntax = code.get() < 1000;
1131            let text = self.src.get(start..line_end(self.src, start))?;
1132            let unquoted = text.find(['\'', '"']).unwrap_or(text.len());
1133            let k = text.get(..unquoted)?.find('\u{feff}')?;
1134            about_syntax.then_some(start.saturating_add(k))
1135        });
1136        // The mark that made this construct fail is reported
1137        // too, with its own span, as a zero-width event right after the
1138        // error (the first error event is the first diagnostic by
1139        // position, the one a strict parse fails with).
1140        //
1141        // A mark is never before `start` (= `self.pos`), so the error always
1142        // comes first. The fallback above searches from `start`;
1143        // `mark_then_fail` callers pass a mark found from the scan position on
1144        // (`directive`) or in a block scalar header that the scan position has
1145        // not passed (`block_scalar`); `block_scalar_error` places `pos` at the
1146        // offending line's start, or past its leading spaces, which hold no
1147        // mark; and a node head's mark (`head`, the mark after its content) is
1148        // used only by a failure at the head's start, or cleared (`commit_key`,
1149        // `emit_node`, `inline_colon`, a head that is not a key) before the
1150        // scan position moves past it. Never a panic: should that ever break,
1151        // an internal error in release, never a misordered report.
1152        let mark_diag =
1153            |m: usize| misplaced_bom_diagnostic(m, m.saturating_add('\u{feff}'.len_utf8()));
1154        if mark.is_some_and(|m| m < start) {
1155            debug_assert!(false, "a byte order mark before the error's start");
1156            self.internal_error("a byte order mark before the error's start");
1157        }
1158        // At a document marker the error is zero-width, so resynchronisation
1159        // resumes at that marker (R4).
1160        let at_marker = start == self.line_start
1161            && (scan::any_marker_at(self.src, start) || self.prefix_mark_at(start));
1162        let end = if at_marker {
1163            start
1164        } else {
1165            line_end(self.src, start)
1166        };
1167        // The report points at the first character that cannot belong, when
1168        // the producer named one; the event keeps covering the line.
1169        let blame = blame.or_else(|| scalar::quoted_fault(self.src, start, code));
1170        let at = blame.filter(|b| *b >= start).unwrap_or(start);
1171        // A byte order mark that made the construct fail is reported right
1172        // after it, so the error never starts after that mark.
1173        let at = mark.filter(|m| *m >= start).map_or(at, |m| at.min(m));
1174        let mut diag = Diagnostic::error(code, message, Label::new(Span::new(at, end.max(at)), ""));
1175        if let Some(help) = self.limit_help(code) {
1176            diag = diag.with_help(help);
1177        } else if let Some(help) = default_help(code) {
1178            diag = diag.with_help(help);
1179        }
1180        let diag = refine(diag);
1181        self.push(EventKind::Error(Box::new(diag)), start, end);
1182        self.pos = end;
1183        if let Some(m) = mark.filter(|m| *m >= start) {
1184            self.push(EventKind::Error(mark_diag(m)), end, end);
1185        }
1186        let mut repairs = core::mem::take(&mut self.flow_repair);
1187        for owed in repairs.drain(..) {
1188            match owed {
1189                Owed::Kind(kind) => self.zero(kind),
1190                Owed::Empty(props) => self.zero_node(EventKind::plain(), props),
1191            }
1192        }
1193        self.flow_repair = repairs;
1194        self.repair(slot_open);
1195        self.materialize();
1196        self.state = State::Skip;
1197    }
1198
1199    /// [`Parser::fail`], also reporting the byte order mark at `mark`, if
1200    /// any, that made the construct fail (after the error
1201    /// when the mark lies inside the offending line).
1202    pub(super) fn mark_then_fail(
1203        &mut self,
1204        mark: Option<usize>,
1205        code: Code,
1206        message: &str,
1207        slot_open: bool,
1208    ) {
1209        self.mark_before_error = mark;
1210        self.fail(code, message, slot_open);
1211    }
1212
1213    /// [`Parser::fail`] for a broken internal invariant: a `Y9000` error
1214    /// with the usual repair and resynchronisation. Callers
1215    /// `debug_assert!` first, so this runs only in release builds.
1216    fn fail_internal(&mut self, what: &str, slot_open: bool) {
1217        let message = format!("internal error: {what}");
1218        self.fail_with(INTERNAL_ERROR, &message, slot_open, |d| {
1219            d.with_help("this is a bug in oneyaml; please report it with this input")
1220        });
1221    }
1222
1223    /// Reports a broken internal invariant where no repair is possible: a
1224    /// zero-width `Y9000` error at `emit_pos` after any owed events, then
1225    /// the pending bytes as skipped trivia, so the stream still tiles (C1).
1226    /// Callers `debug_assert!` first.
1227    fn internal_error(&mut self, what: &str) {
1228        self.materialize();
1229        let at = self.emit_pos;
1230        self.push(EventKind::Error(internal(what, Span::empty(at))), at, at);
1231        if at < self.pos {
1232            let end = self.pos;
1233            self.push(EventKind::Trivia(TriviaKind::Skipped), at, end);
1234        }
1235    }
1236
1237    /// A zero-width error at `emit_pos` that needs no repair (the parse
1238    /// continues as if the problem were absent).
1239    fn soft_error(&mut self, code: Code, message: &str) {
1240        let at = self.emit_pos;
1241        self.soft_error_at(code, message, at);
1242    }
1243
1244    /// [`Parser::soft_error`] whose diagnostic points at `at`, an earlier
1245    /// position than the event (which stays zero-width at the scan end).
1246    fn soft_error_at(&mut self, code: Code, message: &str, at: usize) {
1247        self.zero(EventKind::Error(diagnostic(code, message, Span::empty(at))));
1248    }
1249
1250    /// Completes every open block collection with zero-width events after an
1251    /// error (C6 R2). The document stays open until resynchronisation.
1252    fn repair(&mut self, slot_open: bool) {
1253        // A failed directive block declares nothing (§6.8): its `%TAG`
1254        // handles and `%YAML` must not reach a later document.
1255        self.directives_pending = false;
1256        self.pending_tags.clear();
1257        self.yaml_seen = false;
1258        self.prefix_without_end = None;
1259        if self.doc_start_deferred {
1260            // Nothing of this document was emitted: drop it entirely.
1261            self.doc_start_deferred = false;
1262            self.deferred.clear();
1263            self.props = Properties::none();
1264            self.stack.clear();
1265            return;
1266        }
1267        if slot_open || !self.props.is_empty() {
1268            self.empty();
1269            self.child_done();
1270        }
1271        loop {
1272            match self.stack.last().copied() {
1273                Some(Frame::Map { phase, .. }) if phase != MapPhase::Key => {
1274                    self.empty();
1275                    self.child_done();
1276                }
1277                Some(Frame::Map { .. } | Frame::Seq { .. }) => self.close_block(false),
1278                Some(Frame::Document { root_done }) => {
1279                    if !root_done {
1280                        self.empty();
1281                        self.child_done();
1282                    }
1283                    break;
1284                }
1285                None => break,
1286            }
1287        }
1288    }
1289
1290    /// Skips one line (or line break) after an error, until a document
1291    /// marker at the start of a line or the end of input (C6 R4). One line
1292    /// per step, so skipping never buffers.
1293    fn skip(&mut self) {
1294        let i = self.pos;
1295        let blen = break_len(self.src, i);
1296        if blen > 0 {
1297            let end = i.saturating_add(blen);
1298            self.push(EventKind::Trivia(TriviaKind::LineBreak), i, end);
1299            self.pos = end;
1300            self.line_start = end;
1301            self.line_indent = 0;
1302            return;
1303        }
1304        // A mark at the start of a line may start a document
1305        // prefix (production 211), so recovery resumes there too.
1306        let resync = i >= self.src.len()
1307            || !self.halted
1308                && i == self.line_start
1309                && (scan::any_marker_at(self.src, i) || scan::mark_at(self.src, i));
1310        if !resync {
1311            let end = line_end(self.src, i);
1312            self.push(EventKind::Trivia(TriviaKind::Skipped), i, end);
1313            self.pos = end;
1314            return;
1315        }
1316        self.fresh = true;
1317        self.line_indent = 0;
1318        self.tab_seen = false;
1319        self.state = if self.stack.is_empty() {
1320            State::Stream
1321        } else {
1322            State::Next
1323        };
1324    }
1325}
1326
1327#[cfg(test)]
1328mod equivalence_tests;
1329#[cfg(test)]
1330mod tests;