Module stream 

The parse source for the SATySFi surface grammar.

The parse source is the eagerly lexed Vec<Atom>, wrapped by AtomStream: syan core has no IntoParseStream for Vec<_>, so the buffering lives here.

Stream erasure (a &mut dyn ParseStream tower) does not belong here, and is obsoleted by syan: Parse::parse_stream takes &mut S and recursion reborrows, so S is a genuine fixed point and the instantiation set is finite without erasing anything, and no stream operation is a virtual call.

The high-water mark

This module used to decline a failure high-water mark too, on the grounds that “ParseError is span-generic, every variant carrying the position it failed at, so the error reports itself”. That was false, and this type now carries the mark because of it. ParseError does carry a position, but not a useful one for a failure inside a repetition: Vec<TopBinding> stops on the binding that would not parse and rolls the stream back, and its error is discarded rather than aggregated, so what surfaces is the enclosing rule’s “expected end of input” at the binding’s START. Measured, a 0.0.6 error sixty bytes into a top-level let reported at byte 3; a 0.1 error anywhere in a file reported on the module keyword on line 1, because a 0.1 library IS one binding.

The furthest-position-reached mark is the standard answer for a backtracking parser, and the stream is the only place it can be observed: it is a property of the parse, not of any one error value. next() records the furthest atom ever handed out and never forgets it, so backtracking cannot erase the evidence, and crate::parse_error::locate turns mark + error tree into one diagnostic.

The budget

Both grammars backtrack exponentially on some incomplete inputs — see Budget for the measurements and for why a compiler, and not only a language server, wants a cap.

Structs

AtomStream

A parse source over an eagerly lexed token vector, which remembers how far the parse ever got and stops it if it goes on too long.

Budget

How much backtracking one parse may do before AtomStream declares the input unparseable and reports end of input.