Struct StreamScanner 

pub struct StreamScanner { /* private fields */ }

A streaming scanner over one compiled pattern, or over a set.

Implementations

impl StreamScanner

pub fn new(pattern: Pattern) -> Self

Begin a streaming scan for pattern.

pub fn with_shapes(pattern: Pattern, shapes: ShapeSet) -> Self

Begin a streaming scan for pattern under the shapes and kinds shapes declares, which decide token boundaries as they do for crate::scan_with_shapes: the stream finds the matches that scan finds over the whole input.

pub fn over_set(set: PatternSet) -> Self

Begin a streaming scan for every member of set at once, each match tagged with its member. The stream commits under its most conservative member: a match commits once the decided prefix holds the longest span any member can make, nothing commits before the end where any member depends on the whole input, and a line’s end is no boundary where any member reads whitespace; one window serves them all.

pub fn commits_early(&self) -> bool

Whether a match can commit before the stream ends. A pattern that depends on the whole input retains everything until Self::finish.

pub fn base(&self) -> usize

The absolute offset of the first retained byte: everything before it is committed, and no later byte can reach it.

pub fn retained(&self) -> usize

The bytes retained now.

pub fn rebase(&mut self) -> usize

Count this stream’s offsets from its first retained byte rather than from where it began, and give how far they moved; a reader adds that to every offset reported after. A span’s offsets reach four GiB, so a stream that runs longer, as a file followed for days can, is rebased as it goes. Every committed match must have been drained first, since those carry the offsets from before.

Panics

A committed match has not been drained.

pub fn peak_retained(&self) -> usize

The most bytes retained at once so far.

pub fn push(&mut self, chunk: &[u8])

Feed the next chunk of input. Matches that can no longer change are committed; the rest is retained for the following chunk.

pub fn drain_committed(&mut self) -> Vec<Span>

The matches committed so far, handed over once: a later call returns only what was committed after it, and finish returns what it commits plus whatever was never drained.

pub fn drain_committed_with_members(&mut self) -> Vec<(usize, Span)>

Self::drain_committed, each match with the set member that made it; 0 throughout for a stream over one pattern.

pub fn finish(self) -> Vec<Span>

Finish the stream: scan the retained buffer and return every match not yet drained, in absolute offsets.

pub fn finish_with_members(self) -> Vec<(usize, Span)>

Self::finish, each match with the set member that made it.