Crate scnr

scnr

The ‘scnr’ crate is a library that provides lexical scanner for programming languages. It is designed to be used in a parser of a compiler or interpreter for a programming language or in similar tools that require lexical analysis, e.g. in a language server. It provides multiple scanner modes out of the box, which can be switched at runtime depending on the context of the input. A parser can use different modes for different parts of the input, e.g. to scan comments in one mode and code in another. The scanner is designed to be fast and efficient, and it is implemented with the help of finite state machines. To parse the given regular expressions, the crate uses the regex-syntax crate.

Structs

• FindMatches
  An iterator over all non-overlapping matches.
• Lookahead
  A lookahead is used to restrict a match in the input. The lookahead is a regular expression whose condition must be met after the pattern itself for it to be considered a match. The lookahead can be positive or negative.
• Match
  A match in the haystack.
• MatchExt
  A match with line and column information for start and end positions.
• Pattern
  A pattern that is used to match the input. The pattern is represented by a regular expression and a token type number. The token type number is used to identify the pattern in the scanner. The pattern also has an optional lookahead.
• Position
  A position in the haystack. The position is represented by a line and column number. The line and column numbers are 1-based.
• Scanner
  A Scanner. It consists of multiple DFAs resp. NFAs that are used to search for matches.
• ScannerBuilder
  A builder for creating a scanner.
• ScannerMode
  A scanner mode that can be used to scan specific parts of the input. It has a name and a set of patterns that are valid token types in this mode. The scanner mode can also have transitions to other scanner modes triggered by a token type.
• ScnrError
  The error type for the scrn crate.
• Span
  A span in an input string.
• WithPositions
  An iterator over all non-overlapping matches with positions.

Enums

• PeekResult
  The result of a peek operation.
• ScnrErrorKind
  The error kind type.

Traits

• MatchExtIterator
  An extension trait for iterators over matches.
• PositionProvider
  A trait for providing the line and column information of a given byte offset in the haystack.
• ScannerModeSwitcher
  A trait to switch between scanner modes.

Type Aliases

• Result
  The result type for the scrn crate.