Crate ezno_parser
source ·Expand description
Ezno’s Parser
Contains “string to AST” parser, AST definitions, AST back to text/string form methods and hooks for traversing/visiting AST. Used in the Ezno checker.
This is more of an exercise project in getting better at writing Rust and doesn’t offer too much over other great Rust based JS parsers such as swc, rome, oxc and boa.
Goals:
- Keep under 15k lines of code (excluding
/tests
and/examples
folders) - Easy to use (see
/tests
and/examples
folders) - Keep readable and maintainable
- Designed for analysis and transformations
- See expression identifiers can be used to bind information to
- Retain source positions for throwing errors
- All AST should be visitable. Immutably to collect facts or mutable to transform/remove
- Optionally via configuration extend the ECMAscript language definition
- TypeScript type annotations
- Interfaces, enums and type alias statements
- Parameter, return type and variable annotations
satisfies
andas
😑
- JSX support
- Includes HTML comments, special handing of self closing tags from the specification
- Others under
feature = "extras"
👀
- TypeScript type annotations
Non-goals
- CSTs, close to source operations etc
- Source with unbalanced parenthesis/brackets
- Increase code size or decrease readability for speed improvements
- Allow adding new syntax at runtime, that would require modifying the lexer at runtime adding new tokens
Features
Positions
All syntax has reference to where it was in the source using a Span. This uses the source-map crate, so it can generate source maps.
Identifiers
Most expressions, functions, blocks and some other AST has a unique identifier. This can be used to associate information with the node. For example Ezno’s checker associates type information with variable reference AST for later visitors or for hover information in the LSP.
“Cursors”
Allows holes in AST where a cursor exists. This allows for LSP to provide suggestions here while the whole source might not be valid.
Function extraction
All functions end up in a map rather than in the AST, which can make some static analysis and transformation operations easier.
Visiting
Generator
Easily generate AST nodes with data interpolation using the constant compiled quasi-quoted macro. See example.
Notable structures
- ASTNode, a trait that all AST should implement
- FunctionBase, a abstraction for functions
- Expression
- Statements
- Operators
- TSXToken
- TSXKeyword
Re-exports
pub use cursor::CursorId;
pub use cursor::EmptyCursorId;
pub use declarations::Declaration;
pub use expressions::Expression;
pub use expressions::PropertyReference;
pub use extractor::UniversalFunctionId;
pub use functions::FunctionBase;
pub use functions::FunctionBased;
pub use functions::FunctionHeader;
pub use functions::FunctionId;
pub use parameters::FunctionParameters;
pub use parameters::OptionalOrWithDefaultValueParameter;
pub use parameters::Parameter;
pub use parameters::SpreadParameter;
pub use source_map;
pub use statements::Statement;
pub use types::type_declarations;
pub use types::type_declarations::GenericTypeConstraint;
pub use types::type_declarations::TypeDeclaration;
pub use types::type_references;
pub use types::type_references::TypeReference;
pub use types::TypeId;
Modules
- Re-exports or generator and general use
- Contains the definitions for expressions Operators marked in spec but implemented as a variant of crate::Expression rather than a operator: OptionalChain, OptionalCall, OptionalIndex, Index, Group, Initialize, Call,
Structs
- A “block” of braced statements and declarations
- A identifier for a group of statements
- The current location in the AST
- TODO under cfg if don’t want this could just be
type Decorated<T> = T;
- A keyword
- A error for not parsing
- Settings to customize parsing
- A identifier to some source
- A start and end. Also contains trace of original source
- Settings for serializing ASTNodes
- For function type references
- Id given to AST that declares a variable
- Options for behavior when visiting AST. Customizable behavior is important for analysis
Enums
- TODO not sure about the positions here, is potential duplication if T::OptionalExpression is none
- For ifs and other statements
- Could borrow identifiers but most of them are smaller than a pointer so it doesn’t matter =
- TODO these may go
- TODO spread attributes and boolean attributes
- TODO these may go
- TODO BigInt TODO a mix between runtime numbers and source syntax based number https://tc39.es/ecma262/multipage/ecmascript-language-lexical-grammar.html#sec-literals-numeric-literals
- A key for a member in a class or object literal
- The notation of a string
- All JS Tokens with extensions including TypeScript, JSX and more
- Statements for ‘.d.ts’ files
- A variable declaration name, used in variable declarations and function parameters. See destructuring
Traits
- Defines common methods that would exist on a AST part include position in source, creation from reader and serializing to string from settings.
- Classes and
function
functions have two variants depending whether in statement position or expression position - A trait which means that self can be pushed to a [TokenSender]
- Variable field can be used in type annotations but cannot have a value
- For something to visitable it can visit all nested fields.
- A visitor over something which is hooked/is SelfVisitable with some Data
- A visitor over something which is hooked/is SelfVisitable with some Data
- These are a receiver traits of the visitor
- These are a receiver traits of the visitor
Functions
- Used for lexing
- Used for lexing