Crate ezno_parser
source ·Expand description
§Ezno’s Parser
Contains “string to AST” parser, AST definitions, AST back to text/string form methods and hooks for traversing/visiting AST. Used in ezno-checker
and the Ezno toolchain.
use ezno_parser::{ASTNode, Expression};
fn main() {
let expressions = [
"4 + 2 * 5",
"4 * 2 + 5",
"4 * 2 * 5",
"console.log(4 * 2, t ? true : `Hi`) == 2 && 4 == 2",
];
for expression in expressions {
let expression = Expression::from_string(expression.to_owned(), Default::default());
println!("{expression:#?}");
}
}
Also checkout other parsers such as boa, biome, oxc and swc.
§Goals
- Keep under 15k lines of code (excluding
/tests
and/examples
folders) - Easy to use API (see
/tests
and/examples
folders for example usage) - Run in WASM
- Keep readable and maintainable
- Designed for analysis and transformations
- See expression identifiers can be used to bind information to
- Retain source positions for use in analysis diagnostics and generating source maps
- All AST should be visitable. Immutably to collect facts or mutable to transform/remove
- Optionally via configuration extend the ECMAScript language definition
- TypeScript type annotations
- Interfaces, enums and type alias statements
- Parameter, return type and variable annotations
-
satisfies
andas
😑 - JSX support - Includes HTML comments, special handing of self closing tags from the specification
- Others under
feature = "extras"
👀
- TypeScript type annotations
- Interfaces, enums and type alias statements
- Parameter, return type and variable annotations
-
- Transformation and visiting
- See example
ezno-parser-visitable-derive
is a macro that automates/generates the visiting implementation- The generator macro also makes creating AST easy. See example
- Positions in source
- All syntax has reference to where it was in the source using a Span. This uses the source-map crate crate which makes it trivial to build source maps.
- Partial AST
- Most of the parser requires valid input. However under a option you can enable a option which can add marked nodes for cases where a expression might be missing. This allows tools that require an AST to work with a source that is still being edited (such as in a LSP).
- It checks two cases, if either of these cases is encountered, then it adds a marker node: (1) Whether the expression ends in
)
etc (useful inif
etc) or (2) Whether the next token isconst
etc and on the next line
- Output
- Stripping type annotations can be stripped from output using
ToStringOptions { include_types: false, ..Default::default() }
- Adding indentation under
pretty: true
, not adding whitespace for production builds - Setting
max_line_length
to a size to wrap certain structures - Support for source map mapping generation
- Stripping type annotations can be stripped from output using
§Non-goals
- CSTs, close to source operations etc
- Source with unbalanced parenthesis/brackets
- Increase code size or decrease readability for minor speed improvements
§Testing
If in main root rather than this folder, add
-p ezno-parser
aftercargo run
to the following commands.
For testing whether the parser can lex a file
cargo run --example lex path/to/file.js
and parse
cargo run --example parse path/to/file.js
Note the Ezno CLI includes the
ast-playground
subcommand: a more user oriented version of these commands
Re-exports§
pub use declarations::Declaration;
pub use marker::Marker;
pub use expressions::Expression;
pub use expressions::PropertyReference;
pub use functions::FunctionBase;
pub use functions::FunctionBased;
pub use functions::FunctionHeader;
pub use generator_helpers::IntoAST;
pub use lexer::lex_script;
pub use property_key::PropertyKey;
pub use statements::Statement;
pub use types::type_annotations;
pub use types::type_annotations::TypeAnnotation;
pub use types::type_declarations;
pub use types::type_declarations::TypeParameter;
pub use source_map;
pub use options::*;
Modules§
- Re-exports or generator and general use
- Includes type annotations + syntax added by TypeScript (and Ezno) such as
declare
declarations - Contains logic that makes viewing and transform parts of AST trivial through the visitor pattern
Structs§
- A “block” of braced statements and declarations
- TODO under cfg if don’t want this could just be
type Decorated<T> = T;
- As parsing is forwards, this is ordered
- A error for not parsing
- A identifier for a crate::Source
- Covers [
ArrayDestructuring
] ANDObjectDestructuringField
Enums§
- For
- For ifs and other statements
- TODO spread attributes and boolean attributes
- For
- What surrounds a string
- All JS Tokens with extensions including TypeScript, JSX and more
- A variable declaration name, used in variable declarations and function parameters. See destructuring
Traits§
- Defines common methods that would exist on a AST part include position in source, creation from reader and serializing to string from options.
- Classes and
function
functions have two variants depending whether in statement position or expression position
Functions§
- TODO WIP!
- Used for lexing
- Used for lexing
- For demos and testing