1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168
//! Extremely fast, lossless, and error tolerant JavaScript Parser.
//!
//! The parser uses an abstraction over non-whitespace tokens.
//! This allows us to losslessly or lossly parse code without requiring explicit handling of whitespace.
//! The parser yields events, not an AST, the events are resolved into untyped syntax nodes, which can then
//! be casted into a typed AST.
//!
//! The parser is able to produce a valid AST from **any** source code.
//! Erroneous productions are wrapped into `ERROR` syntax nodes, the original source code
//! is completely represented in the final syntax nodes.
//!
//! You probably do not want to use the parser struct, unless you want to parse fragments of Js source code or make your own productions.
//! Instead use functions such as [parse_script], and [parse_module] which offer abstracted versions for parsing.
//!
//! For more finer control, use [parse](crate::parse::parse) or [parse_js_with_cache],
//!
//! Notable features of the parser are:
//! - Extremely fast parsing and lexing through the extremely fast lexer.
//! - Ability to do Lossy or Lossless parsing on demand without explicit whitespace handling.
//! - Customizable, able to parse any fragments of JS code at your discretion.
//! - Completely error tolerant, able to produce an AST from any source code.
//! - Zero cost for converting untyped nodes to a typed AST.
//! - Ability to go from AST to SyntaxNodes to SyntaxTokens to source code and back very easily with nearly zero cost.
//! - Very easy tree traversal through [`SyntaxNode`](biome_rowan::SyntaxNode).
//! - Descriptive errors with multiple labels and notes.
//! - Very cheap cloning, cloning an ast node or syntax node is the cost of adding a reference to an Rc.
//! - Cheap incremental reparsing of changed text.
//!
//! The crate further includes utilities such as:
//! - ANSI syntax highlighting of nodes or text through `lexer`.
//!
//! It is inspired by the rust analyzer parser but adapted for JavaScript.
//!
//! # Syntax Nodes vs AST Nodes
//! The crate relies on a concept of untyped [biome_js_syntax::JsSyntaxNode]s vs typed [biome_rowan::AstNode]s.
//! Syntax nodes represent the syntax tree in an untyped way. They represent a location in an immutable
//! tree with two pointers. The syntax tree is composed of [biome_js_syntax::JsSyntaxNode]s and [biome_js_syntax::JsSyntaxToken]s in a nested
//! tree structure. Each node can have parents, siblings, children, descendants, etc.
//!
//! [biome_rowan::AstNode]s represent a typed version of a syntax node. They have the same exact representation as syntax nodes
//! therefore a conversion between either has zero runtime cost. Every piece of data of an ast node is optional,
//! this is due to the fact that the parser is completely error tolerant.
//!
//! Each representation has its advantages:
//!
//! ### SyntaxNodes
//! - Very simple traversing of the syntax tree through functions on them.
//! - Easily able to convert to underlying text, range, or tokens.
//! - Contain all whitespace bound to the underlying production (in the case of lossless parsing).
//! - Can be easily converted into its typed representation with zero cost.
//! - Can be turned into a pretty representation with fmt debug.
//!
//! ### AST Nodes
//! - Easy access to properties of the underlying production.
//! - Zero cost conversion to a syntax node.
//!
//! In conclusion, the use of both representations means we are not constrained to acting through
//! typed nodes. Which makes traversal hard and you often have to resort to autogenerated visitor patterns.
//! AST nodes are simply a way to easily access subproperties of a syntax node.event;
//!
//! ## Parser Tests
//!
//! Parser tests are comments that start with `test` or `test_err` followed by the test name, and then the code on its own line.
//!
//! ```rust,ignore
//! // test js feature_name
//! // let a = { new_feature : "" }
//! // let b = { new_feature : "" }
//! fn parse_new_feature(p: &mut Parser) -> ParsedSyntax {}
//! ```
//!
//! * `test`: Test for a valid program. Should not produce any diagnostics nor missing nodes.
//! * `test_err`: Test for a program with syntax error. Must produce a diagnostic.
//!
//! By default, the test runs as a JavaScript Module. You can customize the source type by specifying the
//! file type after `test` or `test_err`
//!
//! ```rust,ignore
//! // test ts typescript_test
//! // console.log("a");
//! if a {
//! // ..
//! }
//! ```
//!
//! The supported source types are:
//! * `js`
//! * `jsx`
//! * `ts`
//! * `tsx`
//! * `d.ts`
//!
//! To enable script mode, add a `// script` comment to the code.
//!
//! To extract the test cases, run `cargo codegen test`. Running the codegen is necessary whenever you add,
//! change, or remove inline tests .
//!
//! To update the test output, run
//!
//!
//! **Linux/MacOs**:
//!
//! ```bash
//! env UPDATE_EXPECT=1 cargo test
//! ```
//!
//! **Windows**
//!
//! ```powershell
//! set UPDATE_EXPECT=1 & cargo test
//! ```
mod parser;
#[macro_use]
mod lexer;
mod parse;
mod rewrite;
mod span;
mod state;
#[cfg(any(test, feature = "tests"))]
pub mod test_utils;
#[cfg(test)]
mod tests;
pub mod options;
mod prelude;
pub mod syntax;
mod token_source;
use crate::prelude::*;
pub(crate) use crate::ParsedSyntax::{Absent, Present};
pub use crate::{
lexer::{JsLexContext, JsReLexContext},
options::JsParserOptions,
parse::*,
};
use biome_js_factory::JsSyntaxFactory;
use biome_js_syntax::{JsLanguage, JsSyntaxKind, LanguageVariant};
use biome_parser::tree_sink::LosslessTreeSink;
pub(crate) use parser::{JsParser, ParseRecoveryTokenSet};
pub(crate) use state::{JsParserState, StrictMode};
use std::fmt::Debug;
pub enum JsSyntaxFeature {
#[allow(unused)]
#[doc(alias = "LooseMode")]
SloppyMode,
StrictMode,
TypeScript,
Jsx,
}
impl SyntaxFeature for JsSyntaxFeature {
type Parser<'source> = JsParser<'source>;
fn is_supported(&self, p: &JsParser) -> bool {
match self {
JsSyntaxFeature::SloppyMode => p.state().strict().is_none(),
JsSyntaxFeature::StrictMode => p.state().strict().is_some(),
JsSyntaxFeature::TypeScript => p.source_type().language().is_typescript(),
JsSyntaxFeature::Jsx => p.source_type().variant() == LanguageVariant::Jsx,
}
}
}
pub(crate) type JsLosslessTreeSink<'source> =
LosslessTreeSink<'source, JsLanguage, JsSyntaxFactory>;