1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
//! *glast* is a crate for parsing and manipulating **gl**sl **a**bstract **s**yntax **t**rees, and a lot more.
//!
//! ⚠ This crate is still heavily **work-in-progress**. The main functionality & API of the lexer and parser are
//! roughly finalized, but small changes and tweaks are still to be expected, and other features are not
//! implemented yet.
//!
//! This crate is part of a larger effort to provide a high-quality LSP implementation for GLSL. Hence, this crate
//! exposes more than just the basic lexing and parsing functionality found in other crates or libraries for other
//! languages:
//! - The lexer changes the grammar depending on an encountered version directive, in line with the specification.
//! - Lexing and parsing functions produce useful metadata about their inputs.
//! - The parser produces not only an AST, but also syntax and (some) semantic diagnostics, as well as syntax
//! highlighting information.
//! - The parser correctly expands arbitrarily-complex macros anywhere that it should, in line with the
//! specification.
//! - The parser correctly handles all conditional compilation, no matter how convoluted. Full control over which
//! conditional branches to include/exclude is provided, include the ability to evaluate conditional directives
//! on-the-fly.
//! - Both the lexer and parser are aiming to be 100% specification compliant.
//!
//! The currently supported GLSL versions are:
//!
//! |GLSL version|OpenGL version|
//! |-|-|
//! |460|4.6|
//! |450|4.5|
//!
//! There are plans to support many more versions, not limited to but including 110 (default), 130, 330, 400, 410,
//! and 100 (es), 300 es & 320 es
//!
//! # The parsing pipeline
//! This crate is split into modules representing the different stages of parsing/manipulation:
//! - [`lexer`] - Lexer and the token stream.
//! - [`parser`] - Parser and the abstract syntax tree.
//! - `analyzer` - High intermediate representation and analysis, such as name resolution.
//!
//! This crate operates only on [`str`] inputs because the GLSL specification states that GLSL source strings must
//! use the UTF-8 encoding. That means that if the source can be parsed into a valid Rust [`str`] then it must be
//! valid, so no support for `[u8]` inputs is provided.
//!
//! ## Source String
//! We start with a string of characters that makes up the glsl source file. We will use the following example to
//! illustrate the pipeline:
//! ```c
//! // Comment
//! int i = 5.0+1;
//! ```
//!
//! ## Lexer
//! This is the first transformation in the parsing pipeline, and it converts a string of characters into a stream
//! of tokens. The source string would become (in pseudocode):
//! ```text
//! (comment " Comment") (ident "int") (ident "i") (op "=") (float "5.0") (op "+") (int "1") (punct ";")
//! ```
//!
//! ## Parser
//! This is the next transformation in the parsing pipeline, and it converts the token stream into a tree that only
//! contains semantic information, loosing things like irrelevant punctuation symbols or comments.
//!
//! Firstly, conditional compilation is applied to the token stream. Then, the resulting token stream is parsed,
//! expanding any macros in the process.
//!
//! The token stream would become (in pseudocode):
//! ```text
//! VariableDeclaration {
//! type: Primitive.Int,
//! ident: Identifier("i"),
//! value: BinaryExpression {
//! left: Float(5.0),
//! op: Addition,
//! right: Int(1),
//! }
//! }
//! ```
//!
//! ## Analyzer
//! This is the final stage of the parsing pipeline, and includes actions such as name resolution and
//! type-checking.
//!
//! ⚠ This module is **not implemented yet**.
//!
//! # Diagnostics
//! The parser produces syntax errors when it encounters invalid syntax. It also produces semantic diagnostics in
//! relation to macros. All other semantic diagnostics are currently waiting on the `analyzer` module to be
//! implemented.
//!
//! For more details, see the [`diag`] module.
//!
//! # Syntax Highlighting
//! The parser produces syntax highlighting tokens as part of the parsing process. These tokens correctly highlight
//! all text that has been parsed, but they are not fully semantically-aware; this is because the parser only
//! creates an abstract syntax tree, it does not perform name-resolution. This means that currently most
//! identifiers are of the `UncheckedIdent` variant. Once the `analyzer` module has been implemented to include
//! name-resolution functionality, these syntax tokens can be converted into more specific variable/type/function
//! identifiers, which would result in fully semantically-aware highlighting information.
//!
//! For more details, see the [`syntax`] module.
//!
//! # Security
//! This crate has zero uses of `unsafe`.
//!
//! This crate does not execute arbitrary source code; any evaluations (such as evaluating `#if` expressions) are
//! free of any side-effects. This crate takes in string slices, performs analysis operations, and returns data
//! structures; it does not perform any I/O or networking calls.
//!
//! # Performance & Testing
//! Currently, this crate has not been particularly optimized for performance; there is a lot of `clone()`-ing. The
//! general internal structure is well suited for optimization, but the work has not yet been done because the
//! internals have undergone numerous large-scale refactors and re-writes whilst the functionality and public api
//! have matured. Unless any massive oversights have occurred, the current structure is the one that will go
//! forward, and once this crate has been given a bit of use to iron out any remaining minor bugs, it will be
//! optimized to minimize data cloning and memory allocations.
//!
//! Currently, this crate does not have extensive testing. Some parts of the crate have good unit test coverage,
//! but other parts lack any tests completely and there are no integration tests yet. This is an important goal
//! that is being progressively worked towards.
pub use *;
/// Describes a GLSL version.
/// Holds either one or the other value.
/// Holds one of 3 possible values.