1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
//! A fast reader for [ɴsɪ](https://nsi.readthedocs.io/) scenes.
//!
//! `nsi-intermediate` writes ɴsɪ out; this reads it back. Together they
//! close the loop: a scene can be captured, inspected, replayed and
//! round-tripped without a renderer.
//!
//! # It calls you
//!
//! Parsing drives [`nsi_trait::Nsi`] rather than producing a scene type
//! of its own, so the same parser feeds a live 3Delight context, an
//! `nsi-intermediate` `Recorder`, or a backend's own implementation. A
//! reader that insisted on its own representation would make every
//! consumer translate.
//!
//! A sink brings its own *behaviour*, not its own argument type: every
//! entry point here binds `Nsi<Arg<'call> = nsi_ffi_wrap::Arg<'call,
//! 'static>>`, because the parser has to build the arguments it hands
//! over and can only build the one shape. Implement [`nsi_trait::Nsi`]
//! with that associated type and the parser will drive it.
//!
//! ```no_run
//! # use nsi_parse::parse_stream;
//! # fn main() -> Result<(), Box<dyn std::error::Error>> {
//! let bytes = std::fs::read("scene.nsi")?;
//! let recorder = nsi_intermediate::Recorder::new();
//! parse_stream(&bytes, &recorder)?;
//! # Ok(())
//! # }
//! ```
//!
//! # The grammar is observed, not specified
//!
//! ɴsɪ publishes no grammar for its stream, only examples. This parser
//! is written against what 3Delight accepts, and the decisive
//! observation is that **an entire scene on one line parses**: the
//! newlines and indents a renderer writes are formatting, not syntax.
//! So a parameter list runs until the next *bare* token naming a
//! statement -- parameter names are always quoted, which makes that
//! unambiguous -- and a line-oriented reader would be wrong on valid
//! input.
//!
//! # Paths are carried verbatim
//!
//! A string value may hold a `${VAR}` reference -- ɴsɪ's
//! `streampathreplacement` writes them so a scene can move between
//! machines -- and this parser **does not expand them**. Rendered,
//! that is what 3Delight does too: `renderdl -cat` echoes
//! `${NSI_PATH_TEST}/out.exr` back unexpanded, while a render whose
//! `imagefilename` holds it writes to the expanded path. Expansion
//! happens at *use*, in whatever opens the file.
//!
//! Expanding here would bake one machine's paths into a stream whose
//! purpose is to move between them. The obligation is the consumer's:
//! open a path without expanding and you open the wrong file, on
//! exactly the machines where the variable was meant to matter. Note
//! that **any** variable expands, not only `NSI_PATH_`-prefixed ones;
//! the prefix governs only which ones 3Delight *writes* as
//! references.
//!
//! # Features
//!
//! | Feature | What it adds |
//! | --- | --- |
//! | *(none)* | The `.nsi` stream reader. |
//! | `lua` | Reading a Lua scene, which **runs** the script. Builds Lua 5.4 from vendored C source. |
//! | `gzip` | Reading a gzip-compressed stream. |
//! | `zstd` | Reading a zstd-compressed stream. |
//! | `parallel` | `parse_stream_parallel`: reading and applying a stream on every core. |
//!
//! Reading a Lua scene means executing it. ɴsɪ's Lua front end is a
//! programming language -- a script may compute the scene it describes
//! -- so an interpreter is the only correct reader, and that is a
//! different trust decision from parsing a data file.
use fmt;
use Nsi;
use Read;
/// Why a scene could not be read.
pub use run_lua;
/// Read an ɴsɪ stream, applying it to `sink`.
///
/// # Errors
///
/// [`Error::Syntax`] and its neighbours for input that is not the
/// grammar, and [`Error::Sink`] when the sink refuses a statement.
/// Read an ɴsɪ stream on every core, applying it to `sink`.
///
/// For a sink that accepts calls from many threads at once -- a
/// renderer's context. The resulting scene is the one [`parse_stream`]
/// produces, but the calls arrive in a different order:
///
/// 1. every `Create`, concurrently, in stream order per handle;
/// 2. then every `SetAttribute` and `SetAttributeAtTime`, concurrently,
/// in stream order per handle, and every `Connect`, concurrently, in
/// stream order per destination attribute.
///
/// `Delete`, `DeleteAttribute`, `Disconnect`, `Evaluate` and
/// `RenderControl` are **barriers**: everything before one is applied
/// before it, it is applied alone, and nothing after it is applied
/// before it. So an interactive stream -- edits, a `RenderControl`,
/// more edits -- keeps its meaning.
///
/// # What it buys
///
/// Measured on 20 000 meshes, 5.3 MiB, 16 threads: into a sink that does
/// nothing, 166 MiB/s becomes about 320. **Into 3Delight 2.9, only
/// 1.15-1.4x**: its context serialises calls internally -- 20 000
/// `SetAttribute`s take 11.8 ms from one thread and 21.4 ms from
/// sixteen -- so what is gained there is parsing overlapped with the
/// renderer, not calls made side by side. A sink that takes calls
/// concurrently gains the most.
///
/// # Differences from `parse_stream`
///
/// - **A reference to a node created later succeeds.** Every `Create`
/// in a segment runs first, so a `SetAttribute` on a node the stream
/// creates further down finds it, where the sequential parser fails.
/// - **On failure, which statements were applied is unspecified.**
/// Statements that do not depend on the failing one may have been
/// applied, before or after it in the stream. The error returned is
/// the earliest failure found, and a stream with a single error
/// reports the same error as [`parse_stream`].
///
/// # Errors
///
/// As [`parse_stream`].
/// Read an ɴsɪ stream that may be compressed, applying it to `sink`.
///
/// The compressor is detected from the input's leading bytes, so a
/// caller that took a file from a pipeline does not have to know how it
/// was written. Uncompressed input is passed straight through.
///
/// Only gzip is a format 3Delight itself reads; zstd is supported here
/// for consumers of this workspace. See `nsi-intermediate`'s
/// `Compression`.
///
/// # Errors
///
/// [`Error::Decompress`] when the input announces a compressor and then
/// fails to decompress, plus everything [`parse_stream`] returns.