tor_netdoc/parse2.rs
1//! New netdoc parsing arrangements, with `derive`
2//!
3//! # Parsing principles
4//!
5//! A parseable network document is a type implementing [`NetdocParseable`].
6//! usually via the
7//! [`NetdocParseable` derive=deftly macro`](crate::derive_deftly_template_NetdocParseable).
8//!
9//! A document type is responsible for recognising its own heading item.
10//! Its parser will also be told other of structural items that it should not consume.
11//! The structural lines can then be used to pass control to the appropriate parser.
12//!
13//! A "structural item" is a netdoc item that is defines the structure of the document.
14//! This includes the intro items for whole documents,
15//! the items that introduce document sections
16//! (which we model by treating the section as a sub-document)
17//! and signature items (which introduce the signatures at the end of the document,
18//! and after which no non-signature items may appear).
19//!
20//! # Ordering
21//!
22//! We don't parse things into a sorted order.
23//! Sorting will be done on output.
24// TODO we don't implement deriving output yet.
25//!
26//! # Types, and signature handling
27//!
28//! Most top-level network documents are signed somehow.
29//! In this case there are three types:
30//!
31//! * **`FooSigned`**: a signed `Foo`, with its signatures, not yet verified.
32//! Implements [`NetdocSigned`],
33//! typically by invoking the
34//! [`NetdocSigned` derive macro](crate::derive_deftly_template_NetdocSigned)
35//! on `Foo`.
36//!
37//! Type-specific methods are provided for verification,
38//! to obtain a `Foo`.
39//!
40//! * **`Foo`**: the body data for the document.
41//! This doesn't contain any signatures.
42//! Having one of these to play with means signatures have already been validated.
43//! Implement `NetdocParseable`, via
44//! [derive](crate::derive_deftly_template_NetdocParseable).
45//!
46//! * **`FooSignatures`**: the signatures for a `Foo`.
47//! Implement `NetdocParseable`, via
48//! [derive](crate::derive_deftly_template_NetdocParseable),
49//! with `#[deftly(netdoc(signatures))]`.
50//!
51//! # Naming conventions
52//!
53//! * `DocumentName`: important types,
54//! including network documents or sub-documents,
55//! eg `NetworkStatsuMd` and `RouterVote`,
56//! and types that are generally useful.
57//! * `NddDoucmnetSection`: sections and sub-documents
58//! that the user won't normally need to name.
59//! * `NdiItemValue`: parsed value for a network document Item.
60//! eg `NdiVoteStatus` representing the whole of the RHS of a `vote-status` Item.
61//! Often not needed since `ItemValueParseable` is implemented for suitable tuples.
62//! * `NdaArgumentValue`: parsed value for a single argument;
63//! eg `NdaVoteStatus` representing the `vote` or `status` argument.
64//!
65//! # Relationship to tor_netdoc::parse
66//!
67//! This is a completely new parsing approach, based on different principles.
68//! The key principle is the recognition of "structural keywords",
69//! recursively within a parsing stack, via the p`NetdocParseable`] trait.
70//!
71//! This allows the parser to be derived. We have type-driven parsing
72//! of whole Documents, Items, and their Arguments and Objects,
73//! including of their multiplicity.
74//!
75//! The different keyword handling means we can't use most of the existing lexer,
76//! and need new item parsing API:
77//!
78//! * [`NetdocParseable`] trait.
79//! * [`KeywordRef`] type.
80//! * [`ItemStream`], [`UnparsedItem`], [`ArgumentStream`], [`UnparsedObject`].
81//!
82//! The different error handling means we have our own error types.
83//! (The crate's existing parse errors have information that we don't track,
84//! and is also a portmanteau error for parsing, writing, and other functions.)
85//!
86//! Document signing is handled in a more abstract way.
87//!
88//! Some old netdoc constructs are not supported.
89//! For example, the obsolete `opt` prefix on safe-to-ignore Items.
90//! The parser may make different decisions about netdocs with anomalous item ordering.
91
92#[doc(hidden)]
93#[macro_use]
94pub mod internal_prelude;
95
96#[macro_use]
97mod structural;
98
99#[macro_use]
100mod derive;
101
102mod error;
103mod impls;
104pub mod keyword;
105mod lex;
106mod lines;
107pub mod multiplicity;
108mod signatures;
109mod traits;
110
111pub mod poc;
112
113#[cfg(test)]
114mod test;
115
116use internal_prelude::*;
117
118pub use error::{ErrorProblem, ParseError, VerifyFailed};
119pub use impls::raw_data_object;
120pub use impls::times::NdaSystemTimeDeprecatedSyntax;
121pub use keyword::KeywordRef;
122pub use lex::{ArgumentStream, ItemStream, NoFurtherArguments, UnparsedItem, UnparsedObject};
123pub use lines::{Lines, Peeked, StrExt};
124pub use signatures::{
125 SignatureHashInputs, SignatureItemParseable, check_validity_time, sig_hash_methods,
126};
127pub use structural::{StopAt, StopPredicate};
128pub use traits::{
129 ItemArgumentParseable, ItemObjectParseable, ItemValueParseable, NetdocParseable,
130 NetdocParseableFields,
131};
132
133#[doc(hidden)]
134pub use derive::netdoc_parseable_derive_debug;
135
136//---------- parser ----------
137
138/// Common code for `parse_netdoc` and `parse_netdoc_multiple`
139///
140/// Creates the `ItemStream`, calls `parse_completely`, and handles errors.
141fn parse_internal<T, D: NetdocParseable>(
142 input: &str,
143 file: &str,
144 parse_completely: impl FnOnce(&mut ItemStream) -> Result<T, ErrorProblem>,
145) -> Result<T, ParseError> {
146 let mut items = ItemStream::new(input)?;
147 parse_completely(&mut items).map_err(|problem| ParseError {
148 problem,
149 doctype: D::doctype_for_error(),
150 file: file.to_owned(),
151 lno: items.lno_for_error(),
152 })
153}
154
155/// Parse a network document - **toplevel entrypoint**
156pub fn parse_netdoc<D: NetdocParseable>(input: &str, file: &str) -> Result<D, ParseError> {
157 parse_internal::<_, D>(input, file, |items| {
158 let doc = D::from_items(items, StopAt(false))?;
159 if let Some(_kw) = items.peek_keyword()? {
160 return Err(EP::MultipleDocuments);
161 }
162 Ok(doc)
163 })
164}
165
166/// Parse a network document - **toplevel entrypoint**
167pub fn parse_netdoc_multiple<D: NetdocParseable>(
168 input: &str,
169 file: &str,
170) -> Result<Vec<D>, ParseError> {
171 parse_internal::<_, D>(input, file, |items| {
172 let mut docs = vec![];
173 while items.peek_keyword()?.is_some() {
174 let doc = D::from_items(items, StopAt(false))?;
175 docs.push(doc);
176 }
177 Ok(docs)
178 })
179}