tor_netdoc/
parse2.rs

1//! New netdoc parsing arrangements, with `derive`
2//!
3//! # Parsing principles
4//!
5//! A parseable network document is a type implementing [`NetdocParseable`].
6//! usually via the
7//! [`NetdocParseable` derive=deftly macro`](crate::derive_deftly_template_NetdocParseable).
8//!
9//! A document type is responsible for recognising its own heading item.
10//! Its parser will also be told other of structural items that it should not consume.
11//! The structural lines can then be used to pass control to the appropriate parser.
12//!
13//! A "structural item" is a netdoc item that is defines the structure of the document.
14//! This includes the intro items for whole documents,
15//! the items that introduce document sections
16//! (which we model by treating the section as a sub-document)
17//! and signature items (which introduce the signatures at the end of the document,
18//! and after which no non-signature items may appear).
19//!
20//! # Ordering
21//!
22//! We don't parse things into a sorted order.
23//! Sorting will be done on output.
24// TODO we don't implement deriving output yet.
25//!
26//! # Types, and signature handling
27//!
28//! Most top-level network documents are signed somehow.
29//! In this case there are three types:
30//!
31//!   * **`FooSigned`**: a signed `Foo`, with its signatures, not yet verified.
32//!     Implements [`NetdocSigned`],
33//!     typically by invoking the
34//!     [`NetdocSigned` derive macro](crate::derive_deftly_template_NetdocSigned)
35//!     on `Foo`.
36//!
37//!     Type-specific methods are provided for verification,
38//!     to obtain a `Foo`.
39//!
40//!   * **`Foo`**: the body data for the document.
41//!     This doesn't contain any signatures.
42//!     Having one of these to play with means signatures have already been validated.
43//!     Implement `NetdocParseable`, via
44//!     [derive](crate::derive_deftly_template_NetdocParseable).
45//!
46//!   * **`FooSignatures`**: the signatures for a `Foo`.
47//!     Implement `NetdocParseable`, via
48//!     [derive](crate::derive_deftly_template_NetdocParseable),
49//!     with `#[deftly(netdoc(signatures))]`.
50//!
51//! # Naming conventions
52//!
53//!   * `DocumentName`: important types,
54//!     including network documents or sub-documents,
55//!     eg `NetworkStatsuMd` and `RouterVote`,
56//!     and types that are generally useful.
57//!   * `NddDoucmnetSection`: sections and sub-documents
58//!     that the user won't normally need to name.
59//!   * `NdiItemValue`: parsed value for a network document Item.
60//!     eg `NdiVoteStatus` representing the whole of the RHS of a `vote-status` Item.
61//!     Often not needed since `ItemValueParseable` is implemented for suitable tuples.
62//!   * `NdaArgumentValue`: parsed value for a single argument;
63//!     eg `NdaVoteStatus` representing the `vote` or `status` argument.
64//!
65//! # Relationship to tor_netdoc::parse
66//!
67//! This is a completely new parsing approach, based on different principles.
68//! The key principle is the recognition of "structural keywords",
69//! recursively within a parsing stack, via the p`NetdocParseable`] trait.
70//!
71//! This allows the parser to be derived.  We have type-driven parsing
72//! of whole Documents, Items, and their Arguments and Objects,
73//! including of their multiplicity.
74//!
75//! The different keyword handling means we can't use most of the existing lexer,
76//! and need new item parsing API:
77//!
78//!  * [`NetdocParseable`] trait.
79//!  * [`KeywordRef`] type.
80//!  * [`ItemStream`], [`UnparsedItem`], [`ArgumentStream`], [`UnparsedObject`].
81//!
82//! The different error handling means we have our own error types.
83//! (The crate's existing parse errors have information that we don't track,
84//! and is also a portmanteau error for parsing, writing, and other functions.)
85//!
86//! Document signing is handled in a more abstract way.
87//!
88//! Some old netdoc constructs are not supported.
89//! For example, the obsolete `opt` prefix on safe-to-ignore Items.
90//! The parser may make different decisions about netdocs with anomalous item ordering.
91
92#[doc(hidden)]
93#[macro_use]
94pub mod internal_prelude;
95
96#[macro_use]
97mod structural;
98
99#[macro_use]
100mod derive;
101
102mod error;
103mod impls;
104pub mod keyword;
105mod lex;
106mod lines;
107pub mod multiplicity;
108mod signatures;
109mod traits;
110
111pub mod poc;
112
113#[cfg(test)]
114mod test;
115
116use internal_prelude::*;
117
118pub use error::{ErrorProblem, ParseError, VerifyFailed};
119pub use impls::raw_data_object;
120pub use impls::times::NdaSystemTimeDeprecatedSyntax;
121pub use keyword::KeywordRef;
122pub use lex::{ArgumentStream, ItemStream, NoFurtherArguments, UnparsedItem, UnparsedObject};
123pub use lines::{Lines, Peeked, StrExt};
124pub use signatures::{
125    SignatureHashInputs, SignatureItemParseable, check_validity_time, sig_hash_methods,
126};
127pub use structural::{StopAt, StopPredicate};
128pub use traits::{
129    ItemArgumentParseable, ItemObjectParseable, ItemValueParseable, NetdocParseable,
130    NetdocParseableFields,
131};
132
133#[doc(hidden)]
134pub use derive::netdoc_parseable_derive_debug;
135
136//---------- parser ----------
137
138/// Common code for `parse_netdoc` and `parse_netdoc_multiple`
139///
140/// Creates the `ItemStream`, calls `parse_completely`, and handles errors.
141fn parse_internal<T, D: NetdocParseable>(
142    input: &str,
143    file: &str,
144    parse_completely: impl FnOnce(&mut ItemStream) -> Result<T, ErrorProblem>,
145) -> Result<T, ParseError> {
146    let mut items = ItemStream::new(input)?;
147    parse_completely(&mut items).map_err(|problem| ParseError {
148        problem,
149        doctype: D::doctype_for_error(),
150        file: file.to_owned(),
151        lno: items.lno_for_error(),
152    })
153}
154
155/// Parse a network document - **toplevel entrypoint**
156pub fn parse_netdoc<D: NetdocParseable>(input: &str, file: &str) -> Result<D, ParseError> {
157    parse_internal::<_, D>(input, file, |items| {
158        let doc = D::from_items(items, StopAt(false))?;
159        if let Some(_kw) = items.peek_keyword()? {
160            return Err(EP::MultipleDocuments);
161        }
162        Ok(doc)
163    })
164}
165
166/// Parse a network document - **toplevel entrypoint**
167pub fn parse_netdoc_multiple<D: NetdocParseable>(
168    input: &str,
169    file: &str,
170) -> Result<Vec<D>, ParseError> {
171    parse_internal::<_, D>(input, file, |items| {
172        let mut docs = vec![];
173        while items.peek_keyword()?.is_some() {
174            let doc = D::from_items(items, StopAt(false))?;
175            docs.push(doc);
176        }
177        Ok(docs)
178    })
179}