1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
//! Runtime for the Protobuf decoders that [`rapidproto-build`] generates.
//!
//! RapidProto only decodes. `rapidproto-build` generates decoders from your schema at build time, and
//! the generated code uses this crate. Add `rapidproto` to your dependencies and `rapidproto-build` to
//! your build dependencies, both pinned to the same version with `=`. The [`example`] module shows a
//! schema, the build script and the generated code.
//!
//! # Arena and stream decoders
//!
//! An arena decoder decodes the whole message. `Msg::decode(input, &arena)` returns a `&Msg` with an
//! accessor for each field. `string` and `bytes` values borrow from `input`, and everything else is
//! stored in the [`Arena`].
//!
//! ```
//! # mod example { include!(concat!(env!("CARGO_MANIFEST_DIR"), "/src/example/weather.rs")); }
//! # #[cfg(feature = "alloc")]
//! # fn main() -> Result<(), rapidproto::DecodeError> {
//! # let input = b"\x0a\x14\x0a\x06Zurich\x18\x01\x22\x08\x00\x00\x48\x41\x00\x00\x50\x41";
//! use example::arena::weather::Report;
//! use rapidproto::Arena;
//!
//! let arena = Arena::new();
//! let report = Report::decode(input, &arena)?;
//! assert_eq!(report.stations()[0].name(), "Zurich");
//! # Ok(())
//! # }
//! # #[cfg(not(feature = "alloc"))]
//! # fn main() {}
//! ```
//!
//! `Msg::decode_owned(input)` returns an [`Owned`], which holds the input and the arena, so you do not
//! have to keep them alive yourself.
//!
//! A stream decoder builds no message. It passes the fields you select to your handlers and skips the
//! rest, so it suits reading a few fields of a large message. `rapidproto-build` generates arena
//! decoders by default, and stream decoders when you turn on [`Config::stream`]. See [`stream`].
//!
//! # Decoding rules
//!
//! A decoder skips unknown fields, which the schema does not declare, and declared fields that arrive
//! with the wrong wire type. It does not check that strings are valid UTF-8. [`ProtoStr::to_str`] does
//! that when you read one.
//!
//! An arena decoder fails if a `required` field is missing, or if a singular message or group field
//! appears twice, where Protobuf would merge the two. A stream decoder checks neither.
//!
//! # Untrusted input
//!
//! An arena decoder fails if sub-messages or groups nest deeper than [`MAX_DECODE_DEPTH`]. Its arena
//! can grow to many times the size of the input, so decode into your own arena and limit it with
//! [`Arena::set_capacity_limit`]. A stream decoder does not allocate, and it decodes a sub-message
//! only when your handler calls `decode` on it.
//!
//! # Features
//!
//! - `alloc`, on by default, lets an [`Arena`] grow on the heap and adds [`Owned`]. Without it, give
//! the arena a buffer with [`Arena::with_buffer`].
//! - `serde` adds `serde::Serialize` impls. See [Serde](#serde).
//!
//! The crate is `no_std`, and without `serde` it has no dependencies.
//!
//! # Debug output
//!
//! Decoded messages implement `Debug`, which prints the fields that are set under their names in the
//! schema, such as `Station { name: "Zurich", unit: Celsius, temperatures: [12.5, 13.0] }`. A field
//! with presence appears when it is set, even at its default value. That covers `optional` fields,
//! message fields and groups, and editions fields with explicit presence. A field without presence,
//! such as a proto3 scalar with no `optional`, appears when its value is not the default. `required`
//! fields always appear. `repeated` and `map` fields appear when they are not empty, and a `oneof`
//! appears as its set member. A field a decode profile drops does not appear, and a `raw` field shows
//! its bytes.
//!
//! An enum prints its `classify()` variant, such as `Celsius`, or `Unknown(7)` for a number the schema
//! does not name. `bytes` print as a byte string, such as `b"\xff"`, and a `string` with invalid UTF-8
//! as `ProtoStr(b"\xff")`. A stream struct prints the length of its encoded message, such as
//! `Station { len: 20 }`.
//!
//! # Serde
//!
//! With `rapidproto-build`'s [`Config::serde`] and this crate's `serde` feature, decoded messages
//! implement `serde::Serialize`. There is no `Deserialize`.
//!
//! A message serializes as a map from field name to value, with the fields `Debug` prints. An enum
//! serializes as its `i32`, and a `raw` field as bytes.
//!
//! `bytes` goes through `serialize_bytes`, so the format decides how it looks. `serde_json` writes an
//! array of numbers. A `string` with invalid UTF-8 serializes as bytes too. A `map` key repeated on the
//! wire appears more than once. A message gives the serializer no length up front, so a format that
//! needs one cannot serialize it.
//!
//! With `serde_json`, a non-finite `float` or `double` becomes `null`, and a `map` fails if a `string`
//! key has invalid UTF-8.
//!
//! [`rapidproto-build`]: https://docs.rs/rapidproto-build/latest/rapidproto_build/
//! [`Config::stream`]: https://docs.rs/rapidproto-build/latest/rapidproto_build/struct.Config.html#method.stream
//! [`Config::serde`]: https://docs.rs/rapidproto-build/latest/rapidproto_build/struct.Config.html#method.serde
// Without `alloc` there is no `Owned`, so its links go to the feature list, which says why.
// The test harness needs `std`, which also lets the no-allocator path be unit-tested with
// `--no-default-features`.
// `ProtoBytes` shows `[u8]`'s methods, and a few would carry a standard library label.
extern crate alloc;
// Rustdoc checks these as `compile_fail` doctests.
// Generated code and benches read the wire through this. Reading it directly is not a user feature.
// The generated code in `example` names this crate `::rapidproto`, as it does in a user's crate.
extern crate self as rapidproto;
// One doc example calls `Arena::new`, so without `alloc` the module stays out of the doctests.
// Generated `Serialize` impls name this, so a crate using them needs no serde dependency of its own.
pub use serde;
pub use Arena;
pub use ;
pub use ;
pub use ;
pub use ProtoBytes;
pub use ProtoStr;
// Generated code, which expands in the user's crate, and tests outside this crate build on these. So
// they stay `pub`, but users never touch them.
pub use ArenaExhausted;
pub use ;
pub use DecodeVec;
pub use decode_owned;
pub use PackedSlice;
/// Re-export of the standard `Box` (only with `alloc`), so a generated `decode_owned` can name
/// `::rapidproto::Box<[u8]>` without the consumer crate spelling an `alloc`/`std` path itself.
/// Internal to the generated glue, not part of the public API.
pub use Box;
// This crate's public items, under one module. Generated stream code re-exports it, so a stream
// macro can name the runtime from a crate that does not depend on it directly. It cannot re-export the
// crate itself: that does not compile in a crate that writes `extern crate rapidproto;`.
// The version check of generated code. Each generated file calls this with its generator's number,
// `kGeneratedAbi` in src/rustgen/generator.cpp. Any other input fails the build, whatever its shape,
// so a later generator may change the call. rustc reports macro errors before name and type errors,
// so a user who mixes versions of rapidproto and rapidproto-build sees this message first, not dozens
// of errors about this crate's internals.
// Bump both numbers when generated code needs an item that earlier versions lack, or when an item it
// uses changes or goes away. This holds for public items as well as hidden ones. A change in
// behavior counts too, even if everything still compiles. The goldens catch a bump on only one side.
// Nothing catches a missing bump, so each release decides whether one is needed.
/// Implement the owning-handle API for a generated message type `$msg`: the [`ArenaDecode`] tag impl
/// on `$msg<'static>` plus the `$msg::decode_owned` convenience over [`decode_owned`]. Generated code
/// invokes this once per message instead of expanding the boilerplate into every file; `$msg`
/// resolves at the call site. Internal to the generated glue, not a consumer-facing macro.
///
/// # Safety of the emitted `unsafe impl`
/// A generated message borrows the input/arena only through shared `&'a` references (scalars,
/// `ProtoStr`, `ProtoBytes`, `PackedSlice`, `MapView`, by-value or `&` sub-messages): it is covariant
/// in `'a`, layout-identical at every lifetime, free of interior mutability over the borrow, and points
/// only into the input, arena-allocated memory or `'static` data, never at the `Arena` value — exactly
/// [`ArenaDecode`]'s contract. Only invoke this for a type with those properties (every `rustgen`
/// message has them).
// Without `alloc` there is no owning handle: generated code invokes the macro on every tier, and it
// expands to nothing here.
/// How deep sub-messages and groups can nest in an arena decoder.
///
/// Deeper nesting fails with [`ErrorKind::DepthLimitExceeded`], which bounds the stack a decode uses.
/// In an unoptimized build stack frames are larger, and a decode this deep can overflow the default
/// 2 MiB stack of a spawned thread.
pub const MAX_DECODE_DEPTH: u32 = 100;