1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
//! The [binary-layout](https://crates.io/crates/binary-layout) library allows type-safe, inplace, zero-copy access to structured binary data.
//! You define a custom data layout and give it a slice of binary data, and it will allow you to read and
//! write the fields defined in the layout from the binary data without having to copy any of the data.
//! It's similar to transmuting to/from a `#[repr(packed)]` struct, but [much safer](#why-not-reprpacked).
//!
//! Note that the data does not go through serialization/deserialization or a parsing step.
//! All accessors access the underlying package data directly.
//!
//! # Example
//! ```
//! use binary_layout::prelude::*;
//!
//! // See https://en.wikipedia.org/wiki/Internet_Control_Message_Protocol for ICMP package layout
//! define_layout!(icmp_packet, BigEndian, {
//! packet_type: u8,
//! code: u8,
//! checksum: u16,
//! rest_of_header: [u8; 4],
//! data_section: [u8], // open ended byte array, matches until the end of the package
//! });
//!
//! fn func(packet_data: &mut [u8]) {
//! let mut view = icmp_packet::View::new(packet_data);
//!
//! // read some data
//! let code: u8 = view.code().read();
//! // equivalent: let code: u8 = packet_data[1];
//!
//! // write some data
//! view.checksum_mut().write(10);
//! // equivalent: packet_data[2..4].copy_from_slice(&10u16.to_le_bytes());
//!
//! // access an open ended byte array
//! let data_section: &[u8] = view.data_section().data();
//! // equivalent: let data_section: &[u8] = &packet_data[8..];
//!
//! // and modify it
//! view.data_section_mut().data_mut()[..5].copy_from_slice(&[1, 2, 3, 4, 5]);
//! // equivalent: packet_data[8..13].copy_from_slice(&[1, 2, 3, 4, 5]);
//! }
//! ```
//!
//! # APIs
//! This library offers two alternative APIs:
//! 1. The [Field](binary_layout::Field) API that offers free functions to read/write the data based on an underlying slice of storage (`packet_data` in the example above) holding the packet data. This API does not wrap the underlying slice of storage data, which means you have to pass it in to each accessor.
//! This is not the API used in the example above, see [Field](binary_layout::Field) for an API example.
//! 2. The [FieldView](binary_layout::FieldView) API that wraps a slice of storage data and remembers it in a `View` object, allowing access to the fields without having to pass in the packed data slice each time. This is the API used in the example above. See [FieldView](binary_layout::FieldView) for another example.
//!
//! # What to use this library for?
//! Anything that needs inplace zero-copy access to structured binary data.
//! - Network packets are an obvious example
//! - File system inodes
//! - Structured binary data in files if you want to avoid explicit (de)serialization, possibly in combination with [memmap](https://docs.rs/memmap).
//!
//! # Why use this library?
//! - Inplace, zero-copy, type-safe access to your data.
//! - Data layout is defined in one central place, call sites can't accidentally use wrong field offsets.
//! - Convenient and simple macro DSL to define layouts.
//! - Define a fixed endianness in the layout, ensuring cross platform compatibility.
//! - Fully written in safe Rust, no [std::mem::transmute] or similar shenanigans.
//! - Const generics make sure that all offset calculations happen at compile time and will not have a runtime overhead.
//! - Comprehensive test coverage.
//!
//! # Why not `#[repr(packed)]`?
//! Annotating structs with `#[repr(packed)]` gives some of the features of this crate, namely it lays out the data fields exactly in the order they're specified
//! without padding. But it has serious shortcomings that this library solves.
//! - `#[repr(packed)]` uses the system byte order, which will be different depending on if you're running on a little endian or big endian system. `#[repr(packed)]` is not cross-platform compatible. This library is.
//! - `#[repr(packed)]` [can cause undefined behavior on some CPUs when taking references to unaligned data](https://doc.rust-lang.org/nomicon/other-reprs.html#reprpacked).
//! This library avoids that by not offering any API that takes references to unaligned data. The only data type you can get a reference to is byte arrays, and they only require an alignment of 1 which is trivially always fulfilled.
//!
//! # When not to use this library?
//! - You need dynamic data structures, e.g. a list that can change size. This library only supports static data layouts.
//! - Not all of your data fits into the memory and you need to process streams of data.
//!
//! # Alternatives
//! To the best of my knowledge, there is no other library offering inplace, zero-copy and type-safe access to structured binary data.
//! But if you don't need direct access to your data and are ok with a serialization/deserialization step, then there is a number of amazing libraries out there.
//! - [Nom](https://crates.io/crates/nom) is a great crate for all your parsing needs. It can for example parse binary data and put them in your custom structs.
//! - [Binread](https://crates.io/crates/binread), [Binwrite](https://crates.io/crates/binwrite), [Binrw](https://crates.io/crates/binrw) are great libraries for (de)serializing binary data.
//!
//! # Supported field types
//! ## Primitive integer types
//! - [u8](::std::primitive.u8), [u16](::std::primitive.u16), [u32](::std::primitive.u32), [u64](::std::primitive.u64)
//! - [i8](::std::primitive.i8), [i16](::std::primitive.i16), [i32](::std::primitive.i32), [i64](::std::primitive.i64)
//!
//! For these fields, the [Field](binary_layout::Field) API offers [Field::read](binary_layout::Field::read), [Field::write](binary_layout::Field::write) and the [FieldView](binary_layout::FieldView) API offers [FieldView::read](binary_layout::FieldView::Read) and [FieldView::write](binary_layout::FieldView::write).
//!
//! ## Fixed size byte arrays: `[u8; N]`.
//! For these fields, the [Field](binary_layout::Field) API offers [Field::data](binary_layout::Field::data), [Field::data_mut](binary_layout::Field::data_mut), and the [FieldView](binary_layout::FieldView) API offers [FieldView::data](binary_layout::FieldView::data) and [FieldView::data_mut](binary_layout::FieldView::data_mut).
//!
//! ## Open ended byte arrays: `[u8]`.
//! This field type can only occur as the last field of a layout and will mach the remaining data until the end of the storage.
//! This field has a dynamic size, depending on how large the package data is.
//! For these fields, the [Field](binary_layout::Field) API offers [Field::data](binary_layout::Field::data), [Field::data_mut](binary_layout::Field::data_mut) and the [FieldView](binary_layout::FieldView) API offers [FieldView::data](binary_layout::FieldView::data), [FieldView::data_mut](binary_layout::FieldView::data_mut) and [FieldView::extract](binary_layout::FieldView::extract).
//!
//! # Data types maybe supported in the future
//! These data types aren't supported yet, but they could be added in theory and might be added in future versions.
//! - [bool](::std::primitive.bool) stored as 1 byte
//! - [bool](::std::primitive.bool) stored as 1 bit
//!
//! ## Data types with dynamic length
//! This crate relies on a static layout, it cannot support data types with dynamic length.
//! In theory, types with dynamic length could be supported if they either
//! - are the last field of a layout, an already implemented example of this are open ended byte arrays.
//! - or they may be in the middle of the package but have a maximal size defined and will always reserve storage for their maximal size, even if smaller.
//! This way, the fields after it would still have a constant offset.
//!
//! Both of these, however, would be some effort to implement and it is unclear if that will ever happen (unless somebody opens a PR for it).
//!
//! ## Strings
//! For strings, note that even fixed-size UTF-8 strings take a variable number of characters because of the UTF-8 encoding and that brings all the issues of data types with dynamic length with it.
//! This is why strings aren't supported yet.
//!
//! ## Fixed-size arrays other than `[u8; N]`
//! Say we wanted to have a `[u32; N]` field. The API couldn't just return a zero-copy `&[u32; N]` to the caller because that would use the system byte order (i.e. endianness) which might be different from the byte order defined in the package layout.
//! To make this cross-platform compatible, we'd have to wrap these slices into our own slice type that enforces the correct byte order and return that from the API.
//! This complexity is why it wasn't implemented yet, but feel free to open a PR if you need this.
pub use ;