Expand description
§Bipack codec
The set of tools to effectively encode and decode bipack values. It is internationally minimalistic to be used wit Divan smart-contracts where number of instructions could be important.
-
bipack_source::BipackSource is used to decode values, there is implementation bipack_source::SliceSource that parses binary slice. The trait only needs byte-read method for the implementation.
-
bipack_sink::BipackSink trait that is also implemented for
Vec<u8>allows to encode values into the bipack format. It is the same simple to implement it for any else binary data source.
§Utilities
-
to simplify encoding of unsigned ints the bipack_sink::IntoU64 trait is used with implementation for usual u* types.
-
tools::to_dump utility function converts binary data into human-readable dump as in old good times (address, bytes, ASCII characters).
-
tools::StringBuilder minimalistic growing strings builder.
§About Bipack format
This is a binary format created around the idea of bit-effectiveness and not disclosing inner data structure. Unlike many known binary and text formats, like JSON, BSON, BOSS, and many others, it does not includes field names into packed binaries.
It also uses rationally-packed variable length format very effective for unsigned integers of
various sizes. This implementation supports sizes for u8, u16, u32 and u64. It is capable of
holding longer values too but for big numbers the fixed size encoding is mostly more effective.
This rational encoding format is called smartint and is internally used everywhere when one
need to pack unsigned number, unless the fixed size is important.
§Varint encoding
Smart variable-length long encoding tools, async. It gives byte-size gain from 64 bits numbers, so it is very useful when encoding big numbers or at least very bui long values. In other cases bipack_sink::BipackSink::put_unsigned works faster, and extra bits it uses does not play
| Bytes sz | varint bits | smartint bits |
|---|---|---|
| 1 | 7 | 6 |
| 2 | 14 | 14 |
| 3 | 21 | 22 |
| 4 | 28 | 29 |
| 5 | 35 | 36 |
| 6+ | 7*N | 7*N+1 |
| 9 | 63 | 64 |
| 10 | 64 | — |
In other words, except for very small numbers smartint gives 1 data bit gain for the same packed byte size. For example, full size 64 bits number with smartint takes one byte less (9 bytes vs. 10 in Varint).
So, except for values in range 32..63 it gives same or better byte size effectiveness
than Varint. In particular:
The effect of it could be interpreted as:
| number values | size |
|---|---|
| 0..31 | same |
| 32..63 | worse 1 byte |
| 64..1048573 | same |
| 1048576..2097151 | 1 byte better |
| 2097152..134217727 | same |
| 134217728..268435456 | 1 byte better |
etc.
§Encoding format
Enncoded data could be 1 or more bytes in length. Data are packed as follows:
| byte offset | bits range | field |
|---|---|---|
| 0 | 0..1 | type |
| 0 | 2..7 | v0 |
| 1 | 0..7 | v1 (when used) |
| 2 | 0..7 | v2 (when used) |
Then depending on the type field:
| type | encoded |
|---|---|
| 0 | v0 is the result 0..64 (or -32..32) |
| 1 | v0 ## v1 are the result, 14 bits |
| 2 | v0 ## v1 ## v2 are the result, 22bits |
| 3 | v0, ## v1 ## v2 ## (varint encoded rest) |
Where ## means bits concatenation. The bits are interpreted as BIG ENDIAN,
for example 24573 will be encoded to EA FF 02
Modules§
- bipack
- bipack_
sink - bipack_
source - buffer_
sink - contrail
- Contrails are byte arrays protected by a short crc8 checksum which is enough to protect against human typing error in most cases.
- crc
- de
- error
- fixint
- Fixed Size Integers
- ser
- tools
Macros§
- bipack
- Pack all arguments according to their type, using variable-length
encoding for integers and default encoding for binaries and string,
and return
Vec<u8>with packed result.
Traits§
- Deserialize
- A data structure that can be deserialized from any data format supported by Serde.
- Serialize
- A data structure that can be serialized into any data format supported by Serde.