Module no_proto::format[−][src]

NoProto Data Format Documentaion

NoProto buffers are built out of pointers and data.

They are designed to hold a variable amount of data that is parsed based on a schema provided by the client.

Pointers

Pointers contain one or more addresses depending on the pointer type. The addresses will point to data or other pointers.

There is only one address size, u32. Addresses are always stored in big endian format and addresses are always zero based from the beginning of the buffer. In other words, address 23 always means 23 bytes from the beginning of the buffer.

Pointer Kind	u32 size (bytes)
Standard	4
Map Item	12
List Item	10

The first byte of every buffer is a flag to show if the buffer is packed with it's schema or not. Packed buffers start with a "1", non packed buffers start with a "0".

The second byte of every buffer is a version/size number to be used by later version of NoProto. The next 2 bytes of every buffer is the root pointer, these bytes should contain the address of the root object in the buffer.

Most of the time these bytes will point to the data immediately following them, but it's possible to clear the root object causing these bytes to be zero, or to update the root data which would cause this address to update to something else.

For example, here is a buffer with u32 address size that contains the string hello, it's schema is just {type: "string"}.

[0,0,   0, 0, 0, 6,    0, 0, 0, 5, 104, 101, 108, 108, 111]
[     root pointer, string length,   h,   e,   l,   l,   o]

It should be noted that a schema is required to parse a buffer, otherwise you don't know the difference between pointers, data and what data types beyond the root.

Let's look at the different pointer types you will encounter in a buffer.

Standard Pointer

This is used for any scalar or collection data types. The standard pointer is just a single u32.

Map Item Pointer

Used by items in a map object. Contains the following:

| address of data | next map item pointer address | address of bytes for this key |
|        u32      |               u32             |            u32                |

Map collections represent a linked list of these pointers. There should only be map item pointers for items in the map that have data.

The last map item pointer in a map should have a zero in the next item address for no further map items.

The key is always stored as a variable sequence of bytes provided by the client. If you go to the address of the key you should find a length byte (u8) followed by a sequence of bytes that represents the key.

List Item Pointer

Used by items in a list object. Contains the following:

| address of data | next list item pointer address | item index |
|      u32        |             u32                |    u16     |

Unlike tables and maps, the order of the list items point to eachother should be kept so that the index is the correct sequence.

You can have gaps in the sequence, but the index should always be in order. So if you have 3 item pointers with indexes 2, 8 and 20 they should point to each other in this order: 2 -> 8 -> 20. This doesn't mean they have to be in order in the buffer, they just have to point to eachother in order.

There should be list item pointers only for indexes that have data in the list.

The last list item pointer in a list should have a zero in the next item address for no further list items.

Data

Data is stored in a specific format based on the data type in the schema. The schema should determine how bytes at a sepcific address are treated.

When a pointer's address "points" to a location in the buffer, you should be able to parse the bytes at the designated location following the rules for the given data type below.

Most data types have a known size ahead of time, some don't, and some have a size dependent on the schema.

Struct (Collection)

The struct data type stores one or more vtables for field values. Each vtable is 20 bytes and contains:

4 address (u32) pointers for the field values
a trailing address(u32) of the next vtable (should be zero if no more vtables)

Each vtable can address up to 4 fields, so if there are 30 fields in a schema there may be as many as 8 vtables in the buffer: 30 / 4 = 7.5

Vtables are created as needed, For example if there are 100 fields in the schema but the client only ever sets values to the first 4 fields there will only ever be 1 vtable in the buffer.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
    "type": "struct",
    "fields": [
        ["age",  {"type": "u8"}]
    ]
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&["age"], 20u8)?;

assert_eq!(vec![0, 0, 0, 0, 0, 6, 0, 0, 0, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 20], new_buffer.finish().bytes());
 
// [0, 0, 0, 0, 0, 6, 0, 0, 0, 26, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,    20]
// [        root ptr,                                                      vtable,  data]

List (Collection)

The list type stores two addresses (u32), one to the first ListItem pointer (head) and one to the last ListItem pointer (tail).

If there is only one list item pointer in the list, the head and tail addresses should be identical.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
    "type": "list",
    "of": {"type": "u8"}
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&["4"], 20u8)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 0, 0, 0, 14, 0, 0, 0, 14, 0, 0, 0, 24, 0, 0, 0, 0, 0, 4, 20], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6,  0, 0, 0, 14, 0, 0, 0, 14, 0, 0, 0, 24, 0, 0, 0, 0, 0, 4,    20]
// [         root ptr,         head,        tail,                 list item ptr,  data]

Map (Collection)

The map type stores a single address (u32) to the first MapItem pointer.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
    "type": "map",
    "value": {"type": "u8"}
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&["age"], 20u8)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 0, 0, 0, 22, 0, 0, 0, 0, 0, 0, 0, 18, 3, 97, 103, 101, 20], new_buffer.finish().bytes());
 
// [0,0,  0, 0, 0, 6,   0, 0, 0, 22, 0, 0, 0, 0, 0, 0, 0, 18,  3, 97, 103, 101,     20]
// [        root ptr,                           map item ptr,      a,   g,   e,   data]

Tuple (Collection)

Tuples are stored by packing the types next to eachother.

Before each type is a byte that is either 1 or 0. It should be zero if the value has not been set, otherwise it should be one.

If a child type is flexible in size, a pointer address is put in the tuple. If a child type is fixed in size, it is placed inline in the tuple.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "tuple",
   "values": [
       {"type": "u8"},
       {"type": "string"},
       {"type": "u32"}
   ]
}"#)?;
 

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&["0"], 20u8)?;
new_buffer.set(&["1"], "hello")?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 1, 20, 1, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 5, 104, 101, 108, 108, 111], new_buffer.finish().bytes());
 
// [0, 0, 0, 0, 0, 6,   1, 20, 1, 0, 0, 0, 18, 0, 0, 0, 0, 0, 0, 0, 0, 5, 104, 101, 108, 108, 111]
// [        root ptr,      u8,        str ptr,           u32,               h,   e,   l,   l,   o]

int8, int16, int32, int64 (Scalar)

Signed integers should be converted to unsigned values, then saved in big endian format.

The size of the integer should determine how many bytes are used. For example, i8 is 1 byte, i16 is 2 bytes, etc.

For example, an i8 of value -20 should be converted to 108, then saved as 108.

When it's requested by the client, it should be converted back to signed before being passed to the client.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "i32"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], -2023830i32)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 127, 225, 30, 106], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6, 127, 225, 30, 106]
// [         root ptr,              data]

uint8, uint16, uint32, uint64 (Scalar)

Unsigned integers should be converted to big endian format, then saved to the buffer.

The size of the integer should determine how many bytes are used. For example, u8 is 1 byte, u32 is 2 bytes, etc.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "u32"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], 28378u32)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 0, 0, 110, 218], new_buffer.finish().bytes());
 
// [0,0,  0, 0, 0, 6, 0, 0, 110, 218]
// [        root ptr,           data]

float, double (Scalar)

Floating point vales should be converted to big endian format, then saved to the buffer.

The size of the floating point value should determine how many bytes are used. float is f32 (4 bytes) and double is f64 (8 bytes)

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "f32"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], 2.389988f32)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 64, 24, 245, 144], new_buffer.finish().bytes());
 
// [0,0,  0, 0, 0, 6, 64, 24, 245, 144]
// [        root ptr,             data]

option (Scalar)

Option values are stored as a single u8 value. The value should represent the zero based location in the choice set.

For example if the schema has choices: ["red", "blue", "yellow"] and the user selects yellow, this value should be 2.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::option::NP_Enum;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "option",
   "choices": ["blue", "orange", "red"]
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], NP_Enum::new("red"))?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 2], new_buffer.finish().bytes());
 
// [0,0,  0, 0, 0, 6,      2]
// [        root ptr,   data]

bool (Scalar)

A single u8 byte. 1 for true, 0 for false.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "bool"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], true)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 1], new_buffer.finish().bytes());
 
// [0,0,  0, 0, 0, 6,      1]
// [        root ptr,   data]

decimal (Scalar)

Stored the same as an i64 value (including converting to unsigned format described above).

The i64 number should be devided by 10 ^ exp to get the true value. The exp value is provided in the schema.

For example, if you pull a 293 i64 value from the buffer and the exp value in the schema is 2, the value is actually 293 / 100 or 2.93.

You should avoid converting the number to floating point values except for display purposes. Study the source code for the NP_Dec type to see how to preserve the internal i64 value correctly.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::dec::NP_Dec;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "decimal",
   "exp": 2
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], NP_Dec::new(200, 0))?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 128, 0, 0, 0, 0, 0, 78, 32], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6, 128, 0, 0, 0, 0, 0, 78, 32]
// [         root ptr,                       data]

geo4, geo8, geo16 (Scalar)

Each geo size uses two signed integers right next to eachother in the buffer. i16/i16 for geo4, i32/i32 for geo8 and i64/i64 for geo16

The two signed integers are converted to unsigned values before being saved into big endian format.

Depending on the size, the floating point value of each geographic coordinate is multiplied by a specific value before being saved as an integer.

Size	Bytes	Factor
4	i16 \| i16	100
8	i32 \| i32	10000000
16	i64 \| i64	1000000000

For example, if a user provides these coordinates: 41.303921, -81.901693

To save into buffer:

geo4:
1 - Multiply by 100: (4130.3921, -8190.1693)
2 - Make i16 (4130, -8190)
3 - Save/convert as unsigned in big endian format

geo8:
1 - Multiply by 10000000: (413039210, -819016930)
2 - Make i32 (413039210, -819016930)
3 - Save/convert as unsigned in big endian format

geo16: ....

This process is reversed when the client requests the geo data from the buffer.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::geo::NP_Geo;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "geo8"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], NP_Geo::new(8, 41.303921, -81.901693))?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 152, 158, 122, 106, 79, 46, 203, 30], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6, 152, 158, 122, 106, 79, 46, 203, 30]
// [         root ptr,           latitude,       longitude]

ulid, uuid (Scalar)

Saved as 16 bytes following the respective formats for each data type.

ULIDs store the date in the first 6 bytes, then the random bytes in the last 10.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::uuid::NP_UUID;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "uuid"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
let uuid = NP_UUID::generate(32);
new_buffer.set(&[], &uuid)?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 202, 230, 170, 176, 127, 103, 66, 13, 89, 65, 221, 4, 153, 160, 117, 252], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6, 202, 230, 170, 176, 127, 103, 66, 13, 89, 65, 221, 4, 153, 160, 117, 252]
// [         root ptr,                              UUID                                       ]

bytes, string (Scalar)

If there is a size property in the schema, store the provided data and pad the remainder of the space with zeros (for bytes) or 32 (for strings).

If the provided data is too large, truncate it.

For example, if the user provideds a single byte [22] and the size is 3, this should be in the buffer:

[22, 0, 0]

If there is no fixed size in the schema, store a size (u32) followed by the actual data.

If it's a string, the data should be utf-8 encoded when it's saved into the buffer and utf-8 decoded when it's retrieved.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "string"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], "hello, world!")?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 0, 0, 0, 13, 104, 101, 108, 108, 111, 44, 32, 119, 111, 114, 108, 100, 33], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6,   0, 0, 0, 13, 104, 101, 108, 108, 111, 44, 32, 119, 111, 114, 108, 100, 33]
// [         root ptr,        length,   h,   e,   l,   l,   o,  ,,   ,   w,   o,   r,   l,   d,  !]

date (Scalar)

This is stored the same as a uint64 value, should be unix timestamp in milliseconds.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::date::NP_Date;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "date"
}"#)?;

let mut new_buffer = factory.new_buffer(None);
new_buffer.set(&[], NP_Date::new(1598490738507))?;
assert_eq!(vec![0, 0, 0, 0, 0, 6, 0, 0, 1, 116, 45, 120, 255, 75], new_buffer.finish().bytes());
 
// [0,0,   0, 0, 0, 6, 0, 0, 1, 116, 45, 120, 255, 75]
// [         root ptr,           timestamp           ]

NoProto Schema Format Documentation

NoProto JSON schemas are compiled into a byte array as part of the parsing process.

The compiled byte array is a significantly more compact and efficient way to store the schema. It also takes almost no time to parse a byte schema, where parsing a JSON schema can be a comparitively expensive operation.

You can use the runtime to parse JSON schemas into byte array schemas at any time, and the JSON/byte array schemas can be used interchangebly.

The byte array schema store default values and all other supported schema properties.

Schema data is stored in a recursive format, each nested schema contains at least one byte that describes the data type. The single data type byte is usually but not always followed by schema data specific to that data type. The document below describes all of the data types and their specifics.

int8, int16, int32, int64, uint8, uint16, uint32, uint64, float, double (Scalar)

Integer values store the data type followed by wether there is a default value or not, followed optionally by the default value

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "i32",
   "default": 56
}"#)?;

assert_eq!(&[6, 1, 0, 0, 0, 56], factory.export_schema_bytes());
 
// [       6,           1,      0, 0, 0, 56]
// [i32 type, has default,    default value]
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "i32"
}"#)?;

assert_eq!(&[6, 0], factory.export_schema_bytes());
 
// [       6,           0]
// [i32 type,  no default]

option (Scalar)

Option types will store the list of options and the index of the default value, if there is one.

The second byte is 0 if there is no default, otherwise it contains the default index + 1.

The third byte contains a u8 that is the number of options available.

The remaining bytes go on a loop for each option, with each loop containing 1 u8 byte at the begining describing the length of the string option, followed by the string value itself.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::option::NP_Enum;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "option",
   "choices": ["blue", "orange", "red"],
   "default": "red"
}"#)?;

assert_eq!(&[20, 3, 3, 4, 98, 108, 117, 101, 6, 111, 114, 97, 110, 103, 101, 3, 114, 101, 100], factory.export_schema_bytes());
 
// [       20,                        3,            3, 4, 98, 108, 117, 101, 6, 111, 114, 97, 110, 103, 101, 3, 114, 101, 100]
// [data type, 1 based index of default, # of options,     b,   l,   u,   e,      o,   r,  a,   n,   g,   e,      r,   e,   d]  
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "option",
   "choices": ["blue", "orange", "red"]
}"#)?;

assert_eq!(&[20, 0, 3, 4, 98, 108, 117, 101, 6, 111, 114, 97, 110, 103, 101, 3, 114, 101, 100], factory.export_schema_bytes());
 
// [       20,          0,             3, 4, 98, 108, 117, 101, 6, 111, 114, 97, 110, 103, 101, 3, 114, 101, 100]
// [data type, no default,  # of options,     b,   l,   u,   e,      o,   r,  a,   n,   g,   e,      r,   e,   d]

bool (Scalar)

The second byte of a bool schema is used to store the default value.

If there is no default value, the second byte is 0.
If the default is true, the second byte is 1.
If the default is false, the second byte is 2.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::option::NP_Enum;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "bool",
   "default": true
}"#)?;

assert_eq!(&[15, 1], factory.export_schema_bytes());
 
// [       15,               1]
// [data type, default is true]  
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "bool",
   "default": false
}"#)?;

assert_eq!(&[15, 2], factory.export_schema_bytes());
 
// [       15,                2]
// [data type, default is false]  
 
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "bool"
}"#)?;

assert_eq!(&[15, 0], factory.export_schema_bytes());
 
// [       15,          0]
// [data type, no default]

decimal (Scalar)

Decimal stores the expontent in the second byte.

The third byte is 0 if there is no default value, otherwise it is 1.

If there is a default value, multiply the default value by (10^exp) and convert it into an i64, then save it in the bytes following the default flag byte.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::dec::NP_Dec;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "decimal",
   "exp": 2
}"#)?;

assert_eq!(&[14, 2, 0], factory.export_schema_bytes());
 
// [       14,         2,                0]
// [data type, expontent, no default value]
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "decimal",
   "exp": 2,
   "default": 521.32
}"#)?;

assert_eq!(&[14, 2, 1, 0, 0, 0, 0, 0, 0, 203, 164], factory.export_schema_bytes());
 
// [       14,         2,                 1, 0, 0, 0, 0, 0, 0, 203, 164]
// [data type, expontent, has default value,              default value]

geo4, geo8, geo16 (Scalar)

Geo stores the size of the data type in the second byte. The third byte is 0 if there is no default, and 1 if there is a default. The remaining bytes are the default value (if there is one) parsed in the specific size designated in the second byte.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::geo::NP_Geo;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "geo8"
}"#)?;

assert_eq!(&[16, 8, 0], factory.export_schema_bytes());
 
// [       16,                 8,                0]
// [data type, geo size (4/8/16), no default value]
 
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "geo8",
   "default": {"lat": 29.2, "lng": -19.2}
}"#)?;

assert_eq!(&[16, 8, 1, 145, 103, 145, 0, 116, 142, 80, 0], factory.export_schema_bytes());
 
// [       16,                 8,                 1, 145, 103, 145, 0, 116, 142, 80, 0]
// [data type, geo size (4/8/16), has default value,             geo8 value (lat/lng) ]

ulid, uuid (Scalar)

UUID and ULID do not have default options, so this data type is very simple.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::uuid::NP_UUID;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "uuid"
}"#)?;

assert_eq!(&[17], factory.export_schema_bytes());
 
// [       17]
// [data type]

bytes, string (Scalar)

The second and third bytes are a u32 of the fixed size. If there is no fixed size, these two bytes are zero.

Thhe length of the default value follows as a u32, if there is no default value the u32 is zero. If there is a default value, it follows the length bytes.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "string"
}"#)?;

assert_eq!(&[2, 0, 0, 0, 0, 0, 0, 0], factory.export_schema_bytes());
 
// [        2,                   0,       0, 0, 0, 0,                 0, 0]
// [data type, uppercase/lowercase, fixed size (u32),  default size (u16) ]

let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "string",
   "size": 20
}"#)?;

assert_eq!(&[2, 0, 0, 0, 0, 20, 0, 0], factory.export_schema_bytes());
 
// [        2,                   0,      0, 0, 0, 20,                 0, 0]
// [data type, uppercase/lowercase, fixed size (u32),  default size (u16) ]
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "string",
   "size": 20,
   "default": "hello"
}"#)?;

assert_eq!(&[2, 0, 0, 0, 0, 20, 0, 6, 104, 101, 108, 108, 111], factory.export_schema_bytes());
 
// [        2,                   0,       0, 0, 0, 20,                0, 6, 104, 101, 108, 108, 111]
// [data type, uppercase/lowercase,  fixed size (u32),  default size (u16),   h,   e,   l,   l,   o]

date (Scalar)

The second byte is a 1 if there is a default value, 0 otherwise.

If there is a default value it follows the second byte.

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
use no_proto::pointer::date::NP_Date;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "date"
}"#)?;

assert_eq!(&[19, 0], factory.export_schema_bytes());
 
// [       19,             0]
// [data type, default flag ]
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "date",
   "default": 1604862252
}"#)?;

assert_eq!(&[19, 1, 0, 0, 0, 0, 95, 168, 65, 44], factory.export_schema_bytes());
 
// [       19,            1, 0, 0, 0, 0, 95, 168, 65, 44]
// [data type, default flag,        default value       ]

Collection Schemas

Collection based schemas nest schemas in a way that allows any type to be the child of any collection, including other collections.

Struct (collection)

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
    "type": "struct",
    "fields": [
        ["age",  {"type": "u8"}],
        ["name", {"type": "string"}]
    ]
}"#)?;


assert_eq!(&[21, 2, 3, 97, 103, 101, 0, 2, 8, 0, 4, 110, 97, 109, 101, 0, 8, 2, 0, 0, 0, 0, 0, 0, 0], factory.export_schema_bytes());
 
// [21, 2, 3, 97, 103, 101, 0, 2, 8, 0, 4, 110, 97, 109, 101, 0, 8, 2, 0, 0, 0, 0, 0, 0, 0]
// [data type,  # of fields,     a,   g,   e,  field schema size (u32),   field schema,      n,  a,   m,   e,   field schema size (u32),     field schema  ]

List (Collection)

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
    "type": "list",
    "of": {"type": "u8"}
}"#)?;

assert_eq!(&[23, 8, 0], factory.export_schema_bytes());
 
// [       23,        8, 0]
// [data type, "of" schema]

Map (Collection)

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
    "type": "map",
    "value": {"type": "u8"}
}"#)?;
 
assert_eq!(&[22, 8, 0], factory.export_schema_bytes());
 
// [       22,         8, 0]
// [data type, value schema]

Tuple (Collection)

use no_proto::error::NP_Error;
use no_proto::NP_Factory;
 
let factory: NP_Factory = NP_Factory::new_json(r#"{
   "type": "tuple",
   "values": [
       {"type": "u8"},
       {"type": "string"}
   ]
}"#)?;

assert_eq!(&[24, 0, 2, 0, 2, 8, 0, 0, 8, 2, 0, 0, 0, 0, 0, 0, 0], factory.export_schema_bytes());
 
// [       24,       0,          2,               0, 2,    8, 0,              0, 8,  2, 0, 0, 0, 0, 0, 0, 0]
// [data type, sorted, length (u8),  schema size (u16),  schema, schema size (u16),         schema         ]

Module no_proto::format[−][src]

NoProto Data Format Documentaion

Pointers

Standard Pointer

Map Item Pointer

List Item Pointer

Data

Struct (Collection)

List (Collection)

Map (Collection)

Tuple (Collection)

int8, int16, int32, int64 (Scalar)

uint8, uint16, uint32, uint64 (Scalar)

float, double (Scalar)

option (Scalar)

bool (Scalar)

decimal (Scalar)

geo4, geo8, geo16 (Scalar)

ulid, uuid (Scalar)

bytes, string (Scalar)

date (Scalar)

NoProto Schema Format Documentation

int8, int16, int32, int64, uint8, uint16, uint32, uint64, float, double (Scalar)

option (Scalar)

bool (Scalar)

decimal (Scalar)

geo4, geo8, geo16 (Scalar)

ulid, uuid (Scalar)

bytes, string (Scalar)

date (Scalar)

Collection Schemas

Struct (collection)

List (Collection)

Map (Collection)

Tuple (Collection)

Portal (Collection/Scalar)

Union (Collection/Scalar)