Struct VariedLengthDelimitedCodec

pub struct VariedLengthDelimitedCodec { /* private fields */ }

Length delimited bytes-to-frame encoding/decoding scheme that uses first n bits to represent header bytes’ length. (1 <= n <= 3)

The implementation of the codec is similar to tokio_util’s implementation of length_delimited; however, the differentiation is that tokio_util’s implementation uses a fixed type (u32, unless specified otherwise using a builder) for frame length, whereas this crate’s implementation uses varying types (u8 - u32) depending on the length of the frame.

Limitations

• This codec allows limited customizations (with_max_frame_length), as it is a minimal implementation; if users would like to customize things such as length_adjustment, length_encoding_endianness, etc., they should use tokio_util::codec::length_delimited::VariedLengthDelimitedCodec instead.
• This codec can encode and decode frames up to 2^29 bytes (536,870,912 bytes = 536GB), and if users would like to send frames that are greater than 536GB, they should break it into multiple byte objects before encoding with this codec.

Getting started

Implementation details

Length byte’s first n bits are reserved to indicate whether or not there exists (n + 1)th length byte (1 <= n <= 3).

Example 1

Say our frame payload is 100 bytes; it would be wasteful to use 2 or 3 bytes to represent the number 100, when it could just be done in a single byte. In this case, our frame will look like below:

+---- header bytes in binary ----+--------------------------------+
|           0 1100100            |          frame payload         |
+--------------------------------+--------------------------------+

First bit 0 indicates that there is no subsequent byte to represent the frame length, meaning the frame header is a single byte. Subsequent 7 bits makes number value 100 in binary.

Since the first bit is used as a flag, a single header byte can only represent a value up to 127; greater values would need an extra byte to represent it.

Example 2

In this example, let’s use a frame with payload of 500 bytes. In this case, it will require at least 2 bytes to represent the value:

+---- header bytes in binary ----+--------------------------------+
|       10 000001 11110100       |          frame payload         |
+--------------------------------+--------------------------------+

Firt bit 1 indicates that there exists a second byte to represent the frame length, and the second bit 0 indicates that there doesn’t exist a third byte in the frame header, meaning that the header frame is represented with 2 bytes. Subsequent 14 bits represent the number value 500.

Since the first 2 bits are used as flags, the 2 header bytes can only represent the byte length up to 16,383. Values greater than this would need another byte.

Example 3

This time, say our frame payload is made of 2,000,000 bytes. In this case, 2 bytes (14 bits more specifically) is not able to capture this value; in this case, it would need at least 3 bytes.

+------ header bytes in binary ------+--------------------------------+
|     110 11110 10000100 10000000    |          frame payload         |
+--------------------------+------------------------------------------+

First two bits 11 indicate that there are at least 3 bytes in the frame header; subsequent bit 0 indicates that there doesn’t exist a 4th byte in the header bytes. And the subsequent 21 bits are used to represent the value 2,000,000. Since 3 bits are used as flags, 3 bytes will only be able to represent values up to 2,097,151 bytes (~2MB).

But what if we wanted to send a frame that was 50GB?

Example 4

Finally, say we want to send a frame payload of length 50GB = 50,000,000 bytes. In this case, 3 bytes is not even close to enough to represent this value. So, we need 4 bytes for it.

+---------- header bytes in binary -----------+--------------------------------+
|     111 00010 11111010 11110000 10000000    |          frame payload         |
+--------------------------+---------------------------------------------------+

First three bits 111 indicate that there are 4 bytes used in the frame header; in this case, we do not add another flag bit because 4 bytes (32 - 3 = 29 bits) are enough to represent values up to 536,870,911 bytes (536GB), which should be more than enough for most cases.

Implementations

impl VariedLengthDelimitedCodec

pub fn new() -> Self

Creates a new VariedLengthDelimitedCodec.

pub fn with_max_frame_length(max_frame_length: usize) -> Self

Returns Self with custom max_frame_length

Defaults to MAX_FRAME_LENGTH = 8 * 1024 * 1024

pub fn into_framed<T>(self, inner: T) -> Framed<T, VariedLengthDelimitedCodec>

where T: AsyncRead + AsyncWrite,

Create a configured length delimited Framed

Example

use tokio::io::{AsyncRead, AsyncWrite};
use tsyncp::util::frame_codec::VariedLengthDelimitedCodec;

fn write_frame<T: AsyncRead + AsyncWrite>(io: T) {
    let framed = VariedLengthDelimitedCodec::new().into_framed(io);
}

Trait Implementations

impl Clone for VariedLengthDelimitedCodec

fn clone(&self) -> VariedLengthDelimitedCodec

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for VariedLengthDelimitedCodec

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Decoder for VariedLengthDelimitedCodec

type Item = BytesMut

The type of decoded frames.

type Error = CodecError

The type of unrecoverable frame decoding errors.

fn decode( &mut self, src: &mut BytesMut, ) -> Result<Option<Self::Item>, CodecError>

Attempts to decode a frame from the provided buffer of bytes.

fn decode_eof( &mut self, buf: &mut BytesMut, ) -> Result<Option<Self::Item>, Self::Error>

A default method available to be called when there are no more bytes available to be read from the underlying I/O.

fn framed<T>(self, io: T) -> Framed<T, Self>

where T: AsyncRead + AsyncWrite, Self: Sized,

Provides a Stream and Sink interface for reading and writing to this Io object, using Decode and Encode to read and write the raw data.

impl Default for VariedLengthDelimitedCodec

fn default() -> Self

Returns the “default value” for a type.

impl Encoder<Bytes> for VariedLengthDelimitedCodec

type Error = CodecError

The type of encoding errors.

fn encode(&mut self, data: Bytes, dst: &mut BytesMut) -> Result<(), CodecError>

Encodes a frame into the buffer provided.

impl Ord for VariedLengthDelimitedCodec

fn cmp(&self, other: &VariedLengthDelimitedCodec) -> Ordering

This method returns an Ordering between self and other.

fn max(self, other: Self) -> Self

where Self: Sized,

Compares and returns the maximum of two values.

fn min(self, other: Self) -> Self

where Self: Sized,

Compares and returns the minimum of two values.

fn clamp(self, min: Self, max: Self) -> Self

where Self: Sized,

Restrict a value to a certain interval.

impl PartialEq for VariedLengthDelimitedCodec

fn eq(&self, other: &VariedLengthDelimitedCodec) -> bool

Tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

Tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl PartialOrd for VariedLengthDelimitedCodec

fn partial_cmp(&self, other: &VariedLengthDelimitedCodec) -> Option<Ordering>

This method returns an ordering between self and other values if one exists.

fn lt(&self, other: &Rhs) -> bool

Tests less than (for self and other) and is used by the < operator.

fn le(&self, other: &Rhs) -> bool

Tests less than or equal to (for self and other) and is used by the <= operator.

fn gt(&self, other: &Rhs) -> bool

Tests greater than (for self and other) and is used by the > operator.

fn ge(&self, other: &Rhs) -> bool

Tests greater than or equal to (for self and other) and is used by the >= operator.

impl Eq for VariedLengthDelimitedCodec

impl StructuralPartialEq for VariedLengthDelimitedCodec