Trait Bincode

pub unsafe trait Bincode: Serialize + DeserializeOwned {
    const HEADER: [u8; 24];
    const VERSION: u8;
    const OS_DIRECTORY: Dir;
    const PROJECT_DIRECTORY: &'static str;
    const SUB_DIRECTORIES: &'static str;
    const FILE: &'static str;
    const FILE_EXT: &'static str;
    const FILE_NAME: &'static str;
    const FILE_NAME_GZIP: &'static str;
    const FILE_NAME_TMP: &'static str;
    const FILE_NAME_GZIP_TMP: &'static str;

    // Provided methods
    fn from_bytes(bytes: &[u8]) -> Result<Self, Error> { ... }
    fn to_bytes(&self) -> Result<Vec<u8>, Error> { ... }
    fn from_reader<R>(reader: &mut R) -> Result<Self, Error>
       where R: Read { ... }
    fn to_writer<W>(&self, writer: &mut W) -> Result<(), Error>
       where W: Write { ... }
    fn file_header_to_string() -> Result<String, Error> { ... }
    fn full_header() -> [u8; 25] { ... }
    fn file_version() -> Result<u8, Error> { ... }
    fn from_versions(
        versions_and_constructors: &'static [(u8, fn() -> Result<Self, Error>)],
    ) -> Result<(u8, Self), Error> { ... }
    fn to_writeable_fmt(&self) -> Result<Vec<u8>, Error> { ... }
    fn into_writeable_fmt(self) -> Result<Vec<u8>, Error> { ... }
    fn into_bytes(self) -> Result<Vec<u8>, Error> { ... }
    fn read_to_bytes() -> Result<Vec<u8>, Error> { ... }
    fn read_to_bytes_gzip() -> Result<Vec<u8>, Error> { ... }
    fn exists_gzip() -> Result<Metadata, Error> { ... }
    fn from_file() -> Result<Self, Error> { ... }
    fn from_file_gzip() -> Result<Self, Error> { ... }
    unsafe fn from_file_memmap() -> Result<Self, Error> { ... }
    unsafe fn from_file_gzip_memmap() -> Result<Self, Error> { ... }
    fn from_path<P: AsRef<Path>>(path: P) -> Result<Self, Error> { ... }
    unsafe fn from_path_memmap<P: AsRef<Path>>(path: P) -> Result<Self, Error> { ... }
    fn save(&self) -> Result<Metadata, Error> { ... }
    unsafe fn save_memmap(&self) -> Result<Metadata, Error> { ... }
    fn save_gzip(&self) -> Result<Metadata, Error> { ... }
    unsafe fn save_gzip_memmap(&self) -> Result<Metadata, Error> { ... }
    fn save_atomic(&self) -> Result<Metadata, Error> { ... }
    fn save_atomic_gzip(&self) -> Result<Metadata, Error> { ... }
    unsafe fn save_atomic_memmap(&self) -> Result<Metadata, Error> { ... }
    unsafe fn save_atomic_gzip_memmap(&self) -> Result<Metadata, Error> { ... }
    fn rm_atomic() -> Result<Metadata, Error> { ... }
    fn rm_atomic_gzip() -> Result<Metadata, Error> { ... }
    fn rm_tmp() -> Result<(), Error> { ... }
    fn absolute_path_gzip() -> Result<PathBuf, Error> { ... }
    fn file_size_gzip() -> Result<Metadata, Error> { ... }
    fn file_bytes(start: usize, end: usize) -> Result<Vec<u8>, Error> { ... }
    fn file_bytes_memmap(start: usize, end: usize) -> Result<Vec<u8>, Error> { ... }
    fn mkdir() -> Result<PathBuf, Error> { ... }
    fn exists() -> Result<Metadata, Error> { ... }
    fn file_size() -> Result<Metadata, Error> { ... }
    fn base_path() -> Result<PathBuf, Error> { ... }
    fn absolute_path() -> Result<PathBuf, Error> { ... }
    fn rm() -> Result<Metadata, Error> { ... }
    fn rm_base() -> Result<Metadata, Error> { ... }
    fn rm_sub() -> Result<Metadata, Error> { ... }
    fn rm_project() -> Result<Metadata, Error> { ... }
    fn sub_dir_size() -> Result<Metadata, Error> { ... }
    fn project_dir_size() -> Result<Metadata, Error> { ... }
    fn project_dir_path() -> Result<PathBuf, Error> { ... }
    fn sub_dir_parent_path() -> Result<PathBuf, Error> { ... }
}

Available on crate feature bincode only.

Bincode (binary) file format

Encoding

The encoding option used is:

bincode::DefaultOptions::new().with_varint_encoding();

File extension is .bin.

Safety

When manually implementing, you are promising that the PATH’s manually specified are correct.

Required Associated Constants

const HEADER: [u8; 24]

A custom 24-byte length identifying header for your binary file.

This is combined with Self::VERSION to prefix your file with 25 bytes.

Note: Self::save_gzip() applies compression AFTER, meaning the entire file must be decompressed to get these headers.

const VERSION: u8

What the version byte will be (0-255).

const OS_DIRECTORY: Dir

Which OS directory it will be saved in.

const PROJECT_DIRECTORY: &'static str

What the main project directory will be.

const SUB_DIRECTORIES: &'static str

Optional sub directories in between the project directory and file.

const FILE: &'static str

What the raw file name will be (no extension).

const FILE_EXT: &'static str

What the file extension will be.

const FILE_NAME: &'static str

What the full filename + extension will be.

const FILE_NAME_GZIP: &'static str

What the gzip variant of the filename will be.

const FILE_NAME_TMP: &'static str

What the tmp variant of the filename will be.

const FILE_NAME_GZIP_TMP: &'static str

What the gzip + tmp variant of the filename will be.

Provided Methods

fn from_bytes(bytes: &[u8]) -> Result<Self, Error>

Create a Self from bytes.

fn to_bytes(&self) -> Result<Vec<u8>, Error>

Convert Self to bytes.

fn from_reader<R>(reader: &mut R) -> Result<Self, Error>

where R: Read,

Create Self directly from reader R.

fn to_writer<W>(&self, writer: &mut W) -> Result<(), Error>

where W: Write,

Convert Self to directly to the writer W without intermediate bytes.

fn file_header_to_string() -> Result<String, Error>

Read the associated file and attempt to convert the first 24 bytes to a String.

This is useful if your Self::HEADER should be bytes representing a UTF-8 String.

fn full_header() -> [u8; 25]

Return the 25 bytes header bytes.

First 24 bytes are the Self::HEADER bytes.

Last byte is Self::VERSION.

fn file_version() -> Result<u8, Error>

Reads the first 24 bytes of the associated file and matches it against Self::HEADER.

If the bytes match, the next byte may be be our Self::VERSION and is returned.

Note

This only works on a non-compressed file.

fn from_versions( versions_and_constructors: &'static [(u8, fn() -> Result<Self, Error>)], ) -> Result<(u8, Self), Error>

This is the function that ties the versioning system together.

It takes a variable static array of (VERSION, Struct::constructor) tuples, attempting to deserialize them starting from index 0.

AKA, you give a list of versions and your choice of disk constructors for various versions of the same-ish struct.

An example:

disk::bincode!(Data0, Dir::Data, "Data", "", "data", [255_u8; 24], 0); // <- note: version 0.
struct Data0 {
    data: Vec<u8>,
}

// This converts a `Data0` into a `Data5`
impl Data0 {
    fn to_data5() -> Result<Data5, anyhow::Error> {
        match Self::from_file() {
            Ok(s)  => Ok(Data1 { data: s.data, ..Default::default() }),
            Err(e) => Err(e),
        }
    }
}

/* ... data1, data2, data3, data4 ... */

disk::bincode!(Data1, Dir::Data, "Data", "", "data", [255_u8; 24], 5); // <- note: version 5.
struct Data5 {
    data: Vec<u8>,
    more_data: Vec<u8>,
}

The Data0::to_data5() can be used as the constructor for this function.

Now, if we’d like to deserialize Data5, but fallback if the file detected is an older version, we can write this:

let data = Data5::from_versions(&[
    (5, Data5::from_file), // Your choice of function here.
    (4, Data4::to_data5),  // These as well.
    (3, Data3::to_data5),
    (2, Data2::to_data5),
    (1, Data1::to_data5),
    (0, Data0::to_data5),
]).unwrap();

This will go top-to-bottom starting at 5, ending at 0, checking if the version matches, then attempting deserialization.

The returned Ok(u8, Self) contains the version that successfully matched and the resulting (converted) deserialized data.

The output data is always Self, so the fn() constructors you input are responsible for converting between the various types.

fn to_writeable_fmt(&self) -> Result<Vec<u8>, Error>

Turn Self into bytes that can be written to disk.

fn into_writeable_fmt(self) -> Result<Vec<u8>, Error>

Consume Self into bytes that can be written to disk.

fn into_bytes(self) -> Result<Vec<u8>, Error>

Consume Self into bytes

fn read_to_bytes() -> Result<Vec<u8>, Error>

Read the file directly as bytes.

fn read_to_bytes_gzip() -> Result<Vec<u8>, Error>

Read the file directly as bytes, and attempt gzip decompression.

This assumes the file is suffixed with .gz, for example:

config.json    // What `.read_to_bytes()` will look for
config.json.gz // What `.read_to_bytes_gzip()` will look for

fn exists_gzip() -> Result<Metadata, Error>

Same as Self::exists() but checks if the gzip file exists.

• Self::exists() checks for file.toml.
• Self::exists_gzip() checks for file.toml.gz.

fn from_file() -> Result<Self, Error>

Read the file as bytes and deserialize into Self.

Internally, this functions calls the most optimal function for the format.

fn from_file_gzip() -> Result<Self, Error>

Read the file as bytes, decompress with gzip and deserialize into Self.

unsafe fn from_file_memmap() -> Result<Self, Error>

Same as Self::from_file but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

unsafe fn from_file_gzip_memmap() -> Result<Self, Error>

Same as Self::from_file_gzip but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

fn from_path<P: AsRef<Path>>(path: P) -> Result<Self, Error>

Reads an arbitrary PATH, and attempts to deserialize into Self.

Internally, this functions calls the most optimal function for the format.

unsafe fn from_path_memmap<P: AsRef<Path>>(path: P) -> Result<Self, Error>

Same as Self::from_path but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

fn save(&self) -> Result<Metadata, Error>

Try saving as a file.

This will return the amount of bytes saved and the PathBuf on success.

Calling this will automatically create the directories leading up to the file.

unsafe fn save_memmap(&self) -> Result<Metadata, Error>

Same as Self::save but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

fn save_gzip(&self) -> Result<Metadata, Error>

Try saving as a compressed file using gzip.

On success, this will return:

• The amount of compressed bytes saved to disk
• The PathBuf of the file

This will suffix the file with .gz, for example:

config.json    // Normal file name with `.save()`
config.json.gz // File name when using `.save_gzip()`

Calling this will automatically create the directories leading up to the file.

unsafe fn save_gzip_memmap(&self) -> Result<Metadata, Error>

Same as Self::save_gzip but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

fn save_atomic(&self) -> Result<Metadata, Error>

Try saving to a TEMPORARY file first, then renaming it to the associated file.

This lowers the chance for data corruption on interrupt.

The temporary file is removed if the rename fails.

The temporary file name is: file_name + extension + .tmp, for example:

config.toml     // <- Real file
config.toml.tmp // <- Temporary version

Already existing .tmp files will be overwritten.

This will return the amount of bytes saved and the PathBuf on success.

Calling this will automatically create the directories leading up to the file.

fn save_atomic_gzip(&self) -> Result<Metadata, Error>

Combines Self::save_gzip() and Self::save_atomic().

unsafe fn save_atomic_memmap(&self) -> Result<Metadata, Error>

Same as Self::save_atomic() but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

unsafe fn save_atomic_gzip_memmap(&self) -> Result<Metadata, Error>

Same as Self::save_atomic_gzip() but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

fn rm_atomic() -> Result<Metadata, Error>

Rename the associated file before attempting to delete it.

This lowers the chance for data corruption on interrupt.

On success, this returns:

• The amount of bytes removed
• The PathBuf that was removed

The temporary file name is: file_name + extension + .tmp, for example:

config.toml     // <- Real file
config.toml.tmp // <- Temporary version

Already existing .tmp files will be overwritten.

fn rm_atomic_gzip() -> Result<Metadata, Error>

Same as Self::rm_atomic() but looks for the .gz extension.

fn rm_tmp() -> Result<(), Error>

Try deleting any leftover .tmp files from Self::save_atomic() or Self::save_atomic_gzip()

This will return success if the files don’t exist or if deleted.

It will return failure if files existed but could not be deleted or if any other error occurs.

fn absolute_path_gzip() -> Result<PathBuf, Error>

The absolute PATH of the file associated with this struct WITH the .gz extension.

fn file_size_gzip() -> Result<Metadata, Error>

Returns the gzip file size in bytes and it’s PathBuf.

fn file_bytes(start: usize, end: usize) -> Result<Vec<u8>, Error>

Available on 32-bit only.

Reads a range of bytes of the associated file of Self.

Errors

If start is greater than end, this returns error.

fn file_bytes_memmap(start: usize, end: usize) -> Result<Vec<u8>, Error>

Available on 32-bit only.

Same as Self::file_bytes but with memmap2.

Safety

You must understand all the invariants that memmap comes with.

More details here.

fn mkdir() -> Result<PathBuf, Error>

Create the directories leading up-to the file.

Returns the PathBuf created on success.

This is not necessary when using any variant of Self::save() as the directories are created implicitly.

fn exists() -> Result<Metadata, Error>

Check if the file exists.

On success, this returns:

• The file size in bytes
• The PathBuf it’s located at

fn file_size() -> Result<Metadata, Error>

Returns the file size in bytes and it’s PathBuf.

fn base_path() -> Result<PathBuf, Error>

Returns the full base path associated with this struct (PATH leading up to the file).

In contrast to Self::sub_dir_parent_path, this returns all sub-directories, e.g: my/sub/dirs would return /.../my/sub/dirs

This includes Self::PROJECT_DIRECTORY, Self::SUB_DIRECTORIES and excludes Self::FILE_NAME.

fn absolute_path() -> Result<PathBuf, Error>

Returns the absolute PATH of the file associated with this struct.

This includes Self::PROJECT_DIRECTORY, Self::SUB_DIRECTORIES and Self::FILE_NAME.

fn rm() -> Result<Metadata, Error>

Try deleting the file.

This will return success if the file doesn’t exist or if deleted.

It will return failure if the file existed but could not be deleted or if any other error occurs.

On success, this returns:

• The amount of bytes removed
• The PathBuf that was removed

fn rm_base() -> Result<Metadata, Error>

Recursively remove this file’s basepath.

This deletes all directories starting from the last Self::SUB_DIRECTORIES.

AKA, the directory returned from Self::base_path.

For example:

disk::toml!(State, disk::Dir::Data, "MyProject", "some/sub/dirs", "state");

Everything starting from ~/.local/share/myproject/some/sub/dirs gets removed recursively.

This is akin to running:

rm -rf ~/.local/share/myproject/some/sub/dirs

This function calls std::fs::remove_dir_all, which does not follow symlinks.

On success, this returns:

• The amount of bytes removed
• The PathBuf that was removed

fn rm_sub() -> Result<Metadata, Error>

Recursively remove this file’s sub-directories.

This deletes all directories starting from the parent Self::SUB_DIRECTORIES. For example:

disk::toml!(State, disk::Dir::Data, "MyProject", "some/sub/dirs", "state");

Everything starting from ~/.local/share/myproject/some gets removed recursively.

This is akin to running:

rm -rf ~/.local/share/myproject/some

This function calls std::fs::remove_dir_all, which does not follow symlinks.

On success, this returns:

• The amount of bytes removed
• The PathBuf that was removed

fn rm_project() -> Result<Metadata, Error>

Recursively remove this file’s project directory.

This deletes all directories starting from Self::PROJECT_DIRECTORY. For example:

disk::toml!(State, disk::Dir::Data, "MyProject", "sub_dir", "state");

This project’s file would be located at ~/.local/share/myproject. This is the PATH that gets removed recursively.

This is akin to running:

rm -rf ~/.local/share/myproject

The input to all disk macros are sanity checked. The worst you can do with this function is delete your project’s directory.

This function calls std::fs::remove_dir_all, which does not follow symlinks.

On success, this returns:

• The amount of bytes removed
• The PathBuf that was removed

fn sub_dir_size() -> Result<Metadata, Error>

Returns the file’s parent sub-directory size in bytes and it’s PathBuf.

This errors if the PATH does not exist.

This starts from the first Self::SUB_DIRECTORIES, and does not include the Self::PROJECT_DIRECTORY.

fn project_dir_size() -> Result<Metadata, Error>

Returns the file’s project directory size in bytes (Self::PROJECT_DIRECTORY) and it’s PathBuf.

This errors if the PATH does not exist.

fn project_dir_path() -> Result<PathBuf, Error>

Return the full parent project directory associated with this struct.

This is the PATH leading up to Self::PROJECT_DIRECTORY.

fn sub_dir_parent_path() -> Result<PathBuf, Error>

Returns the top-level parent sub-directory associated with this struct.

This only returns the top level sub-directory, so if multiple are defined, only the first will be returned, e.g: my/sub/dirs would return /.../my

If no sub-directory is defined, this will return the PATH leading up to Self::PROJECT_DIRECTORY.

Dyn Compatibility

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety", so this trait is not object safe.

Implementors