Crate disk

source ·
Expand description

Disk: serde + directories + various file formats as Traits.

This crate is for:

  • (De)serializing various file formats (provided by serde)
  • To/from disk locations that follow OS specifications (provided by directories)

All errors returned will be an Error (re-exported anyhow::Error).

Implementing disk

use serde::{Serialize, Deserialize};
use disk::Toml;

#[derive(Serialize,Deserialize)] // <- Your data must implement `serde`.
struct State {
	string: String,
	number: u32,
}
// To make this struct a file, use the following macro:
//
//    |- 1. The file format used will be TOML.
//    |
//    |     |- 2. This is implemented for the struct "State".
//    |     |
//    |     |      |- 3. It will be saved in the OS Data directory.
//    |     |      |
//    |     |      |                 |- 4. The main project directory is called "MyProject".
//    |     |      |                 |
//    |     |      |                 |            |- 6. It won't be in any sub-directories.
//    |     |      |                 |            |
//    |     |      |                 |            |   |- 7. The file name will be "state.toml".
//    v     v      v                 v            v   v
disk::toml!(State, disk::Dir::Data, "MyProject", "", "state");

Now our State struct implements the Toml trait.

The PATH the file would be saved in would be:

OSPATH
WindowsC:\Users\Alice\AppData\Roaming\My_Project\state.toml
macOS/Users/Alice/Library/Application Support/My-Project/state.toml
Linux/home/alice/.local/share/myproject/state.toml

.save() and .from_file()

These two functions are the basic ways to:

  • Save a struct to disk
  • Create a struct from disk
// Create our struct.
let my_state = State { string: "Hello".to_string(), number: 123 };

// Save our `State` as a `Toml` file.
match my_state.save() {
	Ok(_) => println!("We saved to disk"),
	Err(e) => eprintln!("We failed to save to disk"),
}

// Let's create a new `State` by reading the file that we just created:
let from_disk = State::from_file().expect("Failed to read disk");

// These should be the same.
assert!(my_state == from_disk);

.save_atomic()

disk provides an atomic version of .save().

Atomic in this context means, the data will be saved to a TEMPORARY file first, then renamed to the associated file.

This lowers the chance for data corruption on interrupt.

The temporary file is removed if the rename fails.

The temporary file name is: file_name + extension + .tmp, for example:

config.toml     // <- Real file
config.toml.tmp // <- Temporary version

Already existing .tmp files will be overwritten.

.save_gzip() & .from_file_gzip()

disk provides gzip versions of .save() and .from_file().

This saves the file as a compressed file using gzip.

This will suffix the file with .gz, for example:

config.json    // Normal file name with `.save()`
config.json.gz // File name when using `.save_gzip()`

To recover data from this file, you must also use the matching .from_file_gzip() when reading the data.

Sub-Directories

Either a single or multiple sub-directories can be specified with a / delimiter.

\ is also allowed but ONLY if building on Windows.

An empty string "" means NO sub directories.

ⓘ
// Windows ... C:\Users\Alice\AppData\Roaming\My_Project\sub1\sub2\state.toml
disk::toml!(State, Data, "MyProject", r"sub1\sub2", "state");

// macOS ... /Users/Alice/Library/Application Support/My-Project/sub1/sub2/state.json
disk::json!(State, Data, "MyProject", "sub1/sub2", "state");

// Linux ... /home/alice/.local/share/myproject/sub1/sub2/state.yml
disk::yaml!(State, Data, "MyProject", "sub1/sub2", "state");

// NO sub directory:
disk::toml!(State, Data, "MyProject", "", "state");

bincode Header and Version

disk provides a custom header and versioning feature for the binary format, bincode.

The custom header is an arbitrary 24 byte array that is appended to the front of the file.

The version is a single u8 that comes after the header, representing a version from 0-255.

These must be passed to the implementation macro.

Example:

const HEADER: [u8; 24] = [1_u8; 24];
const VERSION: u8 = 5;

// Define.
disk::bincode!(State, disk::Dir::Data, "MyProject", "", "state", HEADER, VERSION);
#[derive(Serialize,Deserialize)]
struct State {
	string: String,
	number: u32,
}

// Save file.
let state = State { string: "Hello".to_string(), number: 123 };
state.save().unwrap();

// Assert the file's header+version on
// disk is correct and extract our version.
let version = State::file_version().unwrap();
assert!(version == State::VERSION);

The header and version make up the first 25 bytes of the file, byte 1..=24 being the header and byte 25 being the version. These bytes are checked upon using any .from_file() variant and will return an error if it does not match your struct’s implementation.

bincode2

disk provides two bincode traits, Bincode & Bincode2.

bincode 2.0.0 (currently not stable) brings big performance improvements.

It also no longer requires serde, having it’s own Encode and Decode traits.

This means your type must implement these as well, e.g:

ⓘ
use bincode::{Encode, Decode};

#[derive(Encode, Decode)]
struct State;

To implement bincode 2.x.x’s new traits, add it to Cargo.toml:

bincode = "2.0.0-rc.3"

and add #[derive(Encode, Decode)] to your types, like you would with serde.

Manually implementing disk

The macros verify and sanity check the input data at compile time, while manual unsafe impl does not, and gives you full control over the data definitions, allowing obvious mistakes like empty PATH’s and mismatching filenames to slip through.

It requires 9 constants to be defined:

unsafe impl disk::Toml for State {
    const OS_DIRECTORY:       disk::Dir    = disk::Dir::Data;
    const PROJECT_DIRECTORY:  &'static str = "MyProject";
    const SUB_DIRECTORIES:    &'static str = "";
    const FILE:               &'static str = "state";
    const FILE_EXT:           &'static str = "toml";
    const FILE_NAME:          &'static str = "state.toml";
    const FILE_NAME_GZIP:     &'static str = "state.gzip";
    const FILE_NAME_TMP:      &'static str = "state.toml.tmp";
    const FILE_NAME_GZIP_TMP: &'static str = "state.toml.gzip.tmp";
}

A dangerous example:

ⓘ
unsafe impl disk::Toml for State {
    const OS_DIRECTORY:       disk::Dir    = disk::Dir::Data;
    const PROJECT_DIRECTORY:  &'static str = "";
    const SUB_DIRECTORIES:    &'static str = "";
    const FILE:               &'static str = "";
    [...]
}

// This deletes `~/.local/share`...!
State::rm_rf();

Feature Flags

No file formats are enabled by default, you must enable them with feature flags.

Enabling the bytesize feature makes Metadata use bytesize for its [Display].

For example, println!("{metadata}") which normally looks like:

312445 bytes @ /my/file/path

will turn into:

312.4 KB @ /my/file/path

Use the full feature flag to enable everything.

File FormatFeature flag to enable
Bincodebincode
Bincode2bincode2
Postcardpostcard
JSONjson
TOMLtoml
YAMLyaml
Picklepickle
MessagePackmessagepack
BSONbson
RONron
Plain Textplain
Empty Fileempty

Macros

Structs

  • Error
    The Error type, a wrapper around a dynamic error type.
  • Metadata
    Metadata collected about a file/directory.

Enums

Traits