Crate self_encryption

A file content self_encryptor.

This library provides convergent encryption on file-based data and produces a DataMap type and several chunks of encrypted data. Each chunk is up to 1MB in size and has an index and a name. This name is the SHA3-256 hash of the content, which allows the chunks to be self-validating. If size and hash checks are utilised, a high degree of certainty in the validity of the data can be expected.

Project GitHub page.

Examples

A working implementation can be found in the “examples” folder of this project.

use self_encryption::{encrypt, test_helpers::random_bytes};

#[tokio::main]
async fn main() {
    let file_size = 10_000_000;
    let bytes = random_bytes(file_size);

    if let Ok((_data_map, _encrypted_chunks)) = encrypt(bytes) {
        // .. then persist the `encrypted_chunks`.
        // Remember to keep `data_map` somewhere safe..!
    }
}

Storage of the Vec<EncryptedChunk> or DataMap is outwith the scope of this library and must be implemented by the user.

Re-exports

• pub use bytes;
• pub use xor_name;

Structs

• ChunkInfo
  This is - in effect - a partial decryption key for an encrypted chunk of data.
• DataMap
  Holds the information that is required to recover the content of the encrypted file. This is held as a vector of ChunkInfo, i.e. a list of the file’s chunk hashes. Only files larger than 3072 bytes (3 * MIN_CHUNK_SIZE) can be self-encrypted. Smaller files will have to be batched together.
• EncryptedChunk
  The actual encrypted content of the chunk
• MAX_CHUNK_SIZE
  The maximum size (before compression) of an individual chunk of a file, defaulting as 1MiB.
• StreamSelfDecryptor
  The streaming decryptor to carry out the decryption on fly, chunk by chunk.
• StreamSelfEncryptor
  The streaming encryptor to carry out the encryption on fly, chunk by chunk.
• XorName
  A 256-bit number, viewed as a point in XOR space.

Enums

• Error
  Errors which can arise during self_encryption or -decryption.

Constants

• COMPRESSION_QUALITY
  Controls the compression-speed vs compression-density tradeoffs. The higher the quality, the slower the compression. Range is 0 to 11.
• MIN_CHUNK_SIZE
  The minimum size (before compression) of an individual chunk of a file, defined as 1B.
• MIN_ENCRYPTABLE_BYTES
  The minimum size (before compression) of data to be self-encrypted, defined as 3B.

Functions

• decrypt
  Decrypts data using chunks retrieved from any storage backend via the provided retrieval function.
• decrypt_chunk
  Decrypt a chunk, given the index of that chunk in the sequence of chunks, and the raw encrypted content.
• decrypt_from_storage
  Decrypts data using chunks retrieved from any storage backend via the provided retrieval function. Writes the decrypted output to the specified file path.
• deserialize
  Deserializes bytes into a data structure using bincode.
• encrypt
  Encrypts a set of bytes and returns the encrypted data together with the data map that is derived from the input data.
• encrypt_from_file
  Read a file from the disk to encrypt, and output the chunks to a given output directory if presents.
• get_root_data_map
  Recursively gets the root data map by decrypting child data maps Takes a chunk retrieval function that handles fetching the encrypted chunks
• get_root_data_map_parallel
  Recursively gets the root data map by decrypting child data maps using parallel chunk retrieval.
• serialize
  Serializes a data structure using bincode.
• shrink_data_map
  Shrinks a data map by recursively encrypting it until the number of chunks is small enough Returns the final data map and all chunks generated during shrinking
• streaming_decrypt_from_storage
  Decrypts data from storage in a streaming fashion using parallel chunk retrieval.
• streaming_encrypt_from_file
  Reads a file in chunks, encrypts them, and stores them using a provided functor. Returns a DataMap.
• verify_chunk
  Verifies and deserializes a chunk by checking its content hash matches the provided name.

Type Aliases

• Result
  Specialisation of std::Result for crate.