Crate vach

Expand description

A simple archiving format, designed for storing assets in compact secure containers

vach is an archiving and resource transmission format. It was built to be secure, contained and protected. A big benefit of vach is the fine grained control it grants it’s users, as it allows for per-entry independent configuration. vach also has in-built support for multiple compression schemes (LZ4, Snappy and Brolti), data signing, leaf bitflags, encryption and some degree of archive customization.

Check out the vach spec at spec.txt.

👄 Terminologies

Archive: Any source of data, for example a file or TCP stream, that is a valid vach data source.
Leaf: Any actual data endpoint within an archive, what tar calls archive members, for example footstep1.wav in sounds.vach.
Entry: Some data in the registry section of a vach source on an corresponding leaf. For example, { id: footstep.wav, location: 45, offset: 2345, flags: 0b0000_0000_0000_0000u16 }.

🔫 Cargo Features

archive and builder (default): Turning them off turns off their respective modules. For example a game only needs the archive feature but a tool for packing assets would only need the builder feature.
multithreaded: Pulls rayon as a dependency and adds Send as a trait bound to many generic types. This allows for the auto-parallelization of the Builder::dump(---) function.
compression: Pulls snap, lz4_flex and brotli as dependencies and allows for compression in vach archives.
crypto: Enables encryption and authentication functionality by pulling the ed25519_dalek and aes_gcm crates

🀄 Show me some code dang it!

> Building a basic unsigned `.vach` file

use std::{io::Cursor, fs::File};
use vach::prelude::{Builder, BuilderConfig};

let config = BuilderConfig::default();
let mut builder = Builder::default();

// Use `Builder::add( reader, ID )` to add data to the write queue
// Adds any data that implements `io::Read`
builder.add(File::open("test_data/background.wav")?, "ambient").unwrap();
builder.add(&[12, 23, 34, 45, 56, 67, 78, 90, 69], "ftstep").unwrap();
builder.add(b"Hello, Cassandra!", "hello").unwrap();

// let mut target = File::create("sounds.vach")?;
let mut target = Cursor::new(Vec::new());

// The number of bytes written to the file
let size = builder.dump(&mut target, &config).unwrap();

> Loading resources from an unsigned `.vach` file

use std::fs::File;
use vach::prelude::{Archive, Resource, Flags};

let target = File::open("sounds.vach")?;
let mut archive = Archive::new(target)?;
let resource: Resource = archive.fetch_mut("ambient")?;

// By default all resources are flagged as NOT authenticated
println!("{}", Sound::new(&resource.data)?);
assert!(!resource.authenticated);

let resource = archive.fetch_mut("ftstep")?;

> Build a signed `.vach` file

use std::{io::Cursor, fs::File};
use vach::prelude::{Builder, BuilderConfig, Keypair};
use vach::crypto_utils::gen_keypair;

let keypair: Keypair = gen_keypair();
let config: BuilderConfig = BuilderConfig::default().keypair(keypair);
let mut builder = Builder::default();

// Use `Builder::add( reader, ID )` to add data to the write queue
builder.add(File::open("test_data/background.wav")?, "ambient").unwrap();
builder.add(vec![12, 23, 34, 45, 56, 67, 78], "ftstep").unwrap();
builder.add(b"Hello, Cassandra!" as &[u8], "hello").unwrap();

let mut target = File::create("sounds.vach")?;
builder.dump(&mut target, &config).unwrap();

let mut target = Cursor::new(Vec::new());
builder.dump(&mut target, &config).unwrap();

> Load resources from a signed `.vach` source

// Load public_key
let mut public_key = File::open(PUBLIC_KEY)?;
let mut public_key_bytes: [u8; crate::PUBLIC_KEY_LENGTH];
public_key.read_exact(&mut public_key_bytes)?;

// Build the Loader config
let mut config = ArchiveConfig::default().key(PublicKey::from_bytes(&public_key_bytes)?);

let target = File::open("sounds.vach")?;
let archive = Archive::with_config(target, &config)?;

// Resources are marked as secure (=true) if the signatures match the data
let mut resource = archive.fetch_mut("ambient")?;
println!("{}", Sound::new(&resource.data)?);
assert!(resource.authenticated);

> Serialize and de-serialize a `Keypair`, `SecretKey` and `PublicKey`

As Keypair, SecretKey and PublicKey are reflected from ed25519_dalek, you could refer to their docs to read further about them. These are needed for any cryptography related procedures,

use vach::prelude::{Keypair, SecretKey, PublicKey};
use vach::crypto_utils::gen_keypair;

// Generate keys
let keypair : Keypair  = gen_keypair();
let secret : SecretKey = keypair.secret;
let public : PublicKey = keypair.public;

// Serialize
let public_key_bytes : [u8; vach::PUBLIC_KEY_LENGTH] = public.to_bytes();
let secret_key_bytes : [u8; vach::SECRET_KEY_LENGTH] = secret.to_bytes();
let keypair_bytes : [u8; vach::KEYPAIR_LENGTH]    = keypair.to_bytes();

// Deserialize
let public_key : PublicKey = PublicKey::from_bytes(&public_key_bytes).unwrap();
let secret_key : SecretKey = SecretKey::from_bytes(&secret_key_bytes).unwrap();
let keypair : Keypair   = Keypair::from_bytes(&keypair_bytes).unwrap();

Re-exports

pub use rand;

pub use rayon;

Modules

archivearchive

Loader-based logic and data-structures

builderbuilder

Builder related data structures and logic

cryptocrypto

Import keypairs and signatures from here, mirrors from ed25519_dalek

crypto_utilscrypto

Some utility functions to keep you happy

prelude

Consolidated import for crate logic; This module stores all structs associated with this crate. Constants can be accesses directly with crate::<CONSTANT>

Constants

DEFAULT_MAGIC

The default MAGIC used by vach

KEYPAIR_LENGTH

Size of a keypair: (secret + public)

MAGIC_LENGTH

The standard size of any MAGIC entry in bytes

MAX_ID_LENGTH

Maximum size for any ID

Size of a public key

Size of a secret key

Size of a signature

Current vach spec version. increments by ten with every spec change