bita 0.3.0

Differential file synchronization over http(s).
docs.rs failed to build bita-0.3.0
Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.
Visit the last successful build: bita-0.2.1

bita

Build Status

Tool for fast and low bandwidth file synchronization over http.


The file to synchronize can be any file where data is expected to change partially or completely between updates. Any local file that might contain data of the source file may be used as seed while cloning.

On a system with an A/B partition setup bita can be used to update the B partition while using the A partition as seed. The result written to the B partition will be an exact clone of the remote file but the only data fetched from remote is the data which actually differ between the A partition and the remote file.


On compression the input is scanned for chunk boundaries using a rolling hash. With the default setting a suitable boundary should be found every ~64 KiB. Each found chunk is assigned a strong hash using blake2. If the chunk is unique it will be compressed and inserted into the final archive. A dictionary which describes all chunks by their strong hash and the order for how to rebuild the input file is also attached to the archive.

On clone the dictionary is first fetched from the remote archive. Then bita scans all the given seed files for chunks which are present in the dictionary. Any matching chunk found in a seed will be inserted into the output file. When all seeds has been consumed the chunks still missing is fetched from the remote archive, then unpacked and inserted into the output file.

Each chunk of data is verified by the strong hash before written to the output.


Installing from crates.io

olle@home:~$ cargo install bita

Building from source

olle@home:~$ cargo build

Building in release mode

olle@home:~$ cargo build --release

Running tests

olle@home:~$ cargo test

Example usage

Compress

Create a compressed archive file.cba from stdin stream:

olle@host:~$ gunzip -c file.gz | bita compress --compression-level 9 file.cba

Create an compressed archive file.ext4.cba from file file.ext4:

olle@host:~$ bita compress file.ext4 file.ext4.cba

Clone

Clone file at https://host/new.tar.cba using seed another_old.tar and stdin (-):

olle@device:~$ gunzip -c old.tar.gz | bita clone --seed another_old.tar --seed - https://host/new.tar.cba new.tar

Clone using block device /dev/mmcblk0p1 as seed and /dev/mmcblk0p2 as target:

olle@device:~$ bita clone --seed /dev/mmcblk0p1 https://host/file.ext4.cba /dev/mmcblk0p2

Similar Tools