aol 0.1.4 - Docs.rs

Generic purpose append only log implementation.

Introducation

When developing infrastructure softwares, write-ahead log or append-only log plays an important role, and people re-implement same funcationalities multiple times, but actually, the core for append-only log is just atomic append, append_batch, replay, and rewrite.

This crate provides generic purpose append-only log implementation, there are two kinds of implementations based on std::fs::File and memory map.

aol::fs::AppendLog:

Generic append-only log implementation based on std::fs::File.
- It is good for:
  - The encoded entry size is smaller than 64 bytes.
  - Manifest file.
  - Write is not too frequently.
- Pros:
  - It is growable, do not require pre-allocated.
  - Support automatically rewrite.
  - No holes in the file.
aol::memmap::AppendLog:

Generic append-only log implementation based on memmap.
- It is good for:
  - Any append-only log, if you do not care about pre-allocating the file and you know you data will never larger than the pre-allocating size.
- Pros:
  - Support automatic rewrite.
  - No holes in the file.
  - As this implementation is backed by an ARENA, no allocation required for both read and write.
  - Fast read and write performance, backed by memory map, no extra I/O required.

File Structure

+----------------------+--------------------------+-----------------------+
| magic text (4 bytes) | external magic (2 bytes) | magic (2 bytes)       |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+
| op (1 bit)           | custom flag (7 bits)     | len (4 bytes)         | data (N bytes)        | checksum (8 bytes)    |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+
| op (1 bit)           | custom flag (7 bits)     | len (4 bytes)         | data (N bytes)        | checksum (8 bytes)    |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+
| ...                  | ...                      | ...                   | ...                   | ...                   |
+----------------------+--------------------------+-----------------------+-----------------------+-----------------------+

Installation

[dependencies]
aol = "0.1"

License

aol is under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE, LICENSE-MIT for details.