liteboxfs/

lib.rs

//! LiteboxFS is a modern POSIX filesystem in a SQLite database.
//!
//! ```
//! use std::io::{Read, Write, Seek, SeekFrom};
//!
//! use liteboxfs::{Connection, CreateOptions, FileKind, Owner};
//!
//! fn main() -> liteboxfs::Result<()> {
//!     let mut conn = Connection::open_in_memory(&CreateOptions::new())?;
//!
//!     let buf = conn.exec(|fs| {
//!         let mut file = fs.create("example.txt", FileKind::Regular, Owner::current())?;
//!
//!         file.write_all(b"Hello, world!")?;
//!         file.seek(SeekFrom::Start(0))?;
//!
//!         let mut buf = Vec::new();
//!         file.read_to_end(&mut buf)?;
//!
//!         liteboxfs::Result::Ok(buf)
//!     })?;
//!
//!     assert_eq!(buf, b"Hello, world!");
//!
//!     Ok(())
//! }
//! ```
//!
//! # Getting Started
//!
//! A litebox is a SQLite database containing a filesystem.
//!
//! [`Filesystem`] is the main type you will use to interact with LiteboxFS. To access the
//! filesystem, you will first need to open a [`Connection`] and start a transaction.
//!
//! # Roots
//!
//! Unlike a traditional POSIX filesystem, LiteboxFS supports multiple root directories, each
//! identified by a universally unique [`RootId`]. This allows for multiple independent logical
//! filesystems to coexist within the same database and deduplicate data between them. This feature
//! can also be used to create copy-on-write snapshots of a filesystem.
//!
//! In addition to their UUID, roots can optionally have a name, which is a UTF-8 string that is
//! unique within the litebox.
//!
//! When you create a litebox, it will automatically create a default root with name
//! [`DEFAULT_ROOT_NAME`]. The default root always exists; it cannot be deleted.
//!
//! # Metadata
//!
//! A litebox can have [user-defined metadata](crate::UserMetadata) associated with it in the form
//! of key-value pairs. This metadata is stored persistently in the database and is not touched by
//! LiteboxFS, so applications or downstream libraries can use it for any purpose.
//!
//! Each root can also have its own user-defined metadata.
//!
//! # Compression and Deduplication
//!
//! LiteboxFS has two optional features to improve storage efficiency at the cost of performance:
//! [transparent compression](crate::CreateOptions::compression) and [content-defined
//! chunking](crate::CreateOptions::chunking).
//!
//! Files in a litebox are deduplicated at the block level, including between roots. When enabled,
//! content-defined chunking can improve deduplication ratios between files that are similar but
//! not identical.
//!
//! These features are configurable per-litebox, not per-root. Both these features are
//! feature-gated and disabled by default.
//!
//! # Transactions and Concurrency
//!
//! LiteboxFS is _transactional_, with the same fundamental guarantees and caveats as SQLite.
//!
//! This API makes transaction explicit, which means you can use them to maintain invariants in
//! your application, just like you would with a database.
//!
//! Internally, LiteboxFS uses savepoints to ensure that individual filesystem operations like
//! copying a file, deleting a file, writing data to a file, etc. are atomic and consistent, with
//! the transaction providing the isolation and durability.
//!
//! Like SQLite, the concurrency model of LiteboxFS depends on the [journal
//! mode](crate::JournalMode).
//!
//! - In the default mode, readers block writers and writers block readers.
//! - In [WAL mode](crate::JournalMode::Wal), reading and writing can happen concurrently.
//!
//! In both cases, you can only have a single write transaction active at a time, which means that
//! holding a long-running write transaction will block other connections from writing (and
//! possibly reading) to the litebox.
//!
//! Like SQLite, once a write happens, the transaction is implicitly upgraded to a write
//! transaction and stays that way until it is committed or rolled back. LiteboxFS generally tries
//! to avoid writing to the database unless you tell it to; opening and reading a file should not
//! require a write transaction, for example.
//!
//! # Encryption
//!
//! LiteboxFS supports optional encryption via [SQLCipher](https://github.com/sqlcipher/sqlcipher).
//! If SQLCipher support is enabled (via Cargo features), you can specify either a password or an
//! encryption key when creating or opening a litebox. The password/key can be changed later.
//!
//! # FUSE Support
//!
//! On Linux, a litebox can be mounted in the host filesystem via FUSE. This allows for accessing
//! the contents of a litebox root as if it were any other directory on the host system.
//!
//! Mounting a litebox acquires an exclusive lock on the database for the duration of the mount.
//!
//! You can mount a litebox root using [`Connection::mount`] or via the CLI.
//!
//! # Features
//!
//! | Feature | Default | Description |
//! | --- | --- | --- |
//! | `fs` | Yes | Enable features that require interacting with the host filesystem. |
//! | `compression` | No | Enable support for [transparent compression](crate::CreateOptions::compression) of file contents using Zstandard (zstd). |
//! | `chunking` | No | Enable support for [content-defined chunking](crate::CreateOptions::chunking) of file contents using FastCDC. |
//! | `fuse` | No | Enable support for mounting a litebox as a FUSE filesystem. |
//! | `bundled-sqlite` | No | Use a bundled version of SQLite instead of linking to it. |
//! | `sqlcipher` | No | Link to SQLCipher instead of SQLite. This searches for and links against a system-installed crypto library. |
//! | `bundled-sqlcipher` | No | Use a bundled version of SQLCipher instead of linking to it. This searches for and links against a system-installed crypto library. |
//! | `bundled-sqlcipher-vendored-openssl` | No | Use a bundled version of SQLCipher with a vendored version of OpenSSL. |

// You can define lints in `Cargo.toml`, but then it applies to integration tests as well, which
// isn't what we want.
#![forbid(unsafe_code)]
#![warn(missing_docs)]
#![warn(missing_debug_implementations)]
#![warn(clippy::indexing_slicing)]
#![cfg_attr(test, allow(clippy::indexing_slicing))]
#![cfg_attr(coverage_nightly, feature(coverage_attribute))]

mod acl;
mod archive;
mod block;
mod cache;
#[cfg(feature = "chunking")]
mod cdc;
mod chunker;
#[cfg(feature = "compression")]
mod compression;
mod connection;
mod errors;
mod file_metadata;
mod files;
mod filesystem;
#[cfg(all(feature = "fuse", target_os = "linux"))]
mod fuse;
mod hash;
#[cfg(all(feature = "fs", target_family = "unix"))]
mod host;
mod id;
mod lock;
mod merkle;
mod path;
mod settings;
mod sql;
mod state;
mod stats;
mod transaction;
mod tree;
mod user_metadata;
mod util;

#[cfg(test)]
mod testing;
#[cfg(test)]
mod tests;

/// Types for representing file metadata.
pub mod metadata {
    pub use super::file_metadata::{
        ACCESS_ACL_XATTR_NAME, Acl, AclIntoIter, AclIter, AclMode, AclQualifier,
        DEFAULT_ACL_XATTR_NAME, Device, FileKind, FileMetadata, FileMode, Gid, Owner, Uid, Xattrs,
        XattrsIntoIter, XattrsIter,
    };
    pub use super::sql::FileDiscriminant;
    pub use super::stats::FilesystemStats;
}

/// Types for user-defined metadata associated with filesystems and roots.
pub mod user {
    pub use super::user_metadata::{UserMetadata, UserMetadataIntoIter, UserMetadataIter};
}

/// Types for working with filesystem roots.
pub mod root {
    pub use super::id::{DEFAULT_ROOT_NAME, Root, RootBy, RootId, Roots};
}

/// Error types.
pub mod error {
    pub use super::errors::{Error, ErrorReason, Result, SqliteErrorCode};
}

/// Types for database connections, transactions, and the filesystem.
pub mod db {
    #[cfg(feature = "_encryption")]
    pub use super::connection::EncryptionSecret;
    pub use super::connection::{Connection, CreateOptions, JournalMode, OpenOptions};
    pub use super::filesystem::Filesystem;
    pub use super::id::FilesystemId;
    pub use super::transaction::{Transaction, TransactionBehavior};
}

/// Types for working with files and file contents.
pub mod file {
    pub use super::files::File;
    pub use super::id::{ContentId, FileId};
    #[cfg(all(feature = "fs", target_os = "linux"))]
    pub use super::lock::RawFd;
    pub use super::path::{FileBy, FileOrigin};
}

/// Types for walking the filesystem.
pub mod walk {
    pub use super::tree::{
        Children, Descendants, DirectoryEntry, WalkEntry, WalkOptions, WalkPredicate, WalkVisit,
    };
}

/// Types for interacting with the host filesystem.
pub mod fs {
    pub use super::archive::{ArchiveOptions, ExtractOptions};
    #[cfg(feature = "fuse")]
    pub use super::fuse::MountOption;
}

pub use connection::Connection;
pub use errors::{Error, Result};
pub use files::File;
pub use filesystem::Filesystem;

pub use db::*;
pub use error::*;
pub use file::*;
pub use fs::*;
pub use metadata::*;
pub use root::*;
pub use user::*;
pub use walk::*;