Expand description
A generic sync abstraction: a tree of object::Objects (a vault::Vault) whose
actual storage lives behind a pluggable origin::Origin — a local directory, an HTTP
API, a shell command, or another vault. Syncing “a folder on disk” and “objects behind a
REST API” run through the exact same code path, because both are just implementations of
one Origin trait.
§The model
object::Object— a node in the tree:Leaf(has content),Branch(has children), orRoot. Objects only ever carry metadata (name, id, size, content type, modified time, plus a free-formextramap) — never raw bytes — so listing a tree never materializes its contents into memory.object::ObjectId— a newtype aroundString, opaque and origin-specific (a relative path for the filesystem origin, an arbitrary id for HTTP/command origins).origin::Origin— the trait every backend implements:fetch,list,get,put,send,delete.fetch/sendare streaming (origin::ByteStream=BoxStream<'static, VaultResult<Bytes>>) — content moves in chunks, it’s never buffered whole into RAM.put(object, destination)writesobjectunderdestinationand returns theObjectas it now exists at the origin — implementations may, but aren’t required to, renameobjectin place, so callers should always act on the returned value rather than assumingobjectitself changed.vault::Vault— owns oneOriginplus an in-memory metadata cache.get/listpopulate the cache;listalways re-hits the origin (it’s the source of truth) while refreshing the cache.findresolves a/-separated path to anObjectIdby walking the tree onelistcall per component.putonly updates the cache on success, and caches theObjectputreturned.pull/pushrecursively sync a subtree between the vault’s own origin and any other&dyn Origin, threadingput’s return value through to the followingsend.
Four built-in origins ship in this crate:
origin::fs::OriginFileSystem(type = "fs") — a directory on disk, viatokio::fs.origin::http::OriginHTTP(type = "http") — any REST-ish API, with a{id}-templated URL per operation.origin::command::OriginCommand(type = "command") — a shell command per operation; the universal escape hatch for anything that isn’t a plain filesystem or HTTP API.origin::vault::OriginVault(type = "vault") — anothervault::Vault, wrapped so it can act as an origin in its own right, letting two vaults sync directly withpush/pullwithout either side needing to know the other is aVaultrather than a plain origin.
§Configuration
A vault is fully described by a TOML file, deserialized into config::VaultConfig /
config::OriginConfig:
# vault.toml — backed by a local directory
name = "my-vault"
[origin_config]
type = "fs"
root = "/srv/data"use nimbus_vault::vault::Vault;
let vault = Vault::new("vault.toml".into())?;
let root = vault.find("/".into()).await?;
let children = vault.list(root).await?;An origin_config can also be built standalone, without a name/root_id/Vault
wrapper, via config::OriginConfig::from_file — useful for tooling that talks to an
origin directly, or for building the remote argument to
vault::Vault::pull/vault::Vault::push.
A config file can live anywhere, but config::VaultConfig::default_path gives the
conventional home for one — vaults/<name>.toml under config_home — which is where
nimbus’s own tooling writes new vaults so they can be found again later.
§Quick start
A fully self-contained, end-to-end example — writing a vault config, creating an object, reading it back, and listing its parent:
use std::fs;
use futures::StreamExt;
use nimbus_vault::{object::{Metadata, Object, ObjectId}, vault::Vault};
let dir = tempfile::tempdir()?;
let data_dir = dir.path().join("data");
fs::create_dir_all(&data_dir)?;
let config_path = dir.path().join("vault.toml");
fs::write(
&config_path,
format!(
"name = \"my-vault\"\n\n[origin_config]\ntype = \"fs\"\nroot = \"{}\"\n",
data_dir.display(),
),
)?;
let vault = Vault::new(config_path)?;
// create an object
let mut notes = Object::Leaf {
name: "notes.txt".to_string(),
id: ObjectId::from("notes.txt"),
meta: Metadata::new(),
};
vault.put(&mut notes, &ObjectId::from("")).await?;
vault.send(¬es, Box::pin(futures::stream::once(async {
Ok(bytes::Bytes::from_static(b"hello vault"))
}))).await?;
// read it back
let mut stream = vault.fetch("notes.txt").await?;
let mut bytes = Vec::new();
while let Some(chunk) = stream.next().await {
bytes.extend_from_slice(&chunk?);
}
assert_eq!(&bytes, b"hello vault");
// list the vault's root
let names: Vec<String> = vault.list("").await?.iter().map(Object::get_name).collect();
assert_eq!(names, vec!["notes.txt".to_string()]);§Syncing
vault::Vault::pull/vault::Vault::push sync a subtree between the vault’s own origin
and any other &dyn Origin — another config-defined origin, or another vault wrapped in
origin::vault::OriginVault:
use nimbus_vault::{config::OriginConfig, vault::Vault};
let vault = Vault::new("vault.toml".into())?;
let remote = OriginConfig::from_file("remote.toml".into())?;
let root = vault.find("".into()).await?;
vault.pull(&root, remote.as_ref()).await?; // bring local up to date with remote
vault.push(&root, remote.as_ref()).await?; // push local changes back out§Errors
Every fallible operation in this crate returns VaultResult, an alias for
Result<T, error::VaultError>.
§Constants
The {placeholder} keys, OriginConfig::Command field names, and root id/name
conventions shared across object and the origin::command/origin::http
implementations are defined once here (e.g. ROOT_ID, PLACEHOLDER_ID,
FETCH_CMD_FIELD) rather than being hardcoded independently in each module.
Modules§
- config
- On-disk (TOML) configuration for a
vault::Vaultand theorigin::Originit wraps. - error
- The crate’s error type,
error::VaultError, and theVaultResultalias built from it. - object
- The tree node type,
object::Object, along withobject::ObjectIdandobject::Metadata. - origin
- The
origin::Origintrait and its built-in implementations (fs,http,command,vault). - vault
vault::Vault, the tree-like view over anOriginthat ties everything together.
Constants§
- DELETE_
CMD_ FIELD config::OriginConfig::Command/origin::command::CmdTypefield name for the delete command template.- FETCH_
CMD_ FIELD config::OriginConfig::Command/origin::command::CmdTypefield name for the fetch command template.- GET_
CMD_ FIELD config::OriginConfig::Command/origin::command::CmdTypefield name for the get command template.- LIST_
CMD_ FIELD config::OriginConfig::Command/origin::command::CmdTypefield name for the list command template.- OBJECT_
KIND_ BRANCH - Value substituted for
PLACEHOLDER_KINDwhen putting a directory. - OBJECT_
KIND_ LEAF - Value substituted for
PLACEHOLDER_KINDwhen putting a file. - PLACEHOLDER_
CONTENT_ TYPE - Template placeholder key substituted with an object’s content type in
origin::command::OriginCommandtemplates (rendered as{content_type}). - PLACEHOLDER_
DESTINATION extra_varskeyorigin::command::OriginCommand’sputuses to expose itsdestinationargument to theput_cmdtemplate (rendered as{destination}). Refreshed on everyput.- PLACEHOLDER_
ID - Template placeholder key substituted with an object’s id in
origin::command::OriginCommand/origin::http::OriginHTTPtemplates (rendered as{id}). - PLACEHOLDER_
KIND - Template placeholder key substituted with an object’s kind in
origin::command::OriginCommandtemplates (rendered as{kind}), taking the valueOBJECT_KIND_LEAForOBJECT_KIND_BRANCH. Without it aput_cmdhas no way to tell whether it is being asked to create a file or a directory. - PLACEHOLDER_
MODIFIED - Template placeholder key substituted with an object’s modified timestamp in
origin::command::OriginCommandtemplates (rendered as{modified}). - PLACEHOLDER_
NAME - Template placeholder key substituted with an object’s name in
origin::command::OriginCommandtemplates (rendered as{name}). - PLACEHOLDER_
SIZE - Template placeholder key substituted with an object’s payload size in
origin::command::OriginCommandtemplates (rendered as{size}). - PUT_
CMD_ FIELD config::OriginConfig::Command/origin::command::CmdTypefield name for the put command template.- ROOT_ID
- Conventional root
object::ObjectIdvalue, used byobject::ObjectId::defaultandobject::Object::root. - ROOT_
NAME - Display name reported by
object::Object::get_namefor theRootvariant, which has no real name of its own. - SEND_
CMD_ FIELD config::OriginConfig::Command/origin::command::CmdTypefield name for the send command template.- UNKNOWN_
CONTENT_ TYPE - Fallback content type substituted into command templates when an object’s metadata doesn’t specify one.
Functions§
- config_
home - Directory nimbus keeps its own configuration in:
.nimbusunder the platform config directory ($XDG_CONFIG_HOMEon Linux), falling back to the temp directory when there isn’t one.
Type Aliases§
- Vault
Result - The result type used throughout this crate, shorthand for
Result<T,error::VaultError>.