fslite

A transport-independent, async filesystem interface; a persistent, multi-workspace SQLite implementation suitable for direct embedding; and an HTTP server, typed command codec, and CLI built on top of it.

Docs: guides and concepts · API reference on docs.rs · project homepage

Workspace layout

Quick start

use fslite_core::{RequestContext, VirtualPath, WriteSource};
use fslite_sqlite::SqliteFileSystem;
use futures::StreamExt;

# async fn run() -> Result<(), Box<dyn std::error::Error>> {
let fs = SqliteFileSystem::open_in_memory(Default::default()).await?;
let workspace = fs.create_workspace(Default::default()).await?;
let ctx = RequestContext::trusted(workspace.id);

let path = VirtualPath::parse("/hello.txt")?;
fs.write(&ctx, &path, WriteSource::from_bytes(b"hello".to_vec()), Default::default()).await?;

let read = fs.read(&ctx, &path, Default::default()).await?;
let mut stream = read.into_stream();
while let Some(chunk) = stream.next().await {
    print!("{}", String::from_utf8_lossy(&chunk?));
}
# Ok(()) }

See examples/embedded.rs for a complete, runnable version (cargo run --example embedded), and the full list below for more.

Examples

Every example is self-contained (opens its own in-memory database, needs no setup) and runnable with cargo run --example <name>:

CLI and server

fslite-cli can talk to a local database file, a private in-memory one, or a remote fslite-server over HTTP — the same verb syntax works in all three modes. Flag values use --name=value (the lexer does not treat a following bare word as a flag's value). --memory opens a fresh, unshared database for that single process only, so it's suited to one-off experiments or scripts that create a workspace and use it within the same invocation; use --db <path> whenever a workspace needs to be visible across separate fslite-cli invocations:

For everyday interactive use, fslite also supports named filesystems and workspaces, persisted in a small local registry ($XDG_CONFIG_HOME/fslite or ~/.config/fslite, overridable via FSLITE_CONFIG_DIR) so you don't need to repeat --db/--workspace on every invocation:

fslite create filesystem-main -f fsmain.db -w workspace-main
fslite use filesystem-main -w workspace-main

fslite mkdir /docs
fslite touch /docs/file.md
fslite write /docs/file.md --text=hello
fslite cat /docs/file.md
fslite rm /docs/file.md

fslite delete filesystem-main -y   # permanently deletes fsmain.db

create/use/delete manage this registry only — the names it stores have no meaning to fslite-core/fslite-sqlite/fslite-server, which only ever see raw workspace ids. --filesystem <name> and --workspace <name-or-id> override the persisted context for a single invocation without changing it (fslite --filesystem other-fs mkdir /tmp); the raw --db/--memory/--server + --workspace <uuid> flags shown below remain fully supported and bypass the registry/context entirely — useful for scripting against a database you don't want registered under a name.

# Local, one-shot, persisted to a file. --create-workspace prints the new
# workspace's id as its only line of output (after cargo's own build/run
# noise) — capture it with `-q` to suppress that noise and keep the
# variable clean:
WORKSPACE=$(cargo run -q -p fslite-cli -- --db ./fslite.db --create-workspace)
cargo run -p fslite-cli -- --db ./fslite.db --workspace "$WORKSPACE" mkdir /docs
cargo run -p fslite-cli -- --db ./fslite.db --workspace "$WORKSPACE" write /docs/hello.txt --text=hi
cargo run -p fslite-cli -- --db ./fslite.db --workspace "$WORKSPACE" ls /

# Local, interactive REPL (the workspace must already exist — create it
# first, as above, or against the same --db file)
cargo run -p fslite-cli -- --db ./fslite.db --workspace "$WORKSPACE" --repl

# Remote, against a running fslite-server
cargo run -p fslite-cli -- --server http://localhost:8080 --workspace <id> \
  --token "$FSLITE_TOKEN" ls /

fslite-server is a standalone binary; it reads bearer tokens from FSLITE_TOKENS (token=workspace_uuid pairs) and serves an in-memory SqliteFileSystem on 0.0.0.0:8080:

FSLITE_TOKENS="devtoken=<workspace-uuid>" cargo run -p fslite-server

This shipped main.rs is reference wiring, not a deployable server as-is: the backing store is a fresh in-memory database on every start (nothing survives a restart), and FSLITE_TOKENS is read once at startup, so a workspace created afterward via POST /v1/workspaces has no token that can reach it until the process is restarted with that id added — which then loses the workspace again. A real deployment needs its own main wiring a persistent SqliteFileSystem::open(path, ..) and its own AuthProvider that can mint or look up tokens for workspaces created at runtime.

Every verb fslite-command understands corresponds one-to-one to a FileSystem method: usage, stat, exists, ls, tree, mkdir, cat, write, write-at, append, truncate, touch, cp, mv, rm, ln, readlink, trash, trash-ls, restore, purge, setattr, rmattr, glob, find, grep, changes, and batch — plus --json on the CLI for machine-readable output.

Run fslite help to list every verb with a one-line summary, or fslite help <verb> for a verb's full flag table (the same metadata is also exposed as reference/cli-verbs in the docs site).

Limits

These are the SQLite backend's defaults; per-workspace byte/node/file-size quotas are configurable via WorkspaceOptions at create_workspace time.

• Regular file size: up to the configured max_file_bytes (workspace default: 1 GiB). Enforced incrementally as bytes stream in, before they're flushed to storage — an oversized write fails partway through, not after fully buffering the input.
• Workspace byte/node quotas: max_bytes (workspace default: 10 GiB) and max_nodes (workspace default: 1,000,000), checked against active
  • trashed logical bytes and node counts.
• Custom attributes: a 256-byte key, a 4096-byte value, and 64 attributes per node. Attribute values are arbitrary bytes; since Node::attributes is a JSON map, each value is base64url-encoded into a JSON string.
• Symbolic link resolution: at most 40 hops per path lookup, after which resolution fails with LinkLoop rather than looping forever.
• Content chunking: file content is stored as immutable 1 MiB chunks. Reads and writes are streamed chunk-by-chunk in bounded memory — a read never buffers more than the chunk currently being produced, regardless of file size.

Transaction guarantees

• Every mutation (write, mkdir, copy, move, remove, trash, restore, purge, attribute changes, ...) commits in one short SQLite transaction: the node/content change, its change-feed row, and any cascading cleanup (e.g. reclaiming a replaced file's old content generation) either all land together or none do.
• A replacement upload is invisible until it fully commits. write stages incoming bytes into a brand-new, independent content generation as they stream in. If the source stream errors partway through — or a configured quota is exceeded — the partially staged generation is discarded and the target's previous content and revision are left exactly as they were. Nothing about the existing file is touched until the new content is completely staged.
• batch is all-or-nothing. Every operation in a batch runs against one shared transaction, using each operation's internal transaction-level logic directly (never by recursively opening another connection call, which would deadlock against the single dedicated connection thread). The first failing operation aborts the whole batch — nothing commits — and the returned error's safe details include {"index": N} identifying which operation failed.
• Mutations accept an optional expected revision (expected_revision) and are rejected with RevisionConflict if the target has changed since the caller last observed it — a standard optimistic-concurrency precondition.

Workspace isolation

A single SQLite database holds many independent workspaces. Every stored query is scoped by workspace_id, so two workspaces can contain identically named paths (even at the root) without collision, and no operation in one workspace can observe or mutate another's nodes, trash, attributes, or change feed. Pagination cursors are versioned and bound to the workspace (and, where relevant, the specific parent/root they were issued for) that produced them; presenting a cursor to a different workspace fails with InvalidCursor rather than silently returning the wrong data.

Permanent remove vs. recoverable trash

• remove deletes a node (and, recursively, its entire subtree) permanently in one step. There is no way to recover a removed node short of restoring a database backup.
• trash marks a node's own row as trashed and records its original location; the node and its descendants become invisible to every directory listing and lookup immediately (descendants are never individually marked — they simply become unreachable once their parent is), but the underlying data is untouched. list_trash enumerates trashed subtrees, restore moves one back to its original location (or an alternate destination) if nothing already occupies that name, and purge permanently deletes a trashed subtree — this is the only way a trashed node's content is actually reclaimed.
• The workspace root can never be moved, trashed, or removed.

Development

cargo fmt --all --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace --all-features
cargo run --example embedded
cargo doc --workspace --no-deps

The backend's own test suite (crates/fslite-sqlite/tests/) exercises each operation group in depth; crates/fslite-conformance captures the smaller, backend-agnostic contract every FileSystem implementation must satisfy and is run against SqliteFileSystem in crates/fslite-sqlite/tests/conformance.rs.

Status

The canonical FileSystem trait and the SQLite backend are complete: all 28 trait methods are implemented and covered by the conformance suite plus the SQLite backend's own extensive test suite. fslite-server, fslite-command, and fslite-cli build on this crate's exact public API and are workspace members with their own test suites (HTTP contract tests for the server; lexer/parser/sanitizer/executor tests for the command codec; end-to-end local, remote, and REPL tests for the CLI). They've had several rounds of security hardening (bidi-override stripping, output sanitization, token handling) but are newer than fslite-core/fslite-sqlite and should be treated as less battle-tested.