Module fs

File based blob store.

A file based blob store needs a writeable directory to work with.

General design:

The file store consists of two actors.

The main actor

The purpose of the main actor is to handle user commands and own a map of handles for hashes that are currently being worked on.

It also owns tasks for ongoing import and export operations, as well as the database actor.

Handling a command almost always involves either forwarding it to the database actor or creating a hash context and spawning a task.

The database actor

The database actor is responsible for storing metadata about each hash, as well as inlined data and outboard data for small files.

In addition to the metadata, the database actor also stores tags.

Tasks

Tasks do not return a result. They are responsible for sending an error to the requester if possible. Otherwise, just dropping the sender will also fail the receiver, but without a descriptive error message.

Tasks are usually implemented as an impl fn that does return a result, and a wrapper (named ..._task) that just forwards the error, if any.

That way you can use ? syntax in the task implementation. The impl fns are also easier to test.

Context

The main actor holds a TaskContext that is needed for almost all tasks, such as the config and a way to interact with the database.

For tasks that are specific to a hash, a HashContext combines the task context with a slot from the table of the main actor that can be used to obtain an unique handle for the hash.

Runtime

The fs store owns and manages its own tokio runtime. Dropping the store will clean up the database and shut down the runtime. However, some parts of the persistent state won’t make it to disk, so operations involving large partial blobs will have a large initial delay on the next startup.

It is also not guaranteed that all write operations will make it to disk. The on-disk store will be in a consistent state, but might miss some writes in the last seconds before shutdown.

To avoid this, you can use the crate::api::Store::shutdown method to cleanly shut down the store and save ephemeral state to disk.

Note that if you use the store inside a iroh::protocol::Router and shut down the router using iroh::protocol::Router::shutdown, the store will be safely shut down as well. Any store refs you are holding will be inoperable after this.

Modules

options

Options for configuring the file store.

Structs

FsStore

A file based store.