async-fuser 0.17.0

Async + Sync Filesystem in Userspace (FUSE) building library for Rust
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170

# Async FUSE (Filesystem in Userspace) for Rust

![CI](https://github.com/mhambre/async-fuser/actions/workflows/ci.yml/badge.svg)
[![Crates.io](https://img.shields.io/crates/v/async-fuser.svg)](https://crates.io/crates/async-fuser)
[![status: experimental](https://github.com/GIScience/badges/raw/master/status/experimental.svg)](https://github.com/GIScience/badges#experimental)
[![Documentation](https://docs.rs/async-fuser/badge.svg)](https://docs.rs/async-fuser)
[![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/mhambre/async-fuser/blob/master/LICENSE.md)
[![dependency status](https://deps.rs/repo/github/mhambre/async-fuser/status.svg)](https://deps.rs/repo/github/mhambre/async-fuser)

## About

**Async-FUSE-Rust** is a Rust crate for building FUSE filesystems with both
synchronous and fully asynchronous APIs.

It is a fork of the [`fuser` crate](https://github.com/cberner/fuser), extended with a native async interface designed around Rust’s async ecosystem ([Tokio](https://tokio.rs/)), rather than wrapping a synchronous implementation.

The async API operates directly on `/dev/fuse` using an [`AsyncFd`](https://docs.rs/tokio/latest/tokio/io/unix/struct.AsyncFd.html), enabling non-blocking filesystem implementations without thread-per-request overhead.

The crate remains entirely API-compatible with `fuser`'s main API and can be used as a drop-in replacement for `fuser` version >=0.17.0.

#### Important Notes:
   * The asynchronous API is still in development and may have breaking changes. It is not recommended for production use yet, at least not without thorough testing, but feedback and contributions are welcome.
  * The `async` feature is enabled by default, disable default features to use the synchronous API if you are not using it.
  * Contributions to non-async features and bug fixes should be reported to the upstream `fuser` crate, and will make their way in once their master is updated.

## Why Async?

FUSE filesystems often operate under concurrent access from multiple processes and users. A synchronous implementation typically relies on thread-per-request handling, which can become inefficient under load due to thread contention and context switching.

The asynchronous API in Async-FUSE-Rust enables a different execution model:

* **Efficient multi-user concurrency**: Multiple filesystem operations (e.g. `read`, `lookup`, `readdir`) can be processed concurrently without blocking threads, improving throughput under parallel access patterns.
* **Integration with async ecosystems**: Designed around Tokio, the async API composes naturally with existing async infrastructure such as: networked backends (e.g. HTTP, gRPC, object storage), databases/caches, and distributed systems components. This avoids the need to bridge between blocking and non-blocking code.
* **End-to-end non-blocking design**: The async API operates directly on `/dev/fuse` using `AsyncFd`, rather than wrapping a synchronous interface, enabling fully non-blocking request handling.

## Examples

Minimal examples are available:

- [`examples/hello.rs`](./examples/hello.rs): synchronous filesystem
- [`examples/async_hello.rs`](./examples/async_hello.rs): asynchronous filesystem

These demonstrate the basic structure required to mount and serve a filesystem.

## Documentation

[Async-FUSE-Rust reference][Documentation]

## Details

A working FUSE filesystem consists of three parts:

1. The **kernel driver** that registers as a filesystem and forwards operations into a communication channel to a userspace process that handles them.
1. The **userspace library** (libfuse) that helps the userspace process to establish and run communication with the kernel driver.
1. The **userspace implementation** that actually processes the filesystem operations.

The kernel driver is provided by the FUSE project, the userspace implementation needs to be provided by the developer. Async-FUSE-Rust provides a replacement for the libfuse userspace library between these two. This way, a developer can fully take advantage of the Rust type interface and runtime features when building a FUSE filesystem in Rust.

Except for a single setup (mount) function call and a final teardown (umount) function call to libfuse, everything runs in Rust, and on Linux these calls to libfuse are optional. They can be removed by building without the "libfuse" feature flag.

## Dependencies

FUSE must be installed to build or run programs that use Async-FUSE-Rust (i.e. kernel driver and libraries. Some platforms may also require userland utils like `fusermount`). A default installation of FUSE is usually sufficient.

To build Async-FUSE-Rust or any program that depends on it, `pkg-config` needs to be installed as well.

### Linux

[FUSE for Linux] is available in most Linux distributions and usually called `fuse` or `fuse3` (this crate is compatible with both). To install on a Debian based system:

```sh
sudo apt-get install fuse3 libfuse3-dev
```

Install on CentOS:

```sh
sudo yum install fuse
```

To build, FUSE libraries and headers are required. The package is usually called `libfuse-dev` or `fuse-devel`. Also `pkg-config` is required for locating libraries and headers.

```sh
sudo apt-get install libfuse-dev pkg-config
```

```sh
sudo yum install fuse-devel pkgconfig
```

### macOS (untested)

Install [FUSE for macOS], which can be obtained from their website or installed using the Homebrew or Nix package managers. macOS version 10.9 or later is required. If you are using a Mac with Apple Silicon, you must also [enable support for third party kernel extensions][enable kext].


#### To install using Homebrew

```sh
brew install macfuse pkgconf
```

#### To install using Nix

``` sh
nix-env -iA nixos.macfuse-stubs nixos.pkg-config
```

When using `nix` it is required that you specify `PKG_CONFIG_PATH` environment variable to point at where `macfuse` is installed:

``` sh
export PKG_CONFIG_PATH=${HOME}/.nix-profile/lib/pkgconfig
```

### FreeBSD

Install packages `fusefs-libs` and `pkgconf`.

```sh
pkg install fusefs-libs pkgconf
```

## Usage

```sh
cargo add async-fuser
```

or put this in your `Cargo.toml`:

```toml
[dependencies]
async-fuser = "0.17"
```

To create a new filesystem, implement the trait `async_fuser::Filesystem`. This trait is kept up-to-date with `fuser::Filesystem`. To take advantage of the asynchronous API, enable the "async" feature flag, and use the `async_fuser::lib_async::AsyncFilesystem` trait. See the [documentation][Documentation] for details or the `examples` directory for some basic examples.

Unlike other Asynchronous FUSE-Rust APIs, the asynchronous API is not a wrapper around a synchronous API. It is completely asynchronous down to the `/dev/fuse` file descriptor using [`tokio::io::unix::AsyncFd`](https://docs.rs/tokio/latest/tokio/io/unix/struct.AsyncFd.html). While this isn't truly as far as we can go, see: [tokio-uring](https://github.com/tokio-rs/tokio-uring), for compatibility's sake this is the most practical approach. 

## To Do

Most features of libfuse up to 3.10.3 are implemented. Feel free to contribute. See the [list of issues][Issues (Asynchronous API)] on GitHub and search the source files for comments containing "`TODO`" or "`FIXME`" to see what's still missing.

If any of the issues are unrelated to the asynchronous API, they should be reported to the upstream [list of issues][Issues (Upstream API)] for the `fuser` crate and will make their way in once
their master is updated.

## Compatibility

Developed and tested on Linux. Tested under [Linux][FUSE for Linux] and [FreeBSD][FUSE for FreeBSD] using stable [Rust] (see CI for details).

## License

Licensed under [MIT License](LICENSE.md), except for those files in `examples/` that explicitly contain a different license.

## Contribution

Fork, hack, submit pull request. Make sure to make it useful for the target audience, keep the project's philosophy and Rust coding standards in mind. For larger or essential changes, you may want to open an issue for discussion first. Also remember to update the [Changelog] if your changes are relevant to the users.

[Rust]: https://rust-lang.org
[Homebrew]: https://brew.sh
[Changelog]: https://keepachangelog.com/en/1.0.0/

[FUSE-Rust (Upstream Parent)]: https://github.com/cberner/fuser
[Issues (Upstream API)]: https://github.com/cberner/fuser/issues
[Issues (Asynchronous API)]: https://github.com/mhambre/async-fuser/issues
[Documentation]: https://docs.rs/async-fuser

[FUSE for Linux]: https://github.com/libfuse/libfuse/
[FUSE for macOS]: https://macfuse.github.io
[enable kext]: https://github.com/macfuse/macfuse/wiki/Getting-Started#enabling-support-for-third-party-kernel-extensions-apple-silicon-macs
[FUSE for FreeBSD]: https://wiki.freebsd.org/FUSEFS