tarweb 0.1.1

io-uring based static file web server, with SNI router
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115

# tarweb

io-uring & ktls based webserver serving files from a tar file.

* No memory allocations
* No syscalls (`io_uring`)

https://github.com/ThomasHabets/tarweb

## Prerequisites

You need at least kernel 6.7, with `IO_URING` and `TLS` enabled. 6.7 introduced
io-uring enabled `setsockopt`, which is required.

If `CONFIG_TLS=m` then the `tls` kernel module needs to be loaded. At least on
my system, it doesn't automatically load on demand.

```
$ grep ^CONFIG_TLS= /boot/config-$(uname -r)
CONFIG_TLS=m
$ lsmod | grep ^tls
$ sudo modprobe tls
$ tlmod | grep ^tls
tls                   151552  0
```

To load it automatically on every boot, add it to `/etc/modules`:

```
$ echo tls | sudo tee -a /etc/modules
```

## Example use

Create a tar file with your whole site. There should be an `index.html` in the
root of the tar file.

Alternatively, provide `tarweb` with a prefix to strip.

```
$ tar tf site1.tar | head
blog.habets.se/
blog.habets.se/2015/
blog.habets.se/2015/11/
blog.habets.se/2015/11/Building-pov-ray-on-raspberry-pi.html
blog.habets.se/2015/11/How-I-made-my-custom-keyboard-layout-on-Linux-and-Windows.html
blog.habets.se/2015/03/
blog.habets.se/2015/03/How-to-boot-an-encrypted-system-safely.html
blog.habets.se/2015/03/Raytracing-Quake-demos.html
blog.habets.se/2015/03/My-mechanical-keyboard.html
blog.habets.se/2015/03/Scraping-data-from-a-BT-home-hub-5.html
$ tarweb \
  --listen '[::]:8081' \
  --tls-key privkey.pem \
  --tls-cert fullchain.pem \
  --prefix blog.habets.se/ \
  site1.tar
```

## Benchmarking

oha is a nice tool.

```
cargo install oha
oha https://localhost:8080/
```

Flamegraph

On VisionFive 2:

```
$ sudo sysctl -w kernel.perf_event_paranoid=-1
$ cargo install flamegraph
$ mkdir ~/bin; cat > ~/bin/myperf
set -ueo pipefail
echo "$@" >> wrapcmd
CMD="$1"
shift
if [[ $CMD = "record" ]]; then
        exec perf "$CMD" -e task-clock "$@"
fi
exec perf "$CMD" "$@"
^D
$ chmod +x ~/bin/myperf
$ PERF=$HOME/bin/myperf cargo flamegraph
```

On the VF2 for an ~24kB file, 1-2% of CPU is userspace. 75% of userspace CPU is
the RSA signing operation.
In kernel space it's all AES and TCP.

On a beefier machine, though still a laptop, `parse_request` is 35% of
userspace. `Archive::get_ofs` is 7%. Still, this laptop could probably saturate
a 10Gbps link at least, with small requests.

## Future work

* use `writev` to reduce queue roundtrips.
* `TCP_CORK`?
* SNI router: Per SNI ACL.

## Random notes

* There's no `sendfile()` in io-uring, but while there's `slice()` which could
  be used to implement `sendfile()` while bouncing on a pipe, there's no
  `pipe(2)` in io-uring either, so that'd force a syscall per connection. But I
  guess a pool of pipes can be pre-created?
  * I think I'll just wait for `sendfile()` to be supported natively in
    io-uring.
* Maybe it'd be possible to use io-uring buffers for read operations, but then
  either it'd require copying from the fixed buffer to the connection buffer, or
  parse request from a non-contiguous buffer (all the while holding on to the
  buffer, and there can only be 65536 of them).