usbvfiod 0.2.0

A vfio-user server for USB pass-through.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115

# Quick Start

The specification reference is available at [https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf](https://www.intel.com/content/dam/www/public/us/en/documents/technical-specifications/extensible-host-controler-interface-usb-xhci.pdf).

## Testing with Cloud Hypervisor

An easy way to get a testing setup is to connect `usbvfiod` with Cloud
Hypervisor. For this, start `usbvfiod` in one terminal:

```console
$ cargo run --bin usbvfiod -- --socket-path /tmp/usbvfiod.sock -vv
2025-04-25T09:41:40.891734Z  INFO usbvfiod: We're up!
```

In another terminal, start Cloud Hypervisor. Any recent version will
do:

```console
$ cloud-hypervisor \
   --memory size=4G,shared=on \
   --serial tty \
   --user-device socket=/tmp/usbvfiod.sock \
   --console off \
   --kernel KERNEL \
   --initramfs INITRD \
   --cmdline KERNEL_CMDLINE
```

`KERNEL`, `INITRD`, and `KERNEL_CMDLINE` are placeholders for a Linux
kernel image (`bzImage`), an initrd or initramfs and the corresponding
command line.

> [!TIP]
> To get a kernel and initramfs to play with, you can use the [NixOS](https://nixos.org/)
> [netboot](https://nixos.org/manual/nixos/stable/index.html#sec-booting-from-pxe) binaries.
>
> You will find a kernel (`bzImage`) and initrd. The required command
> line for booting is in `result/netboot.ipxe`. You want to add
> `console=ttyS0` to get console output.
>
> ```console
> $ nix-build -A netboot.x86_64-linux '<nixpkgs/nixos/release.nix>'
> $ ls result/
> bzImage initrd netboot.ipxe
> ...
> $ grep -o "init=[^$]*" result/netboot.ipxe
> init=/nix/store/.../init initrd=initrd nohibernate loglevel=4
> ```

## Attaching USB Devices
Currently USB devices can be attached when (1) `usbvfiod`
is started and (2) at runtime by sending a specific command to the hotplug socket.

In both cases pass-through devices are identified by the path to the USB device node. These
paths are of the form `/dev/bus/usb/$BUS/$DEVICE`.

To figure out the bus and device numbers of a specific USB device, use
the `lsusb` utility (typically installed via the `usbutils` package):

```console
$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:0033 Intel Corp. AX211 Bluetooth
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 003: ID 18a5:0243 Verbatim, Ltd Flash Drive (Store'n'Go)
```

To attach the specific USB device you would:
- (1) add `--device /dev/bus/usb/002/003` as a parameter to `usbvfiod` for attachment at startup, and/or
- (2) add `--hotplug-socket-path /tmp/hotplug.sock` as a parameter to `usbvfiod` and later run the provided `remote` binary as `remote --socket /tmp/hotplug.sock --attach /dev/bus/usb/002/003`.

### Kernel Debug Messages for XHCI

The XHCI driver in the Linux kernel prints helpful messages with `xhci_dbg`.
If your kernel supports `CONFIG_DYNAMIC_DEBUG` (the NixOS netboot kernel does),
you can enable the messages for XHCI with:

```
$ echo "file drivers/usb/host/xhci* +p" | sudo tee /sys/kernel/debug/dynamic_debug/control
```

Then, you can filter the kernel log for the relevant messages:

```
$ sudo dmesg | grep -E xhci\|usb
```

> [!TIP]
> You can also enable the dynamic debug messages on boot by adding
> `xhci_hcd.dyndbg==pmfl xhci_pci.dyndbg==pmfl` to the command line.
>
> Alternatively, provide `dyndbg==pfml` as option to `modprobe` on
> invocation or through a `modprobe` config.

### PCAP Tracing for USB Transfers

To inspect transfer messages more conveniently without searching through
log output, `usbvfiod` can write USB traffic to a PCAP file:

```console
$ cargo run --bin usbvfiod -- --socket-path /tmp/usbvfiod.sock --pcap-path PCAP_PATH -vv
```

`PCAP_PATH` is the path where `usbvfiod` should write the generated PCAP file.
The generated file uses the Linux USB PCAP link type and can be opened with
tools such as Wireshark. It records transfer submissions, completions, and
errors together with the endpoint and transfer metadata.

In the traditional Linux USB PCAP format, the bus number and device address
fields refer to the real USB device path, as in `/dev/bus/usb/$BUS/$DEVICE`.
For `usbvfiod` traces, this project adapts these fields to its own needs:

- the bus number is the USB version (`2` for USB 2.x speeds, `3` for USB 3.x
  speeds, or `0` if the speed is unknown), and
- the device address is the xHCI slot ID.