pbfhogg 0.5.0

Fast OpenStreetMap PBF reader and writer for Rust. Read, write, and merge .osm.pbf files with pipelined parallel decoding.
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81

# Installation

## CLI

Install the `pbfhogg` binary from crates.io:

```sh
cargo install pbfhogg-cli
```

This builds from source with fat LTO. The binary is called `pbfhogg`. Requires Rust 1.96+.

## Library

Add the library to your `Cargo.toml`:

```toml
[dependencies]
pbfhogg = "0.5"
```

This enables the default `commands` feature. If you only need read/write (no extract, check-refs, or geocode index), disable it to skip `serde_json` and `s2` dependencies:

```toml
[dependencies]
pbfhogg = { version = "0.5", default-features = false }
```

### Feature flags

| Feature | Default | Description |
|---------|---------|-------------|
| `commands` | yes | Enables `check_refs`, `extract`, geocode index builder, and their deps (`serde_json`, `s2`) |
| `geocode-reader` | included by `commands` | Enables `geocode_index::Reader` for reverse geocoding queries (depends on `s2`) |
| `linux-direct-io` | no | O_DIRECT read/write paths - bypasses page cache for planet-scale I/O (Linux only) |
| `linux-io-uring` | no | io_uring writer with registered buffers - faster writes above ~4 GB (Linux only, requires kernel 5.1+) |

For reverse geocoding queries without the full `commands` feature:

```toml
[dependencies]
pbfhogg = { version = "0.5", default-features = false, features = ["geocode-reader"] }
```

## Building from source

```sh
git clone https://github.com/folknor/pbfhogg
cd pbfhogg
cargo install --path cli
```

To enable O_DIRECT and io_uring support:

```sh
cargo install --path cli --features linux-direct-io,linux-io-uring
```

### Build requirements

- Rust 1.96 or later
- No C compiler required - all protobuf encoding is hand-rolled wire format, and zlib uses `zlib-rs` (pure Rust)
- No `protoc` or Protocol Buffers toolchain needed

## Platform

pbfhogg is developed on Linux and untested elsewhere. The core read/write functionality has no platform-specific code, but the production-relevant features (`linux-direct-io`, `linux-io-uring`) are Linux-only. O_DIRECT requires a real filesystem (not tmpfs). io_uring requires Linux 5.1+ and sufficient `RLIMIT_MEMLOCK`.

## Verifying the installation

```sh
pbfhogg --version

pbfhogg --help
```

To check that a PBF file can be read:

```sh
pbfhogg inspect denmark.osm.pbf
```