osmpbf 0.1.0

A reader for the OpenStreetMap PBF file format (*.osm.pbf). 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121

osmpbf
======
A Rust library for reading the OpenStreetMap PBF file format (\*.osm.pbf). It
strives to offer the best performance using parallelization and lazy-decoding
with a simple interface while also exposing iterators for items of every level
in a PBF file.

## Usage

Add this to your `Cargo.toml`:

```toml
[dependencies]
osmpbf = "0.1"
```

and this to your crate root:

```rust
extern crate osmpbf;
```

Here's a simple example that counts all the ways in a file:

```rust
extern crate osmpbf;

use osmpbf::*;

fn main() {
    let reader = ElementReader::from_path("tests/test.osm.pbf").unwrap();
    let mut ways = 0_u64;

    // Increment the counter by one for each way.
    reader.for_each(|element| {
        if let Element::Way(_) = element {
            ways += 1;
        }
    }).unwrap();

    println!("Number of ways: {}", ways);
}
```

In this second example, we also count the ways but make use of all cores by
decoding the file in parallel:

```rust
use osmpbf::*;
 
fn main() {
    let reader = ElementReader::from_path("tests/test.osm.pbf").unwrap();

    // Count the ways
    let ways = reader.par_map_reduce(
        |element| {
            match element {
                Element::Way(_) => 1,
                _ => 0,
            }
        },
        || 0_u64,      // Zero is the identity value for addition
        |a, b| a + b   // Sum the partial results
    ).unwrap();
     
    println!("Number of ways: {}", ways);
}
```

## The PBF format

To effectively use the more lower-level features of this library it is useful to
have an overview of the structure of a PBF file. For a more detailed format
description see [here](http://wiki.openstreetmap.org/wiki/PBF_Format) or take a
look at the `.proto` files in this repository.

The PBF format as a hierarchy (square brackets `[]` denote arrays):
```
Blob[]
├── HeaderBlock
└── PrimitiveBlock
    └── Group[]
    	├── Node[]
    	├── DenseNodes
    	├── Way[]
        └── Relation[]
```

At the highest level a PBF file consists of a sequence of Blobs. Each Blob can
be decoded into either a HeaderBlock or a PrimitiveBlock.

Iterating over Blobs is very fast, but decoding might involve a more expensive
decompression step. So especially for larger files it is advisable to
parallelize at the Blob level as each Blob can be decompressed independently.

Usually the first Blob of a file decodes to a HeaderBlock which holds global
information for all following PrimitiveBlocks, such as a list of required
parser features.

A PrimitiveBlock contains an array of Groups (named `PrimitiveGroup` in
osmformat.proto). Each Group only contains one element type: Node, Way,
Relation or DenseNodes. A DenseNodes item is an alternative and space-saving
representation of a Node array. So, do not forget to check for DenseNodes when
aggregating all nodes in a file.

Elements reference each other using integer IDs. Corresponding elements could be
stored in any Blob, so finding them can involve iterating over the whole file.
Some files declare an optional feature "Sort.Type\_then\_ID" in the HeaderBlock to
indicate that elements are stored sorted by their type and then ID. This can be
used to dramatically reduce the search space.

# License

This project is licensed under either of

 * Apache License, Version 2.0, ([LICENSE-APACHE](LICENSE-APACHE) or
   http://www.apache.org/licenses/LICENSE-2.0)
 * MIT license ([LICENSE-MIT](LICENSE-MIT) or
   http://opensource.org/licenses/MIT)

at your option.