# osm-io
This crate provides tools used to manipulate the [Open Street Map](https://wiki.openstreetmap.org/wiki/Main_Page)
data.
The basic OSM data model is broadly defined by [OSM XML](https://wiki.openstreetmap.org/wiki/OSM_XML)
with Node and Way objects defining geometry and Relation and Tag objects providing unstructured
extension mechanisms. Due to different uses of the OSM data, variety of
[data formats](https://wiki.openstreetmap.org/wiki/Databases_and_data_access_APIs) emerged to
store and transmit it.
Our main focus here are the following formats:
* [apidb](https://wiki.openstreetmap.org/wiki/Openstreetmap-website/Database_schema) - database
schema backing [OSM](https://www.openstreetmap.org/) website
* [*.osm.pbf](https://wiki.openstreetmap.org/wiki/PBF_Format) - a very efficient data format
used to transmit OSM data that can be downloaded from http://download.geofabrik.de/ or from
https://planet.openstreetmap.org/pbf/.
The goal at this stage is to be able to load large *.osm.pbf files, such as planet.osm.pbf into a
Postgresql OSM database (apidb schema) and to dump an entire Postgres OSM database into a
planet.osm.pbf file. See [osm-admin](https://github.com/navigatorsguild/osm-admin)
Because the data sets are very large, a special attention is given to maintaining control over
memory size and utilizing multiple CPU cores whenever is possible.
## Roadmap
* implement *.osm.pbf reader and writer - Done
* implement *.osm.pbf parallel reader and parallel writer - Done
* implement apidb reader and writer - Done
* provide basic filtering (see example below) - Done
* convert between *.osm.pbf and apidb and vice versa - Done see examples.
* [S2](http://s2geometry.io/) indexing - index the entire OSM dataset by S2 cells for farther
processing
* context indexing - index the entire OSM dataset by relations between its objects. So, for
example, it would be possible to efficiently discard all Nodes that belong to a deleted Way.
## Issues
Issues are welcome and appreciated. Please submit to https://github.com/navigatorsguild/osm-io/issues
## Examples
Example for filtering out nodes from *.osm.pbf extract
```rust
use std::path::PathBuf;
use anyhow;
use benchmark_rs::stopwatch::StopWatch;
use simple_logger::SimpleLogger;
use osm_io::osm::model::element::Element;
use osm_io::osm::pbf;
use osm_io::osm::pbf::compression_type::CompressionType;
use osm_io::osm::pbf::file_info::FileInfo;
pub fn main() -> Result<(), anyhow::Error> {
SimpleLogger::new().init()?;
log::info!("Started pbf io pipeline");
let mut stopwatch = StopWatch::new();
stopwatch.start();
let input_path = PathBuf::from("./tests/fixtures/niue-230109.osm.pbf");
let output_path = PathBuf::from("./target/results/niue-230109.osm.pbf");
let reader = pbf::reader::Reader::new(&input_path)?;
let mut file_info = FileInfo::default();
file_info.with_writingprogram_str("pbf-io-example");
let mut writer = pbf::writer::Writer::from_file_info(
output_path,
file_info,
CompressionType::Zlib,
)?;
writer.write_header()?;
for element in reader.elements()? {
let mut filter_out = false;
match &element {
Element::Node { node } => {
for tag in node.tags() {
if tag.k() == "natural" && tag.v() == "tree" {
filter_out = true;
break;
}
}
}
Element::Way { way: _ } => {}
Element::Relation { relation: _ } => {}
Element::Sentinel => {
filter_out = true;
}
}
if !filter_out {
writer.write_element(element)?;
}
}
writer.close()?;
log::info!("Finished pbf io pipeline, time: {}", stopwatch);
Ok(())
}
```
## Similar Software
* [libosmium](https://osmcode.org/libosmium/) - very fast and very mature with a Python wrapper.
* [osmosis](https://wiki.openstreetmap.org/wiki/Osmosis) - reference implementation for most if
not all features.
* [osmpbf](https://crates.io/crates/osmpbf) - very efficient *.osm.pbf reader written in Rust
License: MIT OR Apache-2.0