Struct ParallelWriter

pub struct ParallelWriter { /* private fields */ }

Write *.osm.pbf file while performing concurrently significant parts of work.

The parallel writer accepts somewhat unordered stream of elements, orders these elements, splits them into FileBlocks and writes them to the target file. The writer is composed from an ordering thread that maintains a large enough buffer to provide high probability of restoring the order, multiple encoding threads that do the heavy lifting of encoding the PBF and compressing, and finally the writing thread tht writes encoded blobs to file. The ParallelWriter uses more memory because of the internal ordering buffers controlled by the element_ordering_buffer_size parameter to constructor. It is limited to use cases where the processing of each element takes roughly the same time, as in simple filtering tasks or that elements were ordered before calling the writer. For example please see ./examples/parallel-bf-io.rs

Implementations

impl ParallelWriter

pub fn from_file_info( element_ordering_buffer_size: usize, file_block_size: usize, path: PathBuf, file_info: FileInfo, compression_type: CompressionType, ) -> Result<ParallelWriter, Error>

Create ParallelWriter from FileInfo

pub fn write_header(&mut self) -> Result<(), Error>

Write the *.osm.pbf header.

Must be called before writing the first element.

pub fn write_element(&mut self, element: Element) -> Result<(), Error>

Write an Element

pub fn write_elements(&mut self, elements: Vec<Element>) -> Result<(), Error>

Write list of Elements

pub fn close(&mut self) -> Result<(), Error>

Flush internal buffers.