Skip to main content

Crate tol_compress

Crate tol_compress 

Source
Expand description

Tolerance-bounded compression of spatial coordinate data.

This crate stores spatial coordinates using a practical tolerance-compressed method, which takes advantage of two specific features of real world measurement data:

  1. Real world measurement systems have known precisions below which differences in position do not represent meaningful information, and users of measurement data have knowledge about where increasingly small differences in values cease to be relevant to their use case.
  2. Values produced by the measurement of physical objects rarely span more than a few orders of magnitude more than the smallest meaningful precision for the measurement system and/or the application.

If a user can supply a tolerance that represents the largest acceptable round-trip accuracy of any value in their dataset, the storage algorithm can find a representation scheme that uses the smallest amount of bytes necessary to guarantee every position is recovered within that tolerance.

§Where to start

Most callers want one of the four container modules, which read and write whole files:

moduleholdsextension
meshtriangle meshes in 3D.tcmesh
polylineordered polylines in 2D or 3D, open or closed.tccurve2, .tccurve3
cloudunordered point sets in 2D or 3D.tccloud2, .tccloud3
row_pointspoints in 3D grouped into the rows a rasterizing sensor produced.tcrpf3
use std::path::Path;
use tol_compress::{Mesh3, mesh};

let mesh = Mesh3::new(
    vec![[0.0, 0.0, 0.0], [1.0, 0.0, 0.0], [0.0, 1.0, 0.0]],
    vec![[0, 1, 2]],
);

// Every vertex is guaranteed back within one micron of where it started.
mesh::write_one_file(Path::new("part.tcmesh"), &mesh, 1e-6)?;
let back = mesh::read_one_file(Path::new("part.tcmesh"))?;

Every container is an ordered collection, so a file may hold one item or many. The write_one_* and read_one_* functions are conveniences over that rather than a separate format, and reading one item from a file holding several is an error rather than a silent truncation. probe identifies a file without decoding its geometry.

Items carry an optional name and an optional Metadata map, which is stored and never interpreted, so a caller can record whatever their domain needs without this crate learning anything about it.

How hard the encoder works for a smaller file is one setting, Effort, rather than a flag per technique. Each container’s write_to_with and write_file_with take a WriteOptions carrying it, alongside that container’s own semantic settings. See the effort module for what does and does not belong on that dial.

§Underneath

The containers are built from block encoders that are useful on their own if you are storing geometry inside some other format:

  • points encodes coordinates in any number of dimensions against a tolerance, and quantize and bounds are where the tolerance guarantee is actually decided.
  • indices encodes simplex connectivity, exactly, at whichever of its two codings is smaller.
  • rows encodes the integer grouping that assigns a sequence of points to sensor rows.
  • reorder renumbers a mesh so its indices compress. mesh applies this by default; see its documentation if you hold per-vertex data outside the file.
  • segment plans where a point sequence wants to be cut into partitions, and transform carries the frame a partition can be written in, which nothing currently chooses.
  • blocks and bits are the packing layer everything else sits on.

§Errors and forward compatibility

Everything fallible returns Error, which implements std::error::Error, so a caller using a boxed-error alias absorbs it through ? with no glue.

Two fields in the format are reserved for features not yet implemented: compression and per-item attributes. A reader from this version meeting a file that uses one of them fails with a specific error rather than misreading it, and files written today stay readable as those features arrive. The multiple-partition field was reserved the same way and is now in use; because the decoder always read it generically, a reader built before partitioning shipped reads a partitioned file correctly.

The point block’s per-partition transform is further along than either: both ends of it are implemented and tested, but nothing chooses a frame, so no file this version writes uses it. See transform for what it is worth and why it is idle.

Re-exports§

pub use blocks::BLOCK;
pub use bounds::Bounds;
pub use cloud::Cloud;
pub use cloud::Cloud2;
pub use cloud::Cloud3;
pub use container::Header;
pub use container::Kind;
pub use container::Named;
pub use container::find_by_name;
pub use container::probe;
pub use effort::Effort;
pub use error::Error;
pub use error::Result;
pub use indices::bits_for_count;
pub use indices::read_indices;
pub use indices::write_indices;
pub use mesh::Mesh3;
pub use mesh::VertexOrder;
pub use metadata::Metadata;
pub use metadata::Value;
pub use points::read_points;
pub use points::write_points;
pub use points::write_points_with;
pub use polyline::Polyline;
pub use polyline::Polyline2;
pub use polyline::Polyline3;
pub use quantize::Quantizer;
pub use quantize::bits_for_tol;
pub use reorder::Reordering;
pub use reorder::permute;
pub use row_points::PointRow3;
pub use row_points::RowPoints3;
pub use rows::RowLayout;
pub use rows::read_rows;
pub use rows::write_rows;

Modules§

bits
Bit-level stream I/O.
blocks
Per-block adaptive bit widths, the PFor / SIMD-BP128 technique without the SIMD.
bounds
Axis-aligned bounding intervals in any number of dimensions.
cloud
Point cloud containers, conventionally .tccloud2 and .tccloud3.
container
File framing: the header that turns blocks into a file, and the per-item metadata shared by every kind of container.
effort
How hard should an encoder look to make a smaller file?
error
The crate’s error type.
indices
Encoding and decoding blocks of simplex connectivity.
mesh
Triangle mesh containers, conventionally .tcmesh.
metadata
Optional key-value metadata, carried by files and by individual items.
points
Encoding and decoding blocks of points in any number of dimensions.
polyline
Ordered polyline containers, conventionally .tccurve2 and .tccurve3.
quantize
This module has the machinery to map real values onto integer lattices, and the supporting tooling to decide the lattice width needed to meet a desired tolerance.
reorder
Reordering a mesh’s faces and vertices so its indices compress.
row_points
Row-organized point containers, conventionally .tcrpf3.
rows
The row block: how points are grouped into ordered rows.
segment
This module is for cutting a point sequence into contiguous runs, where each will end up being written against its own tighter bounding box.
transform
This module is for reading, writing, and applying transformations to point partitions so that the points inside it can be quantized over a smaller space, rather than whatever they happened to be in world axes.