Skip to main content

Crate geopackage

Crate geopackage 

Source
Expand description

Read and write OGC GeoPackage files.

A GeoPackage is an SQLite database with a standardised schema for vector features and raster tiles. GeoPackage::create, GeoPackage::open and GeoPackage::open_read_only open the container with pragma and schema validation; GeoPackage::layer returns a Layer handle for reading and writing features, and GeoPackage::layers enumerates them. GeoPackage::tiles returns a TilePyramid handle for the tile side, described under Tiles below.

A command-line companion, gpkg, is built by the geopackage-cli crate: gpkg info, gpkg validate, gpkg index, gpkg repair, gpkg copy and gpkg tiles inspect, check and convert files without writing any code. Install it with cargo install geopackage-cli.

§Quick start

Create a file, define a point layer, write features, query by bounding box:

use geo_types::Point;
use geopackage::core::types::{ColumnType, GeometryType};
use geopackage::{
    BoundingBox, ColumnSpec, GeoPackage, GeometrySpec, NewFeature, TableSchemaBuilder, Value,
    ValueRef,
};

let gpkg = GeoPackage::create(path)?;

gpkg.create_layer(
    &TableSchemaBuilder::new("cities")
        .column(ColumnSpec::new("name", ColumnType::Text(None)))
        .geometry(GeometrySpec::new(GeometryType::Point, 4326)),
)?;

let layer = gpkg.layer("cities")?;
layer.write_all(
    vec![
        NewFeature::new(Point::new(-6.26, 53.35), vec![Value::Text("Dublin".into())]),
        NewFeature::new(Point::new(-0.13, 51.51), vec![Value::Text("London".into())]),
    ],
    0,
)?;

// Served by the layer's RTree index, which `create_layer` builds unless
// `TableSchemaBuilder::spatial_index(false)` turns it off.
for feature in layer.features_in(BoundingBox::new(-7.0, 53.0, -6.0, 54.0))? {
    let feature = feature?;
    assert_eq!(feature.value("name"), Some(ValueRef::Text("Dublin")));
}

§Features

§Geometry round trips

Feature::geometry parses the stored blob into a GpbGeometry, a view over the row’s bytes that implements geo_traits::GeometryTrait, and every write method accepts any impl GeometryTrait<T = f64>. A geometry can therefore be streamed out of one file, measured, and written into another without being converted to a geo-types value in either direction: the analysis reads coordinates from the stored encoding, and the writer encodes WKB from the same view. What does allocate is each row’s blob, copied out of SQLite, and the new blob the writer serialises; an algorithm that produces new geometry also allocates its output.

use geo_traits::{CoordTrait, GeometryTrait, GeometryType as Kind, LineStringTrait};
use geopackage::core::types::{ColumnType, GeometryType};
use geopackage::{ColumnSpec, GeoPackage, GeometrySpec, TableSchemaBuilder, Value, ValueRef};

/// Planar length, read from the trait: no geometry object is built.
fn length(geometry: &impl GeometryTrait<T = f64>) -> f64 {
    let Kind::LineString(line) = geometry.as_type() else {
        return 0.0;
    };
    let mut sum = 0.0;
    let mut prev: Option<(f64, f64)> = None;
    for coord in line.coords() {
        let (x, y) = (coord.x(), coord.y());
        if let Some((px, py)) = prev {
            sum += ((x - px).powi(2) + (y - py).powi(2)).sqrt();
        }
        prev = Some((x, y));
    }
    sum
}

let src = GeoPackage::open_read_only(&src_path)?;
let dst = GeoPackage::create(&dst_path)?;
dst.create_layer(
    &TableSchemaBuilder::new("measured")
        .column(ColumnSpec::new("length", ColumnType::Double))
        .geometry(GeometrySpec::new(GeometryType::LineString, 4326)),
)?;

let roads = src.layer("roads")?;
let measured = dst.layer("measured")?;
let mut writer = measured.writer()?;
let mut cursor = roads.cursor()?;
for feature in cursor.features()? {
    let feature = feature?;
    if let Some(geometry) = feature.geometry()? {
        // `geometry` borrows the row's blob; `length` reads coordinates
        // from it, and `insert` encodes WKB from the same view.
        let l = length(&geometry);
        writer.insert(None, &geometry, &[ValueRef::Float(l)])?;
    }
}
writer.commit()?;

let mut total = 0.0;
for feature in measured.features()? {
    if let Some(l) = feature?.value("length").and_then(|v| v.as_f64()) {
        total += l;
    }
}
assert_eq!(total, 7.0);

§Tiles

A tile pyramid is the container’s other data type: pre-rendered raster tiles, addressed by zoom level, column and row, with a gpkg_tile_matrix_set row fixing the ground extent they are indexed against and a gpkg_tile_matrix row per zoom level. GeoPackage::create_tile_pyramid writes one (from a TilePyramidBuilder), GeoPackage::tiles opens one, and GeoPackage::tile_pyramids enumerates them.

Payloads are opaque. This crate stores, indexes and validates tiles; it decodes none of them, and depends on no image codec. It reads each payload’s header, which is how a tile written at the wrong pixel size, or in a format the table may not contain, is rejected rather than stored. Turning a tile into pixels, or a source raster into a pyramid, needs an image library or GDAL on top of this one.

Rows count from the top of the extent downwards, as WMTS and XYZ do and TMS does not, and the indices are relative to the pyramid’s own extent rather than to a global grid. TileMatrix::flip_row converts to and from the TMS sense, and TileMatrixSet::xyz_to_tile errors rather than mis-addressing when a pyramid is not the standard web mercator quad.

use geopackage::core::tiles::{TileCoord, TileMatrixSet, ZoomLadder};
use geopackage::{GeoPackage, TilePyramidBuilder};

let gpkg = GeoPackage::create(path)?;
gpkg.add_epsg_srs(3857)?;

// The spec's default arrangement: each zoom level doubles the grid, with
// pixel sizes derived from the extent so they span it exactly.
let matrix_set = TileMatrixSet::web_mercator_quad();
let matrices = matrix_set.ladder(ZoomLadder::new(0, 4))?;
let tiles = gpkg.create_tile_pyramid(
    &TilePyramidBuilder::new("basemap", matrix_set).matrices(matrices),
)?;

tiles.put_tile(TileCoord::new(1, 0, 0), &png(256, 256))?;
assert!(tiles.get_tile(TileCoord::new(1, 0, 0))?.is_some());

// Streaming a pyramid borrows each payload from the row it was read
// from, so nothing is copied to walk one.
let mut cursor = tiles.cursor()?;
let mut stream = cursor.tiles()?;
while let Some(tile) = stream.next()? {
    assert_eq!(tile.data().len(), 33);
}

§Columnar I/O

Enabled by the arrow feature, Layer::read_arrow reads a layer as Arrow record batches, multithreaded by default, and Layer::write_arrow writes batches back through the same path as Layer::write_all. Geometry is a GeoArrow WKB column whose metadata includes the CRS as PROJJSON. TableSchemaBuilder::from_arrow_schema is the layer definition an Arrow schema implies, so a layer can be copied without its schema being restated; the type mapping both directions share is documented on the arrow module.

use geopackage::arrow::ArrowReadOptions;

let src = GeoPackage::open_read_only(&src_path)?;
let cities = src.layer("cities")?;

let dst = GeoPackage::create(&dst_path)?;
let schema = cities.arrow_schema()?;
dst.create_layer(&TableSchemaBuilder::new("cities").from_arrow_schema(&schema)?)?;

let batches = cities.read_arrow(ArrowReadOptions::default())?;
dst.layer("cities")?.write_arrow(batches, 0)?;
assert_eq!(dst.layer("cities")?.features()?.len(), 2);

§Extensions

gpkg_extensions is where a file declares what it uses beyond the core spec. GeoPackage::extensions reads that catalogue, and Layer::extensions and TilePyramid::extensions narrow it to one table. Every row identifies as an Extension and has an ExtensionSupport: read and written here, identified and left alone, removed from the standard in 2016 and accepted on read, or not recognised at all.

That last one is not only informational. Writing to a table covered by an extension this crate cannot identify fails with Error::UnsupportedExtension, because such an extension may constrain the rows, triggers or encodings of the table it covers, and writing beside it could produce a file its own producer can no longer read. Reading never fails for this reason. GeoPackage::blocking_extension asks the question directly, and OpenOptions::allow_unsupported_extension_writes disables the check.

Two extensions are surfaced as part of the model rather than as catalogue rows. gpkg_crs_wkt puts a WKT2 CRS definition and a coordinate epoch on Srs, which is how a CRS with no WKT1 form is represented at all. gpkg_schema describes columns and constrains their values: GeoPackage::data_columns and Column::data_column give the descriptions, GeoPackage::column_constraint resolves what a column’s values are limited to, and GeoPackage::set_data_column and GeoPackage::add_column_constraint write them.

use geopackage::{ColumnConstraint, ConstraintKind, DataColumn, GeoPackage, OpenOptions};

let gpkg = GeoPackage::open(&path)?;
gpkg.add_column_constraint(&ColumnConstraint {
    name: "years".into(),
    kind: ConstraintKind::Range {
        min: 1900.0,
        min_is_inclusive: true,
        max: 2000.0,
        max_is_inclusive: false,
    },
    description: None,
})?;
gpkg.set_data_column(
    "sites",
    &DataColumn {
        column_name: "year".into(),
        name: Some("Year surveyed".into()),
        title: None,
        description: None,
        mime_type: None,
        constraint_name: Some("years".into()),
    },
)?;

// The constraints are advisory in the format, so checking written values
// against them is opt-in rather than assumed.
let gpkg = OpenOptions::new()
    .enforce_column_constraints(true)
    .open(&path)?;

§Cargo features

  • geo-types (on by default): forwards geopackage-core’s feature of the same name, which adds GpbGeometry::to_geo. Disable it with default-features = false.
  • arrow (off by default): the columnar paths above. It pulls in arrow-array and arrow-schema, which a caller using only the scalar API does not need.

§Configuration

The defaults are intended to support the common case: a single-file GeoPackage, an indexed feature layer, and values read in keeping with other popular implementations. Each of these types documents possible trade-off behind its defaults:

Two settings are available outside the options types. Layer::write_all and Layer::write_arrow take a batch_size, the number of rows sharing a transaction, where 0 writes all of them in one. Layer::with_geometry_type_validation checks each geometry against its column’s declared type while reading, and is off by default.

Under the arrow feature, ArrowReadOptions configures the columnar read: rows per batch (batch_size, default DEFAULT_BATCH_SIZE, 65,536), how many threads may read at once (threads, default 0, meaning min(4, available parallelism)), and a ceiling on the geometry bytes one batch may hold (max_batch_bytes, default default_max_batch_bytes, min(INT32_MAX, RAM / 4)): the geometry column’s Arrow offsets are 32-bit, so no batch can address more than 2 GB of WKB, and a batch that would cross the ceiling is emitted short so a layer of very large geometries still reads. The columnar write has no options type of its own: Layer::write_arrow_with takes the same BulkIndexOptions as Layer::write_all_with.

Anything not covered here is reachable as SQL: GeoPackage::connection returns the underlying rusqlite connection.

§What writes, and when

Most of this crate divides cleanly into reads and writes, but three calls do not, so the whole surface is tabulated here rather than left to be inferred from the names. Layer::extent records what it had to measure; Layer::repair_spatial_index writes only when there is something to repair; and Layer::writer opens without writing, because SQLite’s BEGIN DEFERRED takes no lock, so the first failure lands on the first row.

CallWrites to the fileOn a read-only connection
Layer::features, Layer::cursor, Layer::features_in, Layer::selectneverworks
Layer::spatial_index_status, Layer::has_spatial_indexneverworks
Layer::audit_spatial_indexneverworks
GeoPackage::contentsneverworks
Layer::extentonly where the recorded bounds are unusableworks: measures, returns, records nothing
Layer::repair_spatial_indexonly where the trigger set is not currentworks where there is nothing to repair
Layer::recompute_extentalwaysfails
Layer::create_spatial_index, Layer::drop_spatial_index, Layer::rebuild_spatial_indexalwaysfails
Layer::writeron its row methods and its commit, not on the callopens; the first row written fails
Layer::write_all, Layer::write_arrowalwaysfails
GeoPackage::tiles, TilePyramid::get_tile, TilePyramid::cursorneverworks
TilePyramid::validateneverworks
TilePyramid::put_tile, TilePyramid::delete_tile, TilePyramid::write_allalwaysfails
TilePyramid::writeron its tile methods and its commit, not on the callopens; the first tile written fails

Reading an extent therefore modifies the file when the recorded bounds are unusable, which is deliberate and matches GDAL: the file stops being wrong for every later reader rather than only for this one. Layer::extent documents why, and the two ways to avoid it.

§What can fail, and why

§Reading untrusted files

The wkb 0.9.2 reader this crate parses geometry with pre-allocates from element counts read out of the blob without bounding them against the buffer, so a malformed geometry declaring a 0xFFFFFFFF-member collection drives a multi-gigabyte allocation. The fix belongs upstream in georust/wkb; until it lands and this crate bumps its dependency, you should take care when parsing GeoPackage files from untrusted sources.

Re-exports§

pub use extensions::ExtensionRow;
pub use geopackage_core as core;

Modules§

arrow
Arrow schema derivation: GeoPackage column types to Arrow field types.
extensions
gpkg_extensions: the catalogue of extensions a file declares, the one place this crate writes to it, and what this crate can do with each row.

Structs§

BoundingBox
An XY bounding box for spatial queries (Layer::features_in).
BulkIndexOptions
Tuning for the RTree bulk-build path.
Column
Metadata for one column of a user table, from PRAGMA table_info.
ColumnConstraint
A constraint on a column’s values, assembled from the gpkg_data_column_constraints rows sharing one constraint_name.
ColumnSpec
One non-geometry, non-primary-key column of a TableSchemaBuilder.
ContentsEntry
A row of gpkg_contents.
ConversionOptions
Options controlling Value conversion from stored SQLite values.
DataColumn
A row of gpkg_data_columns: what one column of one table means.
Feature
A single row of a layer, owned so it outlives the SQLite cursor.
FeatureCursor
A prepared streaming read over a layer, owning its statement.
FeatureStream
A streaming iterator over a FeatureCursor’s rows.
FeatureWriter
A prepared-statement writer over one layer, owning a transaction.
Features
A fallible iterator of Features from a layer read.
GeoPackage
An open GeoPackage.
GeometryColumn
A row of gpkg_geometry_columns (spec Table 21): the geometry column of a feature table, its declared geometry type, spatial reference system, and z/m dimension constraints.
GeometrySpec
The geometry column of a feature table: its name, type, spatial reference system, and z/m dimension constraints.
Layer
A handle to one feature or attribute layer of a GeoPackage.
MetadataRecord
A row of gpkg_metadata: one metadata document.
MetadataReference
A row of gpkg_metadata_reference: what a record is attached to.
NewFeature
A new row for crate::Layer::write_all: an optional explicit feature id, an optional geometry, and the value-column values in column order.
NewMetadata
A metadata document to add, before it has an id.
NewRelation
A relationship to create.
OpenOptions
A builder for opening or creating a GeoPackage with an explicit journal mode and/or synchronous level.
Relation
A row of gpkgext_relations: one relationship.
SpatialIndexAudit
The result of Layer::audit_spatial_index: how the index’s contents compare with the geometries they are supposed to describe.
Srs
A row of gpkg_spatial_ref_sys.
TableSchema
The introspected schema of a user table: its columns and, for feature tables, the attached GeometryColumn.
TableSchemaBuilder
A declarative builder for a user table’s schema.
Tile
One tile of a pyramid, borrowing its payload from the row it was read from.
TileCursor
A prepared tile scan, owning its statement so that TileCursor::tiles can borrow each payload straight out of the row.
TilePyramid
A handle to one tile pyramid of a GeoPackage.
TilePyramidBuilder
A declarative builder for a tile pyramid.
TileStream
A scan in progress, borrowing one tile at a time.
TileWriter
A transaction over one pyramid, with per-tile put and delete.

Enums§

BulkVerification
How much of a bulk-built RTree is checked before it is trusted.
ConstraintKind
The three constraint forms Requirement 108 allows.
ContentsDataType
gpkg_contents.data_type values.
DateTimeParsing
How Value conversion interprets DATETIME text.
Error
Errors returned by this crate.
Extension
An extension name this workspace can identify.
ExtensionScope
gpkg_extensions.scope: what an extension affects (Requirement 64).
ExtensionSupport
What this workspace can do with a registered extension.
Finding
Something GeoPackage::validate found.
GpkgVersion
GeoPackage specification version, as declared by a file’s pragmas.
JournalMode
SQLite journal mode for a GeoPackage connection.
LayerKind
Which kind of user table a Layer wraps.
MetadataScope
What a metadata record describes (md_scope, spec Table 15).
MetadataTarget
What a metadata reference is attached to, as a caller states it.
OpenWarning
A non-fatal condition GeoPackage::open_lenient accepted while opening a file. Retrieve the list with GeoPackage::open_warnings.
ReferenceScope
The granularity of a metadata attachment (reference_scope).
RelationName
A relationship’s kind (relation_name).
Severity
How much a Finding matters.
SpatialIndexStatus
The health of a layer’s RTree spatial index, from Layer::spatial_index_status.
StorageStrictness
How Value conversion treats a stored value that its declared type does not strictly permit but that can still be read as that type.
Synchronous
SQLite synchronous durability level (PRAGMA synchronous).
Value
A typed, non-geometry column value.
ValueRef
A borrowed Value: the same cases, with text and binary borrowed from whatever stores the row’s bytes.

Constants§

DEFAULT_BULK_THRESHOLD
Default candidate-row count at or above which crate::Layer::create_spatial_index and crate::Layer::write_all choose the bulk shadow-table build over the per-row triggered build.
DEFAULT_BUSY_TIMEOUT
How long a statement waits for another connection’s lock before failing, unless OpenOptions::busy_timeout says otherwise.
DEFAULT_FILL_FACTOR
Default fraction of each RTree node’s capacity used by the bulk build.
DEFAULT_GEOMETRY_COLUMN
The conventional geometry column name for a GeoPackage feature table.
DEFAULT_PRIMARY_KEY
The conventional primary-key column name for a GeoPackage feature or attribute table.

Type Aliases§

Result
Convenience alias.