hecate 0.10.1

OpenStreetMap Inspired Data Storage Backend Focused on Performance and GeoJSON Interchange
Documentation

Table Of Contents

  1. Brief
  2. Table of Contents
  3. Build Environment
  4. Docker File
  5. Feature Format
  6. Server
  7. API

Build Environment

  • Start by installing Rust from rust-lang.org, this will install the current stagle version
curl https://sh.rustup.rs -sSf | sh
  • Source your bashrc/bash_profile to update your PATH variable
source ~/.bashrc        # Most Linux Distros, some OSX
source ~/.bash_profile  # Most OSX, some Linux Distros
  • Install the nightly build of rust, Rocket, the web-framework relies on some advanced compiler options not yet included in the default build.
rustup install nightly
rustup default nightly
  • Download and compile the project and all of it's libraries
cargo build
  • Create the hecate database using the provided schema file.
echo "CREATE DATABASE hecate;" | psql -U postgres

psql -U postgres -f src/schema.sql hecate
  • Start the server
cargo run
  • Test it is working - should respond with HTTP200
curl 'localhost:8000

Docker File (Coverage Tests)

The Docker file is designed to give the user a testing environment to get rust tests to generate code coverage information

Install docker and then run

docker build .

docker run  --security-opt seccomp=unconfined {{HASH FROM ABOVE}}

The --security-opt flag is required to be able to run and view kcov output.

Feature Format

Hecate is designed as a GeoJSON first interchange and uses standard GeoJSON with a couple additions and exceptions as outlined below.

Supported Geometry Types

  • Point
  • MultiPoint
  • LineString
  • MultiLineString
  • Polygon
  • MultiPolygon

Unsupported Geometry Types

  • GeometryCollection

Additional Members

The following table outlines top-level members used by hecate to handle feature creation/modification/deletion.

Key/Value pairs in the .properties of a given feature are never directly used by the server and are simply passed through to the storage backend. This prevents potential conflicts between user properties and required server members.

Member Notes
id The unique integer id of a given feature. Note that all features get a unique id accross GeoJSON Geometry Type
version The version of a given feature, starts at 1 for a newly created feature
action Only used for uploads, the desired action to be performed. One of create, modify or delete

Examples

Downloaded Features

{
    "id": 123,
    "version": 2,
    "type": "Feature",
    "properties": {
        "shop": true,
        "name": "If Pigs Could Fly"
    },
    "geometry": {
        "type": "Point",
        "coordinates": [0,0]
    }
}

Downloaded Features will return the integer id of the feature, the current version and the user supplied properties and geojson. action is not applicable for downloaded features, it is only used on upload.

Create Features

{
    "action": "create",
    "type": "Feature",
    "properties": {
        "shop": true,
        "name": "If Pigs Could Fly"
    },
    "geometry": {
        "type": "Point",
        "coordinates": [0,0]
    }
}

A features being uploaded for creation must have the action: create property. Since an id and version have not yet been assigned they should be omitted. Should an id or version be included the server will ignore them, assigning a new id and version per the servers internal id provisioner.

Modify Features

{
    "id": 123,
    "version": 1,
    "action": "modify",
    "type": "Feature",
    "properties": {
        "shop": true,
        "name": "If Pigs Could Fly"
    },
    "geometry": {
        "type": "Point",
        "coordinates": [0,0]
    }
}

A feature being uploaded for modification must have the action: modify as well as the id and version property. The id is the integer id of the feature to modify and the version property is the current version of the feature as stored by the server. If the version uploaded does not match the version that the server has stored, the modify will fail. This prevents consecutive edits from conflicting.

Note that the modify operation is not a delta operation and the full feature with the complete Geometry & All Properties must be included with each modify.

Also note that since the id pool is shared accross geometry types, an id is allowed to change it's geometry type. eg. If id: 1 is a Point and then a subsequent action: modify with a Polygon geometry is performed, id: 1 is allowed to switch to the new Polygon type.

Delete Features

{
    "id": 123,
    "version": 1,
    "action": "delete",
    "type": "Feature",
    "properties": null,
    "geometry": null
}

A feature being uploaded for deletion must have the action: delete as well as the id and version property. See Modify Features above for an explanation of those properties.

Note the properties and geometry attributes must still be included. They can be set to null or be their previous value. They will be ignored.

Samples

Server

This section of the guide goes over various options on has when launching the server

Hecate can be launched with default options with

cargo run

Database

By default hecate will attempt to connect to postgres@localhost:5432/hecate.

Note that only postgres/postgis backed databases are currently supported.

This database should be created prior to launching hecate. For instructions on setting up the database see the Build Environment section of this doc.

A custom database name, postgres user or port can be specified using the database flag.

cargo run -- --database "<USER>:<PASSWORD>@<HOST>/<DATABASE>"

cargo run -- --database "<USER>@<HOST>/<DATABASE>"

JSON Validation

By default hecate will allow any properties on a given GeoJSON feature, including nestled arrays, maps, etc.

A custom property validation file can be specified using the schema flag.

cargo run -- --schema <PATH-TO-SCHEMA>.json

Note hecate currently supports the JSON Schema draft-04. Once draft-06/07 support lands in valico we can support newer versions of the spec.

API

Index

GET /

Healthcheck URL, currently returns Hello World!

Example

curl -X GET 'http://localhost:8000/

View the Admin Interface in your browser by pointing to 127.0.0.1:8000/admin/index.html

GET /api/tiles/<z>/<x>/<y>

Request a vector tile for a given set of coordinates. A Mapbox Vector Tile is returned.

Options

Option Notes
<z> REQUIRED Desired zoom level for tile
<x> REQUIRED Desired x coordinate for tile
<y> REQUIRED Desired y coordinate for tle

Example

curl -X GET 'http://localhost:8000/api/user/create?ingalls&password=yeaheh&email=ingalls@protonmail.com

GET /api/user/create

Create a new user, provied the username & email are not already taken

Options

Option Notes
username REQUIRED Desired username, must be unique
password REQUIRED Desired password
email REQUIRED Desired email, must be unique

Example

curl -X GET 'http://localhost:8000/api/user/create?ingalls&password=yeaheh&email=ingalls@protonmail.com

GET /api/data/bounds/

Return an array of possible boundary files with which data can be extracted from the server with

Example

curl -X GET 'http://localhost:8000/api/data/bounds

GET /api/data/bounds/<bounds>

Return line delimited GeoJSON Feature of all the geometries within the specified boundary file.

Options

Option Notes
<bounds> REQUIRED One of the boundary files as specified via the /ap/data/bounds

Example

curl -X GET 'http://localhost:8000/api/data/bounds/us_dc

GET /api/data/feature/<id>

Return a single GeoJSON Feature given its' ID.

Options

Option Notes
<id> REQUIRED Numeric ID of a given feature to download

Example

curl -X GET 'http://localhost:8000/api/data/features/1542

GET /api/data/features

Return a FeatureCollection of all features within a given bbox

Options

Option Notes
bbox REQUIRED Bounding Box in format left,bottom,right,top

POST /api/data/feature Auth Required

Create, Modify, or Delete an individual GeoJSON Feature

The Feature must follow format defined in Feature Format.

The feature also must contain a top-level String message attribute describing the changes being made (The delta message)

Example

curl \
    -X POST \
    -H "Content-Type: application/json" \
    -d '{"action": "create", "message": "Random Changes", "type":"Feature","properties":{"shop": true},"geometry":{"type":"Point","coordinates":[0,0]}}' \
    'http://username:password@localhost:8000/api/data/feature'

POST /api/data/features Auth Required

Create, Modify, and/or Delete many features via a GeoJSON FeatureCollection

The Features in the FeatureCollection must follow format defined in Feature Format.

The FeatureCollection also must contain a top-level String message attribute describing the changes being made (The delta message)

Note that a mix of create, modify, and delete operatioons are allowed within each FeatureCollection

Example

curl \
    -X POST \
    -H "Content-Type: application/json" \
    -d '{"type":"FeatureCollection","message":"A bunch of changes","features": [{"action": "create", "type":"Feature","properties":{"shop": true},"geometry":{"type":"Point","coordinates":[0,0]}}]}' \
    'http://username:password@localhost:8000/api/data/features'

GET /api/deltas

Returns an array of the last 20 deltas with their corresponding metadata. Does not include geometric data on the delta. Request a specific delta to get geometric data.

Options

Option Notes
offset OPTIONAL Offset the returned 20 values by a given integer

Example

curl -X GET 'http://localhost:8000/api/deltas

curl -X GET 'http://localhost:8000/api/deltas?offset=3

GET /api/deltas/<id>

Returns all data for a given delta as a JSON Object, including geometric data.

Options

Option Notes
<id> REQUIRED Get all data on a given delta

Example

curl -X GET 'http://localhost:8000/api/delta/4

The primary goal of the hecate project is a very fast GeoJSON based Interchange. That said, the tooling the OSM community has built around editing is unparalled. As such, Hecate provides a Work-In-Progress OpenStreetMap Shim to support a subset of API operations as defined by the OSM API v0.6 document.

Important Notes

  • All GeoJSON types can be downloaded via the API and viewed in JOSM
  • MultiPoints
    • Are represented using an OSM Relation
    • The type will be multipoint
    • The member type will be point
  • MultiLineStrings
    • Are represented using an OSM Relation
    • The type will be multilinestring
    • The member will be line
  • Uploading Way & Relation types are not currently supported, attempting to upload them may produce undesirable results.

The following incomplete list of endpoints are implemented with some degree of coverage with the OSM API Spec but are likely incomplete/or written with the minimum flexibility required to support editing from JOSM. See the code for a full list.

GET /api/capabilities

GET /api/0.6/capabilities

Return a static XML document describing the capabilities of the API.

Example

curl -X GET 'http://localhost:8000/api/capabilities

GET /api/0.6/user/details Auth Required

Returns a static XML document describing the number of unread messages that a user has. Every n minutes JOSM checks this and displays in the interface if there is a new message, to cut down on errors it simply returns a 0 message response.

Example

curl -X GET 'http://localhost:8000/api/0.6/user/details

PUT /api/0.6/changeset/create Auth Required

Create a new changset and set the meta information, returning the opened id.

Example

curl \
    -X PUT \
    -d '<osm><changeset><tag k="comment" v="Just adding some streetnames"/></changeset></osm>' \
    'http://localhost:8000/api/0.6/changeset/create

GET /api/0.6/changeset/<changeset_id>/upload Auth Required

Upload osm xml data to a given changeset

Example

curl \
    -X POST \
    -d '<diffResult version="0.6">NODE/WAY/RELATIONS here</diffResult>' \
    'http://localhost:8000/api/0.6/changeset/1/upload'

PUT /api/0.6/changeset/<changeset_id>/close Auth Required

Close a given changeset, preventing further modification to it

Example

curl -X PUT 'http://localhost:8000/api/0.6/changeset/1/close'