Please check the build logs for more information.
See Builds for ideas on how to fix a failed build, or Metadata for how to configure docs.rs builds.
If you believe this is docs.rs' fault, open an issue.
Hecate Feature Comparison
Feature | Hecate | ESRI MapServer | OSM Backend |
---|---|---|---|
Vector Tile Creation | :heavy_check_mark: | :heavy_check_mark: | :x: |
Streaming Query API | :heavy_check_mark: | :x: | :x: |
Multi User Support | :heavy_check_mark: | :heavy_check_mark: | :heavy_check_mark: |
Feature History | :heavy_check_mark: | :x: | :heavy_check_mark: |
Atomic API Operations | :heavy_check_mark: | :heavy_check_mark: | :x: |
GeoJSON-LD Based API | :heavy_check_mark: | :x: | :heavy_check_mark: |
Mapbox GL JS Styling | :heavy_check_mark: | :heavy_check_mark: | :x: |
Integrated Data Stats | :heavy_check_mark: | :heavy_check_mark: | :x: |
Table Of Contents
- Brief
- Why Use Hecate
- Table of Contents
- Related Libraries
- Build Environment
- Docker File
- Feature Format
- Server
- API
Related Libraries
- HecateJS Javascript Library & CLI Tool for interacting with the Hecate API
- Hecate-Example Script for importing some fake data for testing
Built something cool that uses the Hecate API? Let us know!
Build Environment
- Start by installing Rust from rust-lang.org, this will install the current stable version
|
- Source your
bashrc/bash_profile
to update yourPATH
variable
- Install the
nightly-2018-12-01
build of rust,Rocket
, the web-framework relies on some advanced compiler options not yet included in the default build.
- Download and compile the project and all of it's libraries
-
Ensure you have database dependencies
postgres
andpostgis
installed. -
Create the
hecate
database using the provided schema file. These instructions assume you have set up a rolepostgres
with sufficient privileges.
|
- This step will also create a database role called
hecate
andhecate_read
. If the connection fails due to authentication, your pg_hba file may not be set up to trust local connections.
Your pb_hba file location can be found using echo "show hba_file;" | psql -U postgres
Replace the file with the following:
local all postgres trust
local all all trust
host all all 127.0.0.1/32 trust
host all all ::1/128 trust
host replication postgres samenet trust
- Start the server
- Test it is working - should respond with
HTTP200
You will now have an empty database which can be populated with your own data/user accounts.
If you want to populate the database with sample data for testing, ingalls/hecate-example has a selection of scripts to populate the database with test data.
Docker File (Coverage Tests)
The Docker file is designed to give the user a testing environment to easily run rust tests.
Install docker and then run
docker build .
docker run {{HASH FROM ABOVE}}
Feature Format
Hecate is designed as a GeoJSON first interchange and uses standard GeoJSON with a couple additions and exceptions as outlined below.
Supported Geometry Types
Point
MultiPoint
LineString
MultiLineString
Polygon
MultiPolygon
Unsupported Geometry Types
GeometryCollection
Additional Members
The following table outlines top-level members used by hecate to handle feature creation/modification/deletion.
Key/Value pairs in the .properties
of a given feature are never directly used by the server and are simply
passed through to the storage backend. This prevents potential conflicts between user properties and required
server members.
Member | Notes |
---|---|
id |
The unique integer id of a given feature. Note that all features get a unique id accross GeoJSON Geometry Type |
version |
The version of a given feature, starts at 1 for a newly created feature |
action |
Only used for uploads, the desired action to be performed. One of create , modify , delete , or restore |
key |
Optional A String containing a value that hecate will ensure remains unique across all features. Can be a natural id (wikidata id, PID, etc), computed property hash, geometry hash etc. The specifics are left up to the client. Should an attempt at importing a Feature with a differing id but identical key be made, the feature with will be rejected, ensuring the uniqueness of the key values. By default this value will be NULL . Duplicate NULL values are allowed. |
force |
Optional Boolean allowing a user to override version locking and force UPSERT a feature. Disabled by default |
Examples
Downloaded Features
Downloaded Features will return the integer id
of the feature, the current version
and the user supplied properties
and geojson
.
action
is not applicable for downloaded features, it is only used on upload.
Create Features
A features being uploaded for creation must have the action: create
property. Since an id
and version
have not yet been
assigned they must be omitted. Should an id
be included it will be ignored. Adding a version
property will throw an error.
Optionally create actions can use the force: true
option to perform an UPSERT
like option. In this mode the uploader must
specify the key
value. Hecate will then INSERT
the feature if the key
value is new, if the key
is already existing, the
existing feature will be overwritten with the forced feature. Note that this mode ignores version checks and is therefore unsafe.
Force Prerequisites
- Disabled by default, must be explicitly enabled via Custom Authentication
- Can only be performed on a feature with
action: create
- Must specify a valid
key
Modify Features
A feature being uploaded for modification must have the action: modify
as well as the id
and version
property. The id
is the integer id of the feature to modify and the version
property is the
current version of the feature as stored by the server. If the version uploaded does not match the version that the server has stored, the modify will fail. This prevents consecutive edits from conflicting.
Note that the modify operation is not a delta operation and the full feature with the complete Geometry & All Properties must be included with each modify.
Also note that since the id
pool is shared accross geometry types, an id is allowed to change it's geometry type. eg. If id: 1
is a Point
and then a subsequent action: modify
with a Polygon
geometry is performed, id: 1
is allowed to switch to the new Polygon
type.
Delete Features
A feature being uploaded for deletion must have the action: delete
as well as the id
and version
property. See Modify Features above for an explanation of those properties.
Note the properties
and geometry
attributes must still be included. They can be set to null
or be their previous value. They will be ignored.
Restore Features
A feature being uploaded for restoration must have the action: restore
as well as the id
and version
properties. A restore
action is just a modify
on a deleted feature.
Restore places the new given geometry/properties at the id specified. It does not automatically roll back the feature to it's state before deletion, if this is desired, one
must use the Feature History API to get the state before deletion and then perform the restore
action.
Note: Restore will throw an error if an feature still exists.
Server
This section of the guide goes over various options for launching the server
Hecate can be launched with default options with
cargo run
Database
Main Connection
By default hecate will attempt to connect to hecate@localhost:5432/hecate
for read/write
operations and simultaneously connect to hecate_read@localhost:5432/hecate
for
sandboxed read only operations.
Note that only postgres w/ postgis enabled is supported.
This database should be created prior to launching hecate. For instructions on setting up the database see the Build Environment section of this doc.
A custom database name, postgres user or port can be specified using the database flag.
Example
Sandbox Connection
A second read-only account should also be created with permissions to SELECT from the
geo
& deltas
table. This endpoint will only be used for the query
endpoint, which
allows arbitrary user query execution. A sample implementation can be found in the schema.sql
document
Note: It is up to the DB Admin to ensure the permissions are limited in scope for this user. Hecate will expose access to this user via the query endpoint.
If multiple instances of database_sandbox
are present, hecate will load balance accross the multiple read instances.
Replica Connection [optional]
Finally, optionally multiple --database_replica
conncetions can be specified which hecate
will use to load balance read traffic accross, alleviating capacity on the master db for write operations.
JSON Validation
By default Hecate will allow any property on a given GeoJSON feature, including nestled arrays, maps, etc.
A custom property validation file can be specified using the schema flag.
Example
Note hecate currently supports the JSON Schema draft-04. Once draft-06/07 support lands in valico we can support newer versions of the spec.
Custom Authentication
By default the Hecate API is most favourable to a crowd-sourced data server. Any users can access the data/vector tiles, users can create & manage data, and admins can manage user accounts.
This provides a middle ground for most users but all endpoints are entirely configurable and can run from a fully open server to fully locked down.
If the default values aren't suitable for what you intend, passing in an authentication configuration JSON document will override the defaults.
Example
cargo run -- --auth path/to/auth.json
Contents of auth.json
{
"endpoints": {
"server": "public",
"schema": null,
"mvt": {
"get": "user",
"regen": "admin",
"meta": null
},
"users": {
"info": "admin",
"create": "admin",
"create_session": null
},
....
}
}
It is important to note that if custom authentication is used, every category must be either disabled or have an option for every sub category within it set. One cannot conditionally override only a subset of of the default options. This is for the security of private servers, since adding a new API endpoint is a non-breaking change, the server checks that you have specified a policy for every endpoint or are happy with just the defaults before it will start.
IE:
The below schema is invalid. Each category (schema, user, style) etc. must be specified as disabled or have a map containing the auth for each subkey.
{
"endpoint": {
"schema": null
}
}
Behavior Types
Type | Description |
---|---|
"public" |
Allow any authenticated or unauthenticated user access |
"admin" |
Allow only users with the access: 'admin' property on their user accounts access |
"user" |
Allow any user access to the endpoint |
"self" |
Only the specific user or an admin can edit their own metadata |
"null" |
Disable all access to the endpoint (Must be explicitly null |
Endpoint Lookup
Example Endpoint | Config Name | Default | Supported Behaviors | Notes |
---|---|---|---|---|
GET /api |
server |
public |
All | |
Server Meta | meta |
null |
2 | |
GET /api/meta |
meta::list |
public |
All | |
GET /api/meta/<key> |
meta::get |
public |
All | |
POST /api/meta/<key> |
meta::set |
admin |
user , admin , null |
|
JSON Schema | schema |
null |
2 | |
GET /api/schema |
schema::get |
public |
All | |
Custom Auth JSON | auth |
null |
2 | |
GET /api/auth |
auth::get |
public |
All | |
Mapbox Vector Tiles | mvt |
null |
2 | |
`DELETE /api/tiles | mvt::delete |
admin |
All | |
GET /api/tiles/<z>/<x>/<y> |
mvt::get |
public |
All | |
GET /api/tiles/<z>/<x>/<y>/regen |
mvt::regen |
user |
All | |
GET /api/tiles/<z>/<x>/<y>/meta |
mvt::meta |
public |
All | |
Users | user |
null |
2 | |
GET /api/users |
user::list |
user |
All | |
GET /api/user/info |
user::info |
self |
self , admin , null |
|
GET /api/create |
user::create |
public |
All | |
GET /api/create/session |
user::create_session |
self |
self , admin , null |
|
Mapbox GL Styles | style |
null |
2 | |
POST /api/style |
style::create |
self |
self , admin , null |
|
PATCH /api/style |
style::patch |
self |
self , admin , null |
|
POST /api/style/<id>/public |
style::set_public |
self |
All | |
POST /api/style/<id>/private |
style::set_private |
self |
self , admin , null |
|
DELETE /api/style/<id> |
style::delete |
self |
self , admin , null |
|
GET /api/style/<id> |
style::get |
public |
All | 1 |
GET /api/styles |
style::list |
public |
All | 1 |
Deltas | delta |
null |
2 | |
GET /api/delta/<id> |
delta::get |
public |
All | |
GET /api/deltas |
delta::list |
public |
All | |
Data Stats | stats |
public |
All | |
GET /api/data/stats |
stats::get |
public |
All | |
GET /api/data/bounds/<id>/stats |
stats::bounds |
public |
All | |
Features | feature |
null |
2 | |
POST /api/data/feature(s) |
feature::create |
user |
user , admin , null |
|
GET /api/data/feature/<id> |
feature::get |
public |
All | |
GET /api/data/feature/<id>/history |
feature::history |
public |
All | |
POST /api/data/feature(s) w/ force` |
feature::force |
admin |
user , admin , null |
|
Clone | clone |
null |
2 | |
GET /api/data/clone |
clone::get |
user |
All | |
GET /api/data/query |
clone::query |
user |
All | |
Bounds | bounds |
null |
2 | |
GET /api/bounds |
bounds::list |
public |
All | |
GET /api/bounds/<id> |
bounds::get |
public |
All | |
POST /api/bounds/<id> |
bounds::create |
admin |
All | |
DELETE /api/bounds/<id> |
bounds:delete |
admin |
All | |
OpenStreetMap Shim | osm |
null |
2 | |
GET /api/0.6/map |
osm::get |
public |
All | 3 |
PUT /api/0.6/changeset/<id>/upload |
osm::create |
user |
user , admin , null |
3 |
Notes
- This only affectes
public
styles. Theprivate
attribute on a style overrides this. Aprivate
style can never be seen publicly regardless of this setting. - This is a category, the only valid option is
null
this will disable access to the endpoint entirely - OSM software expects the authentication on these endpoints to mirror OSM. Setting these to a non-default option is supported but will likely have unpredicable support when using OSM software. If you are running a private server you should disable OSM support entirely.
API
GET
/
HTTP Healthcheck URL, currently returns Hello World!
Example
View the Admin Interface in your browser by pointing to 127.0.0.1:8000/admin/index.html
GET
/api
Return a JSON object containing metadata about the server
Example
Note: Analyze stats depend on the database having ANALYZE
run.
For performance reasons these stats are calculated from ANALYZEd stats
where possible to ensure speedy results. For more up to date stats,
ensure your database is running ANALYZE
more often. This can be done
manually in the database or by using the /api/data/stats/regen
API.
GET
/api/data/stats
Return a JSON object containing statistics and metadata about the geometries stored in the server
Example
GET
/api/data/stats/regen
Perform an ANAYLZE
call on the geo
table to update
the global stats.
Example
GET
/api/styles
Return an array containing a reference to every public style
Example
GET
/api/styles/<user id>
Return an array containing styles owned by a particular user.
By default any request will only return the public styles for a given user.
If an authenticated user requests their own styles, it will return their public and private styles.
Options
Option | Notes |
---|---|
<user id> |
REQUIRED Numeric ID of the user to get styles from |
Example
Return only public styles of user 1
User requesting their own styles will get public & private styles
POST
/api/style
Create a new private style attached to the authenticated user
Example
DELETE
/api/style/<id>
Delete a particular style by id. Users must be authorized and can only delete styles created by them.
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given style to delete |
Example
GET
/api/style/<id>
Get a particular style by id, public styles can be requested unauthenticated, private styles can only be obtained by the corresponding user making the request.
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given style to download |
Example
PATCH
/api/style/<id>
Update a style - auth required - users can only update their own styles
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given style to download |
Example
POST
/api/style/<id>/private
Update a public style and mark it as private.
Note: Once a style is public other users may have cloned it. This will not affect cloned styles that were made when it was public.
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given style to download |
Example
POST
/api/style/<id>/public
Update a style to make it public.
It will then appear to all users in the global styles list and other users will be able to download, clone, and use it
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given style to download |
Example
GET
/api/schema
Return a JSON object containing the schema used by the server or return a 404 if no schema file is in use.
Example
GET
/api/auth
Returns a JSON object containing the servers auth permissions as defined by the default
auth rules or the custom JSON auth as defined in the Custom Authentication
section
of this guide
Example
DELETE
/api/tiles
Remove all tiles from the integrated tile cache
Example
GET
/api/tiles/<z>/<x>/<y>
Request a vector tile for a given set of coordinates. A Mapbox Vector Tile is returned.
Options
Option | Notes |
---|---|
<z> |
REQUIRED Desired zoom level for tile |
<x> |
REQUIRED Desired x coordinate for tile |
<y> |
REQUIRED Desired y coordinate for tle |
Example
GET
/api/tiles/<z>/<x>/<y>/meta
Return any stored metadata about a given tile.
Options
Option | Notes |
---|---|
<z> |
REQUIRED Desired zoom level for tile |
<x> |
REQUIRED Desired x coordinate for tile |
<y> |
REQUIRED Desired y coordinate for tle |
Example
GET
/api/tiles/<z>/<x>/<y>/regen
Allows an authenticated user to request a new tile for the given tile coordinates, ensuring the tile isn't returned from the tile cache.
Options
Option | Notes |
---|---|
<z> |
REQUIRED Desired zoom level for tile |
<x> |
REQUIRED Desired x coordinate for tile |
<y> |
REQUIRED Desired y coordinate for tle |
Example
GET
/api/users
Get a list of users (up to 100) or filter by a given user prefix.
Options
Option | Notes |
---|---|
filter |
Optional Desired search prefix for username |
limit |
Optional Optionally limit the number of returned results |
Example
GET
/api/user/create
Create a new user, provied the username & email are not already taken
Options
Option | Notes |
---|---|
username |
REQUIRED Desired username, must be unique |
password |
REQUIRED Desired password |
email |
REQUIRED Desired email, must be unique |
Example
GET
/api/user/session
Return a new session cookie and the uid
given an Basic Authenticated request.
Example
GET
/api/user/info
Allows an authenticated user to obtain information about their own account
Example
GET
/api/user/<id>
Obtain information about any user in the system by their numeric User ID.
Note the information returned is the same information that a user is able to
lookup about themself with the GET /api/user/info
endpoint.
Options
Option | Notes |
---|---|
<id> |
REQUIRED User ID to obtain user information of |
Example
PUT
/api/user/<id>/admin
Allows an admin to add another user to the admin pool.
Options
Option | Notes |
---|---|
<id> |
REQUIRED User ID to obtain user information of |
Example
DELETE
/api/user/<id>/admin
Allows an existing admin to remove another user from the admin pool.
Options
Option | Notes |
---|---|
<id> |
REQUIRED User ID to obtain user information of |
Example
GET
/api/data/clone
Return a Line-Delimited GeoJSON stream of all features currently stored on the server.
Note: All streaming GeoJSON endpoints will send the Unitcode End Of Transmission, EOT
(0x04
) on stream completion. This can be used to ensure that a stream did not exit early.
Example
GET
/api/data/query
Return a Line-Delimited GeoJSON stream of all features that match the given query.
The query must be a valid SQL query against the geo
table. Note that the geo
is
the only table that this endpoint can access. Only read operations are permitted.
Note: All streaming GeoJSON endpoints will send the Unitcode End Of Transmission, EOT
(0x04
) on stream completion. This can be used to ensure that a stream did not exit early.
IE:
SELECT count(*) FROM geo
SELECT props FROM geo WHERE id = 1
Options
Option | Notes |
---|---|
query=<query> |
SQL Query to run against Geometries |
limit=<limit> |
Optional Optionally limit the number of returned results |
Examples
Boundaries allow downloading data via a set of pre-determined boundary files.
GET
/api/data/bounds
Return an array of possible boundary files with which data can be extracted from the server with
Options
Option | Notes |
---|---|
filter |
Optional Desired search prefix for username |
limit |
Optional Optionally limit the number of returned results |
Example
GET
/api/data/bounds/<bounds>
Return line delimited GeoJSON Feature
of all the geometries within the specified boundary file.
Note: All streaming GeoJSON endpoints will send the Unitcode End Of Transmission, EOT
(0x04
) on stream completion. This can be used to ensure that a stream did not exit early.
Options
Option | Notes |
---|---|
<bounds> |
REQUIRED One of the boundary files as specified via the /ap/data/bounds |
Example
POST
/api/data/bounds/<bounds>
Create or replace a boundary with the given name.
Note: Boundaries must be a Polygon
or MultiPolygon
Feature GeoJSON.
Options
Option | Notes |
---|---|
<bounds> |
REQUIRED the name of the bounds to create or replace |
Example
DELETE
/api/data/bounds/<bounds>
Delete a bounds file with the given name.
Options
Option | Notes |
---|---|
<bounds> |
REQUIRED the name of the bounds to create or replace |
Example
GET
/api/data/bounds/<bounds>/stats
Return statistics about geometries that intersect a given bounds
Options
Option | Notes |
---|---|
<bounds> |
REQUIRED One of the boundary files as specified via the /ap/data/bounds |
Example
GET
/api/data/feature
Return a single GeoJSON Feature
given a query parameter
Options
Option | Notes |
---|---|
key=<key> |
Optional Key value to retrieve a given feature by |
point=<lng,lat> |
Optional Query for a single feature at the given point |
Example
GET
/api/data/feature/<id>
Return a single GeoJSON Feature
given its' ID.
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given feature to download |
Example
GET
/api/data/feature/<id>/history
Return an array containing the full feature history for the provided feature id.
Options
Option | Notes |
---|---|
<id> |
REQUIRED Numeric ID of a given feature to download |
Example
GET
/api/data/features
Return streaming Line-Delimited GeoJSON within the provided BBOX
Note: All streaming GeoJSON endpoints will send the Unitcode End Of Transmission, EOT
(0x04
) on stream completion. This can be used to ensure that a stream did not exit early.
Options
Option | Notes |
---|---|
bbox |
REQUIRED Bounding Box in format left,bottom,right,top |
POST
/api/data/feature
Auth Required
Create, Modify, or Delete an individual GeoJSON Feature
The Feature must follow format defined in Feature Format.
The feature also must contain a top-level String message
attribute describing the changes being made (The delta message)
Example
POST
/api/data/features
Auth Required
Create, Modify, and/or Delete many features via a GeoJSON FeatureCollection
The Features in the FeatureCollection must follow format defined in Feature Format.
The FeatureCollection also must contain a top-level String message
attribute describing the changes being made (The delta message)
Note that a mix of create
, modify
, and delete
operations are allowed
within each FeatureCollection
Example
GET
/api/deltas
Returns an array of the last limit
defined number of deltas (default: 20). with their corresponding metadata. Does not include geometric
data on the delta. Request a specific delta to get geometric data.
The deltas endpoint has 2 modes. The first is a fixed list of the last n
deltas. The second is listing deltas by time stamp. the query parameters
for these two modes are mutually exclusive.
Limit Options
Return the last n
deltas starting at the specified offset
.
Where n
defaults to 20 and can be up to 100 by utilizing the limit
parameter
Option | Notes |
---|---|
offset=<delta id> |
Returns the last n deltas before the given delta id |
limit=<limit> |
OPTIONAL Increase or decrease the max number of returned deltas (Max 100) |
Date Options
Return deltas between a given start
and end
parameter.
The start
parameter should be the most recent TIMESTAMP, while the end
parameter
should be the furthest back in time.
IE: start
> end
.
|---------|------|
Current start end
Time
- If both
start
andend
are specified, return all deltas by default - If
start
orend
is specified, return last 20 deltas or the number specified bylimit
Option | Notes |
---|---|
start |
OPTIONAL Return deltas after n time - ISO 8601 compatible timestamp |
end |
OPTIONAL Return deltas before n time - ISO 8601 compatible timestamp |
limit |
OPTIONAL Increase or decrease the max number of returned deltas (Max 100) |
Example
GET
/api/deltas/<id>
Returns all data for a given delta as a JSON Object, including geometric data.
Options
Option | Notes |
---|---|
<id> |
REQUIRED Get all data on a given delta |
Example
The primary goal of the hecate project is a very fast GeoJSON based Interchange. That said, the tooling the OSM community has built around editing is unparalleled. As such, Hecate provides a Work-In-Progress OpenStreetMap Shim to support a subset of API operations as defined by the OSM API v0.6 document.
Important Notes
- All GeoJSON types can be downloaded via the API and viewed in JOSM
- MultiPoints
- Are represented using an OSM
Relation
- The type will be
multipoint
- The member type will be
point
- Are represented using an OSM
- MultiLineStrings
- Are represented using an OSM
Relation
- The type will be
multilinestring
- The member will be
line
- Are represented using an OSM
- Uploading
Way
&Relation
types are not currently supported, attempting to upload them may produce undesirable results.
The following incomplete list of endpoints are implemented with some degree of coverage with the OSM API Spec but are likely incomplete/or written with the minimum flexibility required to support editing from JOSM. See the code for a full list.
GET
/api/capabilities
GET
/api/0.6/capabilities
Return a static XML document describing the capabilities of the API.
Example
GET
/api/0.6/user/details
Auth Required
Returns a static XML document describing the number of unread messages that a user has. Every n minutes JOSM checks this and displays in the interface if there is a new message, to cut down on errors it simply returns a 0 message response.
Example
PUT
/api/0.6/changeset/create
Auth Required
Create a new changeset and set the meta information, returning the opened id.
Example
GET
/api/0.6/changeset/<changeset_id>/upload
Auth Required
Upload osm xml data to a given changeset
Example
PUT
/api/0.6/changeset/<changeset_id>/close
Auth Required
Close a given changeset, preventing further modification to it
Example