# ogc-cql2
Rust implementation of the Open Geospatial Consortium (OGC) Common Query Language (CQL2) described [here](https://docs.ogc.org/is/21-065r2/21-065r2.html).
This library aims to fulfill **all** the requirements listed under [Conformance](https://docs.ogc.org/is/21-065r2/21-065r2.html#_conformance) section of the specs.
So far, users of this library will be able to:
* Parse CQL2 _Expressions_ in either Text- or JSON-encoded forms,
* Implement _Evaluators_ to process collections of _Resources_ (a.k.a features) against valid expressions.
* Make use of a comprehensive set of builtin _Functions_ to use in writing their expressions.
* Implement their own versions of _Functions_ and register them w/ _Evaluators_.
* Evaluate CQL2 _Expressions_ against records provided by _Data Sources_ through two traits: _Iterable_ and _Streamable_. So far implementations for _CSV_, _GeoPackage_, and _PostGIS_ data-sources are included.
Changes are tracked in [ChangeLog](CHANGELOG.md).
## Parsing Expression Grammar (PEG) + Typify
The text-encoded parser used in this project was generated by [`rust-peg`](https://github.com/kevinmehall/rust-peg) by manually translating the BNF source into input recognizable by that crate's `peg::parser!` macro. The reference BNF source is included in the [`doc`](doc/cql2.bnf) folder.
The JSON-encoded one started life from code generated by the [`cargo-typify`](https://crates.io/crates/cargo-typify) tool from the JSON Schema given in the Standard. However some limitations w/ that tool meant that manual intervention was required.
The modified JSON Schema input file is included w/ the source of this crate ([`cql2.json`](doc/cql2.json)) as well as a patch file ([`cql2.json.patch`](doc/cql2.json.patch)) showing the changes from the original copy.
## Change to the BNF Grammar
The BNF grammar specified in the specs limit the Well Known Text (WKT) representation of multi-point geometries to always enclose point coordinates w/in parenthesis as implied from the following rules...
```bnf
multipointTaggedText = "MULTIPOINT" ["Z"] multiPointText;
multiPointText = "(" pointText {"," pointText} ")";
pointText = "(" point ")";
point = xCoord yCoord [zCoord];
```
That meant that text like `MULTIPOINT(1 2, 3 4)` causes a syntax error forcing the user to write it instead as `MULTIPOINT((1 2), (3 4))`.
This implementation allows both forms.
## Deviation from the Stadard
The CQL2 specs state that _Dates_ should be considered as local with regard to time zones. This implementation however always assign them the UTC TZ.
## Conformance tests
The `tests` folder contains tests that implement most of the `Annex A: Abstract Test Suite (Normative)` conformance tests, grouped in folders mirroring the level-1 sections of the specs.
### Test data
The folder `tests/samples/data` contain 3 CSV files created/converted from the same named _Layers_ in the [GeoPackage][1] referenced in the Standard, as well as the GeoPackage DB/file itself.
Those CSV files were first created by exporting each _Layer_ to a CSV file using [DB Browser for SQLite Version 3.13.1][2], then converting the geometries to their WKT form and renaming their column `geom`.
The _GeoPackage_ DB/file is used for testing the _GeoPackage Data Source_ and the _Streamable Data Source_ trait with and without transforming the filter expression to SQL.
The other 2 folders next to `csv` in the same parent folder mirror the same data found [here][3].
As of version 0.4.0, this project supports _PostGIS_ enabled _PostgreSQL_ tables to act as _Data Sources_. Tests and benchmarks expect a database named `cql2`, w/ the _PostGIS_ extension, to be accessible at the URL stored in `$PG_URL` —see Configuration section. The contents of this database was populated from the _GeoPackage_ DB file referenced above using the `ogr2ogr` command line tool.
Finally, the `ne_110m_populated_places_simple` table's `start` and `end` columns were altered from `TIMESTAMPTZ` to be just `TIMESTAMP`.
### Tests not implemented yet
Some tests, which exercise filter expressions with AND, OR, and NOT including sub-expressions of 4 predicates are not implemented yet. Those are:
* A.6.6. Conformance Test 24: `/conf/accent-insensitive-comparison/logical`,
* A.7.3. Conformance Test 27: `/conf/basic-spatial-functions/logical`,
* A.9.10. Conformance Test 39: `/conf/spatial-functions/logical`,
* A.10.4. Conformance Test 43: `/conf/temporal-functions/logical`,
* A.11.2. Conformance Test 45: `/conf/array-functions/logical`,
* A.12.5. Conformance Test 50: `/conf/property-property/logical`,
* A.14.3. Conformance Test 54: `/conf/arithmetic/logical`,
### Test result pending request for clarification
The [issue here](https://github.com/opengeospatial/ogcapi-features/issues/1011) is at the time of this writing still unresolved.
For now, the relevant test (`a6/test_23.rs`) expected results have been amended according to the findings reported in the issue in question.
## Benchmarks
As of version 0.4.0, benchmarks using the `Criterion` crate were added.
There are three (3) groups: `parse`, `eval` and `sql` tracking parsing expressions, evaluating them, and transforming them to SQL for use w/ database based data sources respectively.
To run all the benchmarks do
```bash
$ cargo bench
```
To run a specific group; e.g. `eval`, do
```bash
$ cargo bench --bench eval
```
The final report will be generated in the local project's home under `target/criterion`. An HTML report index page should be generated in `report/index.html` w/in that folder.
## Configuration
To configure this library
1. Make a copy of the file `.env.template` and rename it `.env`.
2. Make sure `.env` is included in `.gitignore`.
3. Edit the contents of `.env` to suit your requirements and environment.
For now these are the environment variables that can be configured:
#### `DEFAULT_CRS`
_Coordinate Reference System_ (CRS) code to use when validating geometry coordinates. Defaults to `EPSG:4326` if/when undefined.
#### `DEFAULT_PRECISION`
_Precision_ (number of digits after the decimal point) to keep/use when processing coordinates. Defaults to `7` if/when undefined. For _WGS 84_ coordinates this translates to approx. `1.11` cm. accuracy when projecting them to _Web Mercator_.
For now only positive integers in the range `0..32` inclusive are allowed.
#### `RUST_LOG`
See <https://docs.rs/env_logger/latest/env_logger/#enabling-logging> for details.
#### `PG_URL`
To exercise the _PostGIS_ data sources, an accessible PostGIS enabled database named `cql2` must be configured. The URL to that database is expected to be of the form `postgresql://user:password@host:port` where `user` and `password` are the credentials of a valid authorized User, and `host` and `port` reflect the actual host IP/name and port where the DB engine is listening for connections.
### PostgreSQL connection pool parameters
#### `PG_MAX_CONNECTIONS`
Maximum number of connections the builtin pool will maintain. Default is `8`.
#### `PG_MIN_CONNECTIONS`
Minimum number of connections the builtin pool will maintain at all times. Default is `2`.
#### `PG_ACQUIRE_TIMEOUT_SECS`
Maximum amount of seconds to wait for a connection to be acquired. Default is `8`.
#### `PG_IDLE_TIMEOUT_SECS`
Maximum idle duration in seconds for individual connections. Default is `8`.
#### `PG_MAX_LIFETIME_SECS`
Maximum lifetime in seconds of individual connections. Default is `8`.
## Required external software
### GEOS
This library relies on _GEOS_ by virtue of its dependence on the [`geos`][5] crate. In my case the native installed version shows the following at the time this page was last updated...
```bash
Name : geos
Epoch : 0
Version : 3.14.1
Release : 1.fc43
Architecture : x86_64
Installed size : 4.1 MiB
Source : geos-3.14.1-1.fc43.src.rpm
From repository : updates
Summary : GEOS is a C++ port of the Java Topology Suite
URL : http://trac.osgeo.org/geos/
License : LGPL-2.1-only
Description : GEOS (Geometry Engine - Open Source) is a C++ port of the Java Topology
: Suite (JTS). As such, it aims to contain the complete functionality of
: JTS in C++. This includes all the OpenGIS "Simple Features for SQL" spatial
: predicate functions and spatial operators, as well as specific JTS topology
: functions such as IsValid()
Vendor : Fedora Project
```
### SQLite + Spatialite extension
This library, as of version 0.3.0, supports _GeoPackage_ database files. This requires an installed version of `sqlite` and `libspatial` binaries. The latter will provide the `mod_spatialite.so` extension usually found in `/usr/lib64/`. Here is the info about my test installation at the time of this page's last update...
```bash
Name : sqlite
Epoch : 0
Version : 3.50.2
Release : 2.fc43
Architecture : x86_64
Installed size : 1.8 MiB
Source : sqlite-3.50.2-2.fc43.src.rpm
From repository : fedora
Summary : Library that implements an embeddable SQL database engine
URL : http://www.sqlite.org/
License : blessing
Description : SQLite is a C library that implements an SQL database engine. A large
: subset of SQL92 is supported. A complete database is stored in a
: single disk file. The API is designed for convenience and ease of use.
: ...
Vendor : Fedora Project
...
Name : libspatialite
Epoch : 0
Version : 5.1.0
Release : 11.fc43
Architecture : x86_64
Installed size : 15.3 MiB
Source : libspatialite-5.1.0-11.fc43.src.rpm
From repository : fedora
Summary : Enables SQLite to support spatial data
URL : https://www.gaia-gis.it/fossil/libspatialite
License : MPL-1.1 OR GPL-2.0-or-later OR LGPL-2.0-or-later
Description : SpatiaLite is a a library extending the basic SQLite core in order to
: get a full fledged Spatial DBMS, really simple and lightweight, but
: mostly OGC-SFS compliant.
Vendor : Fedora Project
```
## TODO
In no particular order...
- [ ] Add an option to the `repl` command line tool to output valid expressions as SQL WHERE clauses.
- [ ] Investigate implementing basic spatial operators for 2D geometries in pure Rust; i.e. removing dependence on the `geos` crate.
- [ ] Implement missing conformance tests preferably after finding an external set of Test Vectors.
- [ ] Add more _Functions_.
- [ ] Implement pooling of _Evaluators_ à la DB connections pools.
- [ ] Refine the _Evaluator_ trait as the external interface to this library.
- [ ] Improve the way we handle user defined closures.
- [ ] Reduce allocation.
- [ ] Improve performance.
- [ ] Reduce code repetition by using more macros.
- [ ] Investigate alternative means for external clients to inject functions logic + metadata.
- [x] Implement more _Data Sources_ such as _Shapefiles_ ~~and _PostGIS tables_.~~
- [x] ~~Investigate ways of translating _Expressions_ to PostGIS clauses + views.~~
- [x] ~~The WKT parsing machinery entry-point is private. Make it public.~~ Done 2025-08-31.
- [x] ~~Properly manage + handle global configurable options such as default CRS bearing in mind how it may affect conformance tests.~~ Done 2025-09-02.
- [x] ~~Add an LRU to store commonly used CRSes.~~ Turns out Proj is not `Send` and hence cannot be cached in an LRU cache safely w/o introducing `unsafe` code.
## License
This product is licensed under the Apache License Version 2.0 —see [included copy](LICENSE) or [online](https://www.apache.org/licenses/LICENSE-2.0)
[1]: https://github.com/opengeospatial/ogcapi-features/blob/master/cql2/standard/data/ne110m4cql2.gpkg
[2]: https://github.com/sqlitebrowser/sqlitebrowser
[3]: https://github.com/opengeospatial/ogcapi-features/tree/master/cql2/standard/schema/examples
[4]: https://docs.geoserver.org/2.27.x/en/user/filter/function_reference.html
[5]: https://crates.io/crates/geos