# admin_level
the `admin_levels` table and everything around it: a country, a state, a city, a neighborhood, a
street — each one a named area with a shape, sitting somewhere on a scale. the ddl, the queries and
the upsert live in `repository`, the rtree in `spatial_index`, the three stages that produce the
rows behind `admin_level::extract(level)`, and the connection lifecycle in `src/database/mod.rs`
calls into the first two; `src/database/admin_levels.rs`, `src/index/coordinates.rs` and
`src/extract/admin_levels/` are gone.
## the scale — `level`
the scale every named area sits on. osm tags levels 1..11 on boundary relations; geolite extends it
with 12 for streets, which are mapped as ways rather than boundaries. a house number is not a
level: a query answers it beside the ladder, in the `house_number` of the match.
| 1 | continent | | 7 | municipality |
| 2 | country | | 8 | city |
| 3 | region | | 9 | locality |
| 4 | state | | 10 | neighborhood |
| 5 | district | | 12 | street |
| 6 | county | | 14 | address |
**the set is closed on the way in.** `new` accepts only the levels above, and everything that
writes a level goes through it: the entity carries a `level`, the presets list `level`s, and
`--admin-level 11` is refused where it is parsed instead of travelling downstream as a number that
would quietly extract nothing. `name()` returns `&'static str`, not an `Option` — there is no
"unknown" to return, because an unnamed level cannot be constructed.
what comes back out is a `level` too: every row the repository returns carries
`admin_level: level`, read through `level_of`, which skips a row whose stored value is outside the
scale with a warning — a defence, since no release ever wrote one. `u8` survives in exactly two
places, both of them edges: the `admin_levels.admin_level` column (and the tantivy term that mirrors
it) and the `level` field of the json response. a level outside the scale is refused where it is
read: `--admin-level 11`, `--last-admin-levels 11` and `?last_admin_levels=11` all answer with an
error instead of extracting or filtering nothing; `level::parse` is the one reading of a level
from text.
ordering is by the level value — a higher level is more specific, so a street sorts after the city
that contains it — and it is implemented explicitly rather than derived, so that moving a variant
cannot silently change it.
## the identity — `admin_level_id`
an area's id is the osm id of the way or relation it came from, shifted left one bit, with the
low bit set for relations: a way and a relation that share an osm id stay distinct, the id is a
pure function of the source and never an insert-order rowid, and `osm_id()` and `kind()` read it
back. the row carries it from the stage that builds it (`from_way` or `from_relation`), so a row
without an origin cannot be built; `batch_upsert` writes the `relation_id`/`way_id` columns back
from `kind()` and `osm_id()`. a street folded from several ways (`optimize-merge-admin-levels`, in
`admin_level_hierarchy`) keeps the id of the smallest of them and its `way_id`, and the rows of the
others are gone.
the ways stay named in `merged_way_ids`, a jsonb array aligned by position with the lines of the
geometry: element `i` is the osm way line `i` came from, the street's own way first. it is `NULL` on
a row that was never folded, and there is no relation counterpart because only ways fold. it
describes the `wkb`, so the two are written by the same statement: `batch_upsert` writes `NULL`
beside the geometry and `batch_upsert_folded`, which only the street merge calls, writes the ways,
and a row a stage writes again loses the fold and its trace together. `add_merged_way_ids` gives the
column to a database built before it on the next writable open, and `merged_way_ids_by_ids` answers
nothing over a read-only database that never got it, so a query keeps answering there.
## the shape — `geometry`
`admin_geometry` is the `wkb` column: spatialite's blob layout on the way in, decoded with
`SpatiaLiteWkb` on the way out, because geozero's writer omits the byte-order byte and uses its own
sub-geometry separator, which the ISO WKB reader rejects. a blob that cannot be read degrades to an
empty geometry with a warning instead of failing the query. `mbr_center` reads the centre of the
blob's MBR header without decoding the geometry, the shortcut the house-number stage snaps with.
`bounding_box` is the envelope the rtree indexes; it moved here from `query` because the
persistence imported it, and a repository importing from the query layer is the wrong direction.
`bounding_geometry` is the region of `--bounding-wkt` and of the http `bounding_wkt`: the polygon
for the exact test, its `bounding_box` for the rtree and the `wkt` it was parsed from, the text
the etag of an http answer is derived from. `parse_bounding_wkt` is its one parse, the cli
`value_parser` and the http check alike, and it takes only a polygon or a multipolygon.
`mbr_of` reads the box of a blob without decoding it. `nearby_pairs` finds which lines touch or come
within a reach of each other, and `fold_lines` puts the lines of the members of a piece in one
multi-line — always a multi-line, each line as its way has it, never stitched and never dropped —
and answers the way of each line beside it: the two pieces of the street merge. a way the geometry
already holds is told by its id, not by its coordinates, and is not folded again. `merged_way_ids`
is that answer and the type of the column: json text on the way in, under `JSONB(?n)`, and read back
through `JSON(merged_way_ids)`.
## the rtree — `spatial_index`
`admin_levels_rtree` is a second table subordinate to the first, the same arrangement
`osm_pbf_blob_chunks` has with `osm_pbf_files`: a box means nothing without the row it bounds. it
is a virtual table with no indexes of its own, dropped and recreated by every run, which is why it
has functions rather than an `impl table`. `run` pages through every row with a geometry and
inserts one box per row; on a file database up to eight readers scan disjoint id ranges in
parallel while the connection that owns the table writes, and an in-memory database is scanned on
the calling thread, because `conn.path()` is empty for it and a worker could not reopen it.
the rtree's two reads live here too. `nearest(conn, point, envelope)` is what the coordinate
service asks: the rtree narrows the streets to a window of 0.1° around the point inside the
envelope, each candidate's closest point is measured on the ground, and the answer comes most
specific level first, closest first within a level, and by id when two are at the same distance; a
street whose blob is missing, is not a line, or is empty is dropped, and the debug output counts
why. `ids_in_bounding_box` is the region filter of the text search. neither `address` nor
`repository` writes sql over the rtree any more.
## the extraction — `extract`
`admin_level::extract(conn, level, opts, on_event)` is the one way rows get into the table. the
domain decides what a level is made of; the caller only watches:
| any other | `relations` | boundary relations tagged `admin_level=N`, assembled into areas |
| 10 | `relations`, then `place_ways` | the same, then ways tagged `place=neighbourhood` or `suburb` |
| 12 | `streets` | every named way the exclude rules let through |
`stages_of(level)` returns that list as `stage { level, source, ordinal, of }`, and `extract` runs
it in order, emitting an `extract_event { stage, step }` at each step: `started`; `candidates`,
from relations stages only, with how many relations the level has and how many are not extracted
yet; `progress(progress_report)` after every batch; `finished`. an event carries its stage, so a
consumer needs no state to know what it is looking at; `finished` carries how many rows the stage
kept, which is what the cli prints, the candidates being what the bar counts. the cli's renderer
keeps one bar per stage and prints the lines it always printed. `extract_opts` carries the thread
count of the relations stage, the `--name-priority` override, the name tags the loaders select and
the countries of the base.
## the country of a row — `country_index`
every row is written under the preset of the country whose boundary holds it
(`presets::for_country`), and carries that country in `country_iso_code`, so the index and the
query read the preset back from the row, the default one for a country without a preset of its
own. the boundary is the country's, compiled into the binary, one per country of the geofabrik
index ([`presets/boundaries`](../presets/readme.md)), never the level 2 relation of the extract:
a sub-region extract carries no closed boundary of its country, and a clipped relation never closes
into a polygon, so a base built from either would have gone under the default preset. `country_index`
is built from the boundaries once per run and asks, for every element of every level, which boundary
holds its centroid; then its first vertex, when the centroid falls in no country, a concave shape
or one drawn along a border; then the nearest boundary, whatever the distance, because an island
or a street over the water can still fall outside every outline, and each of them belongs to the
country closest to it. so every row has a country. the outlines are the ones of the geofabrik
extracts, each drawn a few kilometres into its neighbours, so a point near a border is inside two
of them: it goes to the country whose outline is farthest from it, the one it is deeper inside,
which is the true border when both outlines run the same distance past it. the point test is the
one of the hierarchy resolver, in `point_in_polygon`. the query reads the same index, built once
per process (`shared`), to send a coordinate to the base of the country whose outline holds it:
the rule that wrote the row is the one that finds it.
the country decides two things before the row is written (`extract::identity_of`): whether the
level is one the country keeps at all (`admin_levels`, the reason a brazilian `region` or
`municipality` is never a row), and which name tag names the row (`name_priority`, `name:pt`
before `name` in brazil); the loaders select every tag any country asks for (`presets::name_tags`)
and the choice is made in rust, per element. the row then goes to the base of its country
(`database::store::base_pool`, one `<iso>/data.sqlite3` per country in the data path), whatever the
source: a country, a continent, a local file and a url write every base they touch, but one a
build sealed, which takes no row: its rows are dropped and counted (`dropped N street — the base of
their country is sealed`). a resume subtracts from the candidates what the bases already hold at
the level.
what the cli keeps is the run, not the level: the `--recreate` wipe before the loop, and after the
last level the index and the count on the ledger — the index is created once at the end because
maintaining it through the upserts is the expensive way round.
each stage reads its candidates through the element repositories (`osm_relation::repository`,
`osm_way::repository`) and writes through `repository::batch_upsert`. the relations stage assembles
the member ways of each relation into rings (`geometry::assemble_rings`), closes each ring that
comes back to its start into a polygon wound clockwise, the way spatialite's `st_buildarea` does,
and falls back to a multi-line when nothing closes; a place way closes into one polygon by the same
rule; a street is a line, even when the way is a ring, and always a multi-line, one line per way,
once its ways are folded.
## the rules — `rules`
`rules_of(level)` answers the include and the exclude filters of a level, the defaults below. a
preset cannot override them: a filter runs in sql over every way before any element is read, where
the country of a way is not known yet, and no country has asked for a rule of its own. relations
stages take no rules: they are selected by the `admin_level` tag alone. every source drops
elements without a `name`, because a nameless area cannot produce useful data.
| any other, and 10 | relations tagged `admin_level=N` | the simplest selection; neighbourhood boundaries are mapped as relations too | | |
| 10 | ways tagged `place=neighbourhood` | the simplest selection | | |
| 10 | ways tagged `place=suburb` | in some regions suburbs are the de-facto neighbourhood unit when `place=neighbourhood` is not mapped | | |
| 12 | every way | the simplest selection | ways tagged `place=neighbourhood` or `place=suburb` | already captured at level 10 |
| 12 | | | ways tagged `leisure=park`, `building` or `waterway` | noise in street-level data |
| 12 | | | ways tagged `boundary` | a border drawn on the map, named in the languages of both sides, never a street |
| 12 | | | ways tagged `location=underwater` | a submarine cable or pipeline runs under the sea from one country to another, never a street |
the filter vocabulary is `osm_way::way_filter`: `rules` selects with it, and only the way's
repository knows what each variant means in sql. this table is the reference for those defaults:
a change to `rules.rs` changes it too (CLAUDE.md, rule 8).