Module arrow_tile

Expand description

Arrow-based tile payload format.

A tile’s payload is one or more layers. Each layer is a single Arrow RecordBatch serialised as an Arrow IPC stream. Geometry is encoded using the GeoArrow interleaved-coordinate convention so the payload can be consumed directly by GeoArrow-aware renderers (e.g. @geoarrow/deck.gl).

§Per-layer schema

column	type	notes
`id`	`UInt64`	feature id
`start_time`	`Int64`	Unix ms, absolute
`end_time`	`Int64`	Unix ms, absolute
`geometry`	GeoArrow point / linestring / polygon	interleaved f64 lon/lat
`vertex_time`	`List<UInt16>` deltas or `List<Int64>` (nullable)	per-vertex Unix ms (optional; see [`build_vertex_time_array`])
`vertex_value`	`List<Float32>` (nullable)	per-vertex scalar, e.g. SST (optional)
`triangles`	`List<UInt16>` or `List<UInt32>`	pre-baked earcut indices, feature-local (optional; polygon only)
`<property>`	`Float64` or `Dictionary<UInt16,Utf8>`	one column per property

All layers in one tile are concatenated with a tiny frame so a tile can carry, say, a linestring layer and a point layer side by side:

[u16 layer_count | ALIGNED_FRAME_FLAG]
  repeated: [u16 name_len][name utf8][u32 ipc_len][pad to 8][ipc stream bytes]

The leading u16’s top bit (ALIGNED_FRAME_FLAG) marks the aligned frame: zero padding after each ipc_len places every Arrow IPC stream at an 8-byte boundary relative to the payload start, so readers can hand the stream to an Arrow implementation zero-copy (Arrow buffers are 8-byte aligned within a stream; the stream itself must start aligned for that to survive). The pad length is not stored — readers derive it as (8 - pos % 8) % 8 from the position after ipc_len. Frames without the flag (every archive written before the flag existed) carry no padding and decode exactly as before.

Structs§

AttrQuant: Per-numeric-property quantization affine: value = o + q * s, where o is the column minimum (the dequantization offset) and s the requested ground precision (the step). Reconstruction is lossy to ≤ s/2; the reader applies the identical math (tile.ts).
ColumnarLayer: One decoded/encodable tile layer.
DecodedLayer: A decoded tile layer: its name and the raw Arrow RecordBatch.
EncoderConfig: Resolved, explicit encoder settings — the values the tile encoder reads at encode time (coordinate + attribute quantization, vector grouping, the point-elevation fold, vertex-time precision).
QuantAffine: Per-layer coordinate-quantization affine. Coordinates ship as i32 grid indices; the decoder reconstructs lon = x0 + qx*sx, lat = y0 + qy*sy. sx/sy (degrees per quantum) are sized from a target ground precision in meters at the layer’s mid-latitude, so the worst-case error is ≤ half a quantum (~meters/2). This trades GeoArrow self-describing Float64 for size (coords are the dominant, near-incompressible column) and is opt-in.
VectorGroup: A build-time directive to fuse several scalar numeric properties into one GPU-ready interleaved PropertyColumn::Vector. The component order is the vector’s component order (e.g. ["qx","qy","qz","qw"] → instanceQuaternions). Applied at encode time (like the quantization maps), so the producer keeps emitting plain scalar columns and the renderer still binds the result zero-copy. See set_vector_groups.

Enums§

GeometryColumn: Geometry for one layer, grouped by kind. Every feature in a layer shares one kind — the tiler emits a separate layer per geometry type.
PropertyColumn: A property column. Values are per-feature and may be missing.
VectorElem: Leaf element type of a PropertyColumn::Vector — the GPU upload type the renderer binds the decoded child buffer as.

Constants§

ALIGNED_FRAME_FLAG: Top bit of the layer frame’s leading u16: marks an aligned frame whose IPC streams are padded to 8-byte boundaries (see the module docs). The remaining 15 bits carry the layer count, so a tile may have at most 0x7fff layers. Old frames never set this bit (layer counts are tiny).
DEFAULT_VERTEX_TIME_MAX_STEP_MS: Default ceiling (ms) on the u16-delta vertex_time quantization step.
STT_QUANT_ATTR_META_KEY: Field-metadata key flagging a numeric property column that ships as fixed-point integer indices (smallest of UInt16/Int32) instead of Float64. Its value is the AttrQuant JSON (value = o + q*s). Sibling of STT_QUANT_META_KEY for geometry; lives on the property field, so a reader reconstructs Float64 the same way it does coordinates.
STT_QUANT_META_KEY: Schema-metadata key (on the geometry field) that flags a tile whose coordinates are fixed-point i32 grid indices rather than GeoArrow Float64 lon/lat. Its value is the QuantAffine JSON; absent ⇒ standard Float64.
TRIANGLES_METADATA_KEY: Schema-metadata key set on layers that carry pre-baked triangle indices.

Functions§

decode_layer: Decode a single-layer Arrow IPC stream into a RecordBatch.
decode_tile: Decode a full tile payload (the layer frame) into its layers.
encode_layer: Encode a single layer to an Arrow IPC stream.
encode_layer_quantized: encode_layer with optional fixed-point coordinate quantization.
encode_layer_with: encode_layer with a fully-explicit EncoderConfig — no process-wide globals are read. This is the concurrency- and multi-config-safe entry point (e.g. a dynamic server fronting several datasets with different settings).
encode_tile: Encode a full tile payload (one or more layers) with the layer frame.
encode_tile_quantized: encode_tile with optional fixed-point coordinate quantization applied to every layer (see encode_layer_quantized). quantize_m = None is byte-identical to encode_tile; the other encoder settings come from the process-wide globals.
encode_tile_with: encode_tile with a fully-explicit EncoderConfig — no process-wide globals are read. The concurrency- and multi-config-safe entry point a dynamic per-request tile server uses so each dataset/request encodes with its own settings without touching shared state.
point_elevation_column: The current point-elevation column name (empty ⇒ disabled).
quantize_attrs: The current numeric-property quantization map (a clone).
quantize_attrs_auto: Whether automatic numeric-property quantization is enabled.
quantize_coords_m: The build-global quantization precision in meters, or None when off.
set_point_elevation_column: Set the build-global point-elevation column name (see above). Empty disables.
set_quantize_attrs: Replace the build-global numeric-property quantization map.
set_quantize_attrs_auto: Enable/disable automatic range-adaptive quantization of every otherwise-raw Float64 numeric property (see [QUANTIZE_ATTRS_AUTO]). Explicit precisions in set_quantize_attrs always win.
set_quantize_coords_m: Set the build-global coordinate quantization precision in meters for every subsequent default encode_tile call. <= 0 (the default) turns it off — coordinates stay Float64 GeoArrow. See encode_layer_quantized for the size/precision trade-off.
set_vector_groups: Replace the build-global vector-group list.
set_vertex_time_max_step_ms: Override the u16-delta vertex_time step ceiling for every subsequent encode_layer call. Values below 1 ms clamp to 1 (every layer with a span beyond u16 milliseconds then takes the exact List<Int64> path).
tessellate_polygon: Tessellate one polygon feature (a list of rings) using earcut. Returns the flat triangle index list — each triple of indices is one triangle, indices are LOCAL (relative to the start of the feature’s coordinate run, where the exterior ring sits first followed by every hole).
vector_groups: The current vector-group list (a clone).
vertex_time_max_step_ms: The currently configured u16-delta vertex_time step ceiling (ms).

Type Aliases§

Coord: A single coordinate pair (lon, lat) in WGS84 degrees.