Crate chtypes

Expand description

chtypes — ClickHouse’s own type system, per ClickHouse version, from Rust.

chtypes answers one question, exactly: if this row were inserted into this ClickHouse table on this ClickHouse version, what would happen? It answers it by running ClickHouse’s own C++ machinery — DataTypeFactory, ISerialization, ReadHelpers, evaluateMissingDefaults, the TTL algorithms, MergeTreeDataWriter::mergeBlock — vendored per release and linked behind the frozen chs_* C ABI. Nothing here reimplements a coercion rule, which is why the answers are exact by construction.

This crate is a peer SDK over that ABI, alongside Go, Python and TypeScript. The language-neutral contract is spec/ in this repository; where this crate and spec/ disagree, the spec wins and this is a bug.

use chtypes::{Format, Registry, NO_SETTINGS};

let registry = Registry::from_env_or_default()?;   // $CHTYPES_REGISTRY, else the per-user cache
let lib = registry.for_version("25.8")?;             // minor line or exact patch
let schema = lib.compile("ts DateTime, seq UInt8").compile()?;

let batch = schema.rows(
    Format::JsonEachRow,
    br#"{"ts":"2026-01-15 10:30:00","seq":256}"#,
    NO_SETTINGS,
)?;

println!("{} {:?}", batch.outcome, batch.rows[0].values);
for t in &batch.transformed {
    // seq: 256 -> 0, overflow_wrap, lossy — and ClickHouse returned success.
    println!("row {} {}: {} -> {} ({})", t.row, t.column, t.input, t.stored, t.reason);
}

§What this crate will not do

Never map Error::Unsupported onto a rejection or an acceptance. -2 (CODE_UNSUPPORTED) means “a real server might well have accepted this; I decline to guess”. Mapping it to a rejection manufactures an over-reject; mapping it to an acceptance manufactures an over-accept, which is the cardinal sin — rows stream to subscribers and then the insert fails.
Never infer one version’s answer from another’s. Behaviour is not monotonic: 25.10 rejects a mixed-type DEFAULT that 24.8 through 25.8 and 26.6 onward all accept; JSON is rejected on 24.8 and accepted from 25.3. Registry::for_version fails, naming what is loaded, rather than answering from the nearest artifact.
Never treat a per-row accepted as “stored”. A TTL-expired row is accepted per row and not stored per batch. BatchResult::transformed folds in the batch-level storage_transforms, and BatchResult::engine_rows — when present — is the stored truth, not BatchResult::rows.
Never route a value through a float. Settings values cross as strings and stored values stay raw JSON text; 18446744073709551615 must not become 18446744073709552000.
Never decode a stored value into a language type before comparing it. A ClickHouse String holds arbitrary bytes, so a stored rendering is RawText — bytes, with a fallible UTF-8 view — and never a String that silently carries U+FFFD where the value had bytes. See Value::text.

§Getting artifacts

An artifact is one ClickHouse release compiled behind the C ABI — 166–302 MB each, hours of C++ compute. Fetch prebuilt, signed ones with the crate’s own command (cargo install chtypes → chtypes fetch 25.8) or from Rust with ensure — the docs/fetch.md contract, behind the default-on fetch feature; scripts/fetch.sh is the reference implementation of the same chain. A local build lands in the same per-user cache (~/.cache/chtypes/artifacts/<os>-<arch>). Registry::from_search_path looks there, in $CHTYPES_REGISTRY and in the system locations, and names every place it looked when a line is missing (Error::ArtifactMissing); Registry::new loads one explicit directory.

§Platform

Unix only — the loader is dlopen. Linux is the shipping target; macOS is a development floor and not an oracle: its long double is 53-bit, so float parses diverge from a real server (the float corpus matches 395/395 on Linux and 0/395 on macOS). Any float expectation must come from a Linux artifact or a live server.

Re-exports§

pub use fetch::Action;
pub use fetch::EnsureOptions;
pub use fetch::Installed;
pub use fetch::ensure;

Modules§

fetch: Fetching, verifying and installing artifacts — docs/fetch.md, the contract every SDK implements identically. Behind the fetch feature (on by default).
reason: The stable reason strings. The harness groups on them, so these spellings are fixed by the spec and must not drift.

Structs§

BatchResult: The outcome of one request body, which may hold many rows.
Block: One body, parsed ONCE under one schema handle and one clock instant — see Schema::parse_block for the parse contract and Filter::eval for the evaluation side. The borrow IS the free-order enforcement, exactly as for Filter: a Block cannot outlive its Schema, and its Drop runs first by construction.
Column: One column of a compiled schema, as this build canonicalised it.
CompileRequest: A compile request, built with Library::compile. See the module docs for the common case and the settings-profile case side by side.
Computed: One MATERIALIZED column’s value: stored at insert, frozen thereafter.
DiscoveredColumn: One row of QUERY_TABLE_COLUMNS.
DocFlags: Which document GROUPS the per-row documents carry (revision 3; spec/c-abi.md §Document flags). The verdict channel — batch and per-row outcome/code/err, rows_read, rows_skipped, unsupported_settings, engine_rows, storage_transforms — is ALWAYS emitted and is not a flag.
Filter: One boolean SQL expression compiled against a Schema’s columns — see Schema::compile_filter for the compile contract and the lifetime argument (the borrow IS the free-order enforcement).
FilterResult: One crate::Filter::rows answer.
FilterRowError: One 'e' or 'd' row, itemized: the row’s 0-based index and the code and message verbatim — ClickHouse’s own for an error row, this library’s decline for a declined row.
Library: One dlopen’d vendored ClickHouse build.
Manifest: One artifact’s manifest.json.
RawText: ClickHouse’s own rendering of a value, as bytes.
Registry: Every artifact under one directory, indexed by version — or, built with Registry::from_search_path, the docs/fetch.md §1 search path opened one line at a time.
RegistryOptions: Options for a search-path registry (Registry::from_search_path_with). The default is the §1 search path for this host, UTC, and autofetch from CHTYPES_AUTOFETCH.
RowResult: The outcome of validating and coercing one row.
Schema: A schema compiled inside one specific version’s library.
ServerProfile: What discovery learns about one deployment: the exact release and the settings it runs changed from defaults. Cache one per deployment (or per tenant on bring-your-own-ClickHouse) and declare it.
Span: One row’s byte range inside an export payload: payload[off..off + len] IS that row’s complete serialized line, terminating \n included, and is itself a valid one-row body in the export format. Spans are index-aligned with BatchResult::rows; a non-accepted row (rejected / skipped / unsupported) carries {0, 0}. The concatenation of all non-zero spans reproduces the payload exactly, which is what lets batches merge by byte concatenation.
Substitution: One volatile DEFAULT this library resolved instead of the server.
Transform: One silent change: input 256 into UInt8 stored as 0.
Value: One coerced column value, as ClickHouse itself renders it.

Enums§

CompileMode: The compile MODE — mirrors enum chs_compile_mode in chtypes.h. Numeric values are part of the ABI, exactly like crate::Format.
DefaultKind: What a column’s DEFAULT clause is.
Error: Everything that can go wrong loading an artifact or asking it a question.
FilterOutcome: The CALL-level verdict of crate::Filter::rows — whether evaluation completed at all; per-row failures live in the verdicts, not here. The default (and the degradation for an outcome spelling this crate does not recognise) is Unsupported, never Rejected — the same vocabulary-drift rule as Outcome.
Format: The input encoding of a row, as the chs_format integer codes. The numbers are part of the ABI and must not be renumbered.
Outcome: The verdict on one row or one batch.
Verdict: One row’s answer from crate::Filter::rows. Two of the four states are ANSWERS and two are NOT, and the split is load-bearing: a caller enforcing visibility MUST fail closed (hide the row / fail the request) on Verdict::Error and Verdict::Decline — collapsing either into “false the answer” inverts fail-closed into fail-open under NOT, the measured leak class (spec/bindings.md §Revision 3). The default is Decline, so an unset or unknown verdict is fail-closed by construction.

Constants§

ABI_REVISION: The chs_* ABI revision this crate was written against — CHS_ABI_REVISION in include/chtypes.h.
AUTOFETCH_ENV: CHTYPES_AUTOFETCH=1 turns lazy fetch on for a search-path registry (Registry::from_search_path): opening a missing line runs crate::ensure first (docs/fetch.md §6). Off by default, because a production process must not begin a 250 MB download inside a request.
CODE_ARTIFACT_CORRUPT: CHTYPES_ARTIFACT_CORRUPT — any hash mismatch anywhere in the chain (§3).
CODE_ARTIFACT_MISSING: CHTYPES_ARTIFACT_MISSING — no installed artifact answers for the line (docs/fetch.md §7). The code every SDK shares for Error::ArtifactMissing.
CODE_ARTIFACT_PINNED: CHTYPES_ARTIFACT_PINNED — the release offers something other than what the lock file pins (§5).
CODE_ARTIFACT_UNPUBLISHED: CHTYPES_ARTIFACT_UNPUBLISHED — the release publishes nothing for the requested line or exact patch on this platform (§2).
CODE_ARTIFACT_UNTRUSTED: CHTYPES_ARTIFACT_UNTRUSTED — the release’s SHA256SUMS is unsigned or mis-signed (§3 step 0); nothing was downloaded around it.
CODE_SOURCE_UNREACHABLE: CHTYPES_SOURCE_UNREACHABLE — the source could not be reached, or was not consulted because the fetch was offline.
CODE_UNSUPPORTED: CHS_CODE_UNSUPPORTED from include/chtypes.h: “this build refuses to answer”, and never a real ClickHouse error code.
DEFAULT_TIMEZONE: The server timezone assumed for bare DateTime / DateTime64 columns.
FETCH_COMMAND: This SDK’s fetch command, as the “Install it:” line of Error::ArtifactMissing spells it (docs/fetch.md §6: the crate’s [[bin]], reached through cargo install chtypes).
NO_PARAMS: An empty query-parameter map, for the common Schema::compile_filter call with no {name:Type} parameters. Positionally, like the settings slice Schema::rows and Schema::set_engine already take — the crate’s one-optional-parameter convention (spec/bindings.md §One compile function).
NO_SETTINGS: An empty settings map, for the common call with no per-request settings.
QUERY_CHANGED_SETTINGS: Every query setting the deployment runs at a NON-default value — the whole declared profile, from the server itself. One row per setting: {"name":"flatten_nested","value":"0"}. system.settings.value is already a String; the parsed pairs feed crate::Library::compile’s .settings(...) and per-call settings verbatim.
QUERY_SERVER_VERSION: QUERY_SERVER_VERSION’s result: the deployment’s exact release — the string crate::Registry::for_version resolves (minor line or exact patch both work). One row: {"version":"25.8.28.1"}.
QUERY_TABLE_COLUMNS: One existing table, in declaration order, with everything a column-declaration list needs — INCLUDING default_kind and default_expression, without which a reconstructed schema silently loses its DEFAULT/MATERIALIZED semantics. Uses ClickHouse’s own query parameters: send param_db / param_table (HTTP) or bind {db}/{table} (native).
REGISTRY_ENV: The environment variable a host may point at a registry directory.
SETTING_CLOCK_OFFSET_NANOS: The caller’s measured (server - client) offset, added to every clock read.
SETTING_DEFAULT_EVAL_MEMORY_BYTES: Admission ceiling on DEFAULT/TTL evaluation memory. Process-wide: settable only through Library::set_default_settings.
SETTING_DEFAULT_EVAL_WALL_NANOS: Admission ceiling on DEFAULT/TTL evaluation wall time. Process-wide, as above.
SETTING_MAX_CLOCK_SKEW_NANOS: Refuse to substitute a volatile DEFAULT when |offset| exceeds this; 0 means no budget, and with none of these set the tolerated skew is unbounded.
SETTING_NOW_EPOCH_NANOS: Pin the batch instant outright — tests, replay, anything that must be reproducible. The value is a 19-digit nanosecond epoch and must cross as a string: as a JSON number through a float it becomes 1.7e+18 and the setting is silently ignored.
SYSTEM_ARTIFACT_ROOTS: The system registry roots, docs/fetch.md §1 item 4 — searched after the per-user cache, never written by fetch. <os>-<arch> is appended.

Functions§

cache_dir_for: The per-user artifact cache for one platform — ${XDG_CACHE_HOME:-~/.cache}/chtypes/artifacts/<platform> (docs/fetch.md §1 item 3). Where fetch installs, where a core-repository build lands.
default_registry_dir: The per-user artifact cache for this host — ${XDG_CACHE_HOME:-~/.cache}/chtypes/artifacts/<os>-<arch>, <arch> spelled the artifact way (amd64, arm64). Where scripts/fetch.sh and crate::ensure install, where a core-repository build lands, and what every SDK’s tests and playgrounds fall back to when $CHTYPES_REGISTRY is unset — one directory the four SDKs agree on. A path, not a promise: Registry::new still errors if nothing is there.
host_platform: This host’s platform key, <os>-<arch> in the artifact spelling: linux or darwin, arm64 or amd64 (docs/fetch.md §1).
install_dir: Where fetch writes for this host: the first of explicit, $CHTYPES_REGISTRY and the per-user cache that is set — never a system location (docs/fetch.md §1).
install_dir_for: install_dir for an arbitrary platform key; see search_path_for for why $CHTYPES_REGISTRY only counts for the host platform.
installed_lines: Every minor line installed somewhere on dirs, each with the FIRST directory that holds it (the one lookup would take), in numeric release order. A subdirectory counts when it holds a manifest.json.
locate: The first directory on this host’s search path that holds line — <dir>/<minor>/manifest.json exists — returned as <dir>/<minor>. None when no directory does. line may be a minor line or an exact patch; the minor is what is looked for, as Registry::for_version resolves it.
locate_in: locate over an explicit list of registry directories, in order.
parse_changed_settings_result: Read QUERY_CHANGED_SETTINGS’ JSONEachRow body.
parse_columns_result: Read QUERY_TABLE_COLUMNS’ JSONEachRow body, in the query’s ORDER BY position order.
parse_version_result: Read QUERY_SERVER_VERSION’s JSONEachRow body.
reconstruct_ddl: Turn QUERY_TABLE_COLUMNS’ rows back into the column-declaration list crate::Library::compile takes. It is a spelling exercise, not a semantic one: types and expressions are the server’s own text, passed through verbatim, and the library’s own compile is the judge of the result.
registry_search_path: The docs/fetch.md §1 search path for this host, in order: explicit, $CHTYPES_REGISTRY, the per-user cache, then the system locations (SYSTEM_ARTIFACT_ROOTS). Unset entries are absent; directories that do not exist are kept, so the §7 message can name every place looked in.
search_path_for: registry_search_path for an arbitrary platform key. $CHTYPES_REGISTRY names this host’s registry and joins the path only for the host platform; a foreign platform (Linux artifacts fetched on a Mac, for a container) is looked for in its own cache and system directories.

Type Aliases§

Result: Result with this crate’s Error.

Crate chtypes

Crate chtypes Copy item path

§What this crate will not do

§Getting artifacts

§Platform

Re-exports§

Modules§

Structs§

Enums§

Constants§

Functions§

Type Aliases§

Crate chtypes