Crate clickhouse_dfe 

clickhouse-dfe

The official ClickHouse Rust client speaks HTTP and nothing else. Plenty of deployments expose port 9000 and no HTTP at all. JSON, Variant and Dynamic columns do not read. Inserting needs a struct known at compile time, which a loader taking arbitrary shapes off a topic does not have.

Extensions for the official client, as a separate crate depending on the published clickhouse release. Adds a native TCP transport, runtime-schema inserts, and one client that dispatches over either.

cargo add clickhouse clickhouse-dfe

Extension traits on the HTTP client

ping, kill_query, with_query_id, with_session_id, with_role, and ServerException::parse for a typed error carrying the server’s own code.

use clickhouse::Client;
use clickhouse_dfe::ClientExt;

let client = Client::default()
    .with_url("http://localhost:8123")
    .with_query_id("nightly-rollup");

client.ping().await?;

Native TCP

cargo add clickhouse-dfe --features tcp

Reads are column-typed, and JSON, Variant and Dynamic decode to their per-row document text.

use clickhouse_dfe::TcpClient;

let client = TcpClient::new("localhost:9000").with_database("default");
let blocks = client
    .query("SELECT number FROM system.numbers LIMIT 10")
    .fetch_blocks()
    .await?;

for block in &blocks {
    for n in block.column_as::<u64>("number")? {
        println!("{n}");
    }
}

Runtime-schema inserts

cargo add clickhouse-dfe --features dynamic

Rows arrive as serde_json::Map, column types come from system.columns. Binary on the wire either way – FORMAT RowBinary over HTTP, FORMAT Native over TCP – so the server does no JSON parsing.

Features

Feature	What it adds
ext	Extension traits on clickhouse::Client (default)
lz4	LZ4 for the HTTP transport, forwarded to upstream (default)
zstd	Zstd for the HTTP transport, forwarded to upstream
tcp	Native TCP transport: connection actor, pool, retry, wire codec
tls	rustls trust for the TCP transport (implies tcp)
dynamic	Runtime-schema inserts (implies ext)
unified	One client over both transports (implies tcp)
full	All of the above

Default is ext plus lz4, neither of which adds a dependency beyond upstream’s. tcp brings deadpool, socket2 and backon.

Documentation

Read	When
type-support	Checking whether a column type reads and writes
insert-formats	Asking why there is no JSONEachRow option
unified-client	Choosing a transport at runtime
coverage	Adding to a hot-path module

Upstream

Not a fork. The dependency on the published clickhouse release is the only link – no git, no path, no [patch]. Same edition, MSRV, rustfmt settings and lint set, so ClickHouse/clickhouse-rs can cherry-pick from here without a reformat or a relicence.

Licence

Apache-2.0. See LICENSE and NOTICE.

Context

Depth: docs/architecture.md.

What this is

Extensions to the official ClickHouse Rust client, in a separate crate that depends on the published clickhouse release. It adds a native TCP transport, runtime-schema inserts, and one client dispatching over either. Not a fork, not a rewrite, not a chart, not a schema tool: it runs the SQL it is handed and authors none, so it creates no database, table, view, role or Kafka topic.

Where things live

Path	Holds
src/lib.rs	crate root, the feature-gated module layers, and the README as the crate docs
src/ext.rs	extension traits on clickhouse::Client, plus the typed server-exception parser
src/native/	the Native wire codec – HOT PATH, every row in or out passes through it
src/tcp/	the TCP transport: connect, handshake, connection actor, pool, retry, protocol, reader, writer
src/dynamic/	runtime-schema inserts – the system.columns schema cache, encode, insert
src/unified.rs	UnifiedClient, the dispatch enum over both transports
src/tls.rs, src/worker.rs	rustls trust for TCP, and the internal primitive the connection actor uses
tests/wire_docker.rs	the type matrix against a pinned ClickHouse in Docker
tests/*_live.rs	ignored by default – they need a real cluster
docs/	coverage, insert formats, type support, the unified client, this architecture

Commands that prove a change

hyperi-ci check        # the full local gate, the same suite CI runs
make check-features    # cargo check --all-features, then --no-default-features

Three ways green lies, each recorded in the repo:

It looks like	It is actually	Where
-D warnings covers the crate	It reaches only the features: all set. The feature matrix runs cargo hack --each-feature --no-dev-deps check, and check does not fail on a warning, so feature-gated dead code and feature-conditional lints pass. Four accumulated in the default set before the review gate started linting it on 2026-09-02	.hyperi-ci.yaml
The live tests ran	CI never runs the eleven #[ignore]d tests in tests/*_live.rs. They need a real cluster through CLICKHOUSE_DFE_ENV_FILE, and nextest is given no --run-ignored	tests/*_live.rs
Coverage is held	It reports and does not gate. coverage: false is deliberate – hyperi-ci reads test.min_coverage for Python only. Measure out of band with cargo llvm-cov --all-features --all-targets --summary-only, keeping --all-targets, because a --lib-only run reads lower on the codec and invents regressions	docs/COVERAGE.md

What tends to bite

Don’t	Do	Why
Trust a unit test on a wire format	Add a row to tests/wire_docker.rs	The sparse kind-stack bug sat in code the unit tests executed happily – the fixtures agreed with the decoder because both were written from the same wrong reading of the spec. A real server writing the bytes caught it (0ee6201, docs/COVERAGE.md)
Raise the advertised protocol revision on its own	Consume the fields the new gates add, in the handshake and in Progress and ProfileInfo	88137d3 advertised 54473 and stalled against a real server until 64baef4 and aae05d3 read the added fields
Add #[ignore] to a test in tests/wire_docker.rs	Leave it running and let it skip when there is no container runtime	nextest has no --run-ignored here, so an ignored test in that file is one CI never runs (tests/wire_docker.rs:16)
Assume the default feature set is linted	Run make check-features, and lint the sets by hand	A lint ratchet macro hid 64 warnings until 66b271f retired it, and d49c721 then had to lint the sets nobody was linting
Start a container per test and leave it to the runner	Keep the suite in the docker-server nextest group	The wire suite leaked a ClickHouse container per run until db68d25. nextest gives each test its own process, so an in-binary semaphore cannot serialise them, and seven servers at 2 GiB do not fit a build host (.config/nextest.toml)
Reach for git =, path = or [patch.crates-io] on clickhouse	Depend on the published version	The crates.io release is the only link, which is what lets files travel back to ClickHouse/clickhouse-rs without a reformat or a relicence (CONTRIBUTING.md)

Where this sits

dfe-stack suite --consumer clickhouse-dfe and --producer clickhouse-dfe both return this node with an EMPTY edge list – registered as a general-audience OSS library with default_in_pass: false. The one relationship that matters is upstream, and the graph does not carry it:

Repo	Direction	Kind	Mechanism
ClickHouse/clickhouse-rs	inbound	published crate, by version	Cargo.toml depends on clickhouse 0.15.2 with default-features = false, and on nothing from that repo’s git. Edition 2024, MSRV 1.89.0 and the lint set are kept in step with it so code can be cherry-picked in either direction

Re-exports

pub use error::Error;

pub use error::Result;

pub use tcp::TcpClient;tcp

pub use unified::Columns;unified

pub use unified::Transport;unified

pub use unified::UnifiedClient;unified

pub use ext::ClientExt;ext

pub use ext::ServerException;ext

Modules

dynamicdynamic

Insert rows whose shape is only known at run time.

error

Contains Error and corresponding Result.

extext

Extensions to clickhouse::Client, built only on its public API. Upstream flattens every rejection into Error::BadResponse text, so branching on an error code means reading it back out.

native

ClickHouse Native columnar block format – the format=Native payload encoding, independent of the transport that carries it.

tcptcp

ClickHouse TCP transport (native binary protocol, port 9000).

tlstls

TLS trust configuration for the TCP transport.

unifiedunified

One client over both transports. Reads land in Columns whichever transport answered.