Nativeclick

Fork of Protryon/klickhouse (master branch).

Nativeclick is a pure Rust client for ClickHouse over the native TCP protocol, for async (tokio) applications, with derive-based rows and minimal boilerplate.

Versioning and compatibility

• Version scheme: the crate version follows the ClickHouse version it targets, 0.MAJOR.MINOR. 0.26.9 targets ClickHouse 26.9.
• Supported servers: each release supports the targeted version and the 10 releases before it. For 0.26.9 that is ClickHouse 25.11 to 26.9, and CI runs the whole test suite against each of them.
• Protocol: the client speaks native protocol revision 54493, and each connection negotiates min(client, server). Every revision-dependent field and column format follows the negotiated revision, so older servers keep working.
• Rust: MSRV is 1.89, tested in CI.

See CHANGELOG.md for the changes since klickhouse and how to migrate.

nativeclick vs klickhouse

Compared with klickhouse 0.15 (the fork point, protocol revision 54448):

Decoding speed of this crate's decoder before and after the 0.26.9 optimizations. Measured with examples/throughput.rs (2M rows × 5 columns, ClickHouse 26.9 on localhost, ZSTD answers):

Getting started

[dependencies]
nativeclick = "0.26.9"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

use nativeclick::{Client, ClientOptions, Row};

#[derive(Row, Debug, Clone)]
struct Event {
    id: u64,
    name: String,
    tags: Vec<String>,
    score: Option<f64>,
}

#[tokio::main]
async fn main() -> nativeclick::Result<()> {
    let client = Client::connect("127.0.0.1:9000", ClientOptions::default()).await?;

    client
        .execute("CREATE TABLE IF NOT EXISTS events (id UInt64, name String, tags Array(String), score Nullable(Float64)) ENGINE = MergeTree ORDER BY id")
        .await?;

    client
        .insert_native_block(
            "INSERT INTO events FORMAT Native",
            vec![Event { id: 1, name: "start".into(), tags: vec!["a".into()], score: None }],
        )
        .await?;

    let events: Vec<Event> = client.query_collect("SELECT * FROM events").await?;
    println!("{events:?}");
    Ok(())
}

More in the examples.

Queries

Rows are any struct with #[derive(Row)], RawRow (dynamic access by index or name), or UnitValue<T> for a single column. Field names match column names, and can be adjusted with #[nativeclick(rename = "...")], rename_all, default, flatten, nested.

Per-query options

Client::with_options returns a handle on the same connection whose queries use the given options. The original handle is unchanged.

# async fn f(client: nativeclick::Client) -> nativeclick::Result<()> {
use std::time::Duration;
use nativeclick::{QueryOptions, Uuid};

let id = Uuid::new_v4();
let tuned = client.with_options(
    QueryOptions::new()
        .query_id(id)                          // visible in system.query_log, KILL QUERY
        .setting("max_threads", 4)             // any ClickHouse setting
        .setting("max_execution_time", 60)
        .param("min_id", 10)                   // {min_id:UInt64}
        .param("ids", "[1, 2, 3]")             // {ids:Array(UInt8)}, as clickhouse-client --param_ids
        .param_string("name", "it's \"quoted\"") // any text for a {name:String}, escaped for you
        .timeout(Duration::from_secs(30)),     // cancels the query on the server when exceeded
);
tuned
    .execute("SELECT count() FROM events WHERE id >= {min_id:UInt64} AND name != {name:String}")
    .await?;
# Ok(()) }

• Parameters are values sent separately from the SQL, so they cannot inject SQL.
  • param takes the text exactly as clickhouse-client --param_x=... does: numbers, dates, [1,2], ['a','b'].
  • param_string takes any text for a String parameter and escapes it.
• Timeouts fail the query with NativeclickError::Timeout and cancel it on the server, so the connection stays usable.
• Dropping a result stream before its end also cancels the query on the server.

Connection

# async fn f() -> nativeclick::Result<()> {
use std::time::Duration;
use nativeclick::{Client, ClientOptions};

let client = Client::connect(
    "127.0.0.1:9000",
    ClientOptions {
        username: "default".into(),
        password: String::new(),
        default_database: "default".into(),
        quota_key: "tenant-42".into(),
        connect_timeout: Some(Duration::from_secs(5)),
        ..Default::default()
    },
)
.await?;

let server = client.server_info();
println!("{} {}.{} ({:?}), protocol {}", server.name, server.version_major, server.version_minor, server.timezone, server.negotiated_revision);
client.ping().await?;
# Ok(()) }

A Client is cheap to clone and can be used from many tasks; queries on one connection run one after the other. For parallelism, use several connections, for example with the bb8 pool:

// features = ["bb8"] (and "tls" for with_tls)
let manager = nativeclick::ConnectionManager::new("127.0.0.1:9000", ClientOptions::default()).await?;
let pool = nativeclick::bb8::Pool::builder().max_size(8).build(manager).await?;
let client = pool.get().await?;

Connection-level failures are reported to every waiting call, and later calls on the same Client return the error that closed it. Client::is_closed() tells when to reconnect, and the bb8 pool does it for you.

Errors

NativeclickError is #[non_exhaustive] and Clone. The main variants:

Types

Notes:

• JSON and Dynamic formats: the client asks the server for JSON columns as text and for the flattened Dynamic layout, with two settings sent before the caller's own (which can override them).
• Writing Variant / Dynamic / Geometry:
  • a DynamicValue chooses the type explicitly;
  • a plain value goes to the first variant that accepts it (Variant) or to its natural type (Dynamic).
• Recent column formats: sparse and replicated columns, and the size-stream string layout (revision 54492), are decoded transparently.

Feature flags

• derive: the #[derive(Row)] macro. Default.
• compression: compression of client/server traffic: writes lz4, reads lz4 and zstd. Default.
• serde: serde support on values, and the Json<T> wrapper. Default.
• bfloat16: the BFloat16 type. Default.
• tls: TLS via tokio-rustls, for Client::connect_tls and the bb8 pool.
• bb8: a bb8 ConnectionManager.
• refinery: migrations via refinery.
• geo-types: conversions with the geo-types crate.
• rust_decimal: conversions with rust_decimal.

Running the tests

The integration tests need a ClickHouse server:

docker run -d --name clickhouse -p 19000:9000 -e CLICKHOUSE_SKIP_USER_SETUP=1 --ulimit nofile=262144:262144 clickhouse/clickhouse-server:26.9
export NATIVECLICK_TEST_ADDR=127.0.0.1:19000
# export NATIVECLICK_TEST_USER=default NATIVECLICK_TEST_PASSWORD= NATIVECLICK_TEST_DATABASE=default
cargo test --all-features -- --test-threads=1

The tests share one server and its tables, so run them on one thread.

To measure throughput:

NATIVECLICK_TEST_ADDR=127.0.0.1:19000 cargo run --release --example throughput

Credit

Nativeclick started as a fork of klickhouse by Protryon. nativeclick_derive was made by copy/paste/simplify of serde_derive to get maximal functionality and performance at the lowest time cost. In a prototype, serde was used directly, but this was abandoned because of the lock-in of serde's data model.