arrow-tiberius

arrow-tiberius bridges Apache Arrow and Microsoft SQL Server through the Tiberius TDS driver.

The v0.1 API focuses on Arrow-to-SQL Server writes:

• plan SQL Server-compatible schemas from Arrow schemas,
• render deterministic CREATE TABLE SQL,
• report unsupported mappings as structured diagnostics,
• write Arrow RecordBatch values with a selectable SQL Server bulk writer,
• emit sanitized writer and protocol tracing through tracing.

SQL Server-to-Arrow reads are reserved for a later release.

Install

[dependencies]
arrow-tiberius = "0.1"

Quick Start

Plan an Arrow schema and render SQL Server DDL:

use arrow_schema::{DataType, Field, Schema};
use arrow_tiberius::{
    MssqlProfile, PlanOptions, TableName, create_table_sql_from_mappings,
    plan_arrow_schema_to_mssql_mappings,
};

fn main() -> arrow_tiberius::Result<()> {
    let schema = Schema::new(vec![
        Field::new("id", DataType::Int64, false),
        Field::new("name", DataType::Utf8, true),
    ]);

    let outcome = plan_arrow_schema_to_mssql_mappings(
        &schema,
        MssqlProfile::sql_server_2016_compat_100(),
        PlanOptions::default(),
    )?;

    let table = TableName::new("dbo", "people")?;
    let ddl = create_table_sql_from_mappings(&table, outcome.value());

    assert!(ddl.contains("CREATE TABLE [dbo].[people]"));
    Ok(())
}

Write a batch to an existing SQL Server table:

use arrow_array::RecordBatch;
use arrow_tiberius::{
    MssqlProfile, PlanOptions, TableName, WriteBackend, WriteOptions,
    connect_mssql_client_from_ado_string, plan_arrow_schema_to_mssql_mappings,
};

async fn write_batch(
    connection_string: &str,
    batch: &RecordBatch,
) -> arrow_tiberius::Result<()> {
    let mut client = connect_mssql_client_from_ado_string(connection_string).await?;
    let outcome = plan_arrow_schema_to_mssql_mappings(
        batch.schema().as_ref(),
        MssqlProfile::sql_server_2016_compat_100(),
        PlanOptions::default(),
    )?;

    let table = TableName::new("dbo", "people")?;
    let mut writer = client
        .bulk_writer(
            table,
            outcome.value().to_vec(),
            WriteOptions {
                backend: WriteBackend::DirectRawBulk,
                ..WriteOptions::default()
            },
        )
        .await?;

    writer.write_batch(batch).await?;
    writer.finish().await?;
    Ok(())
}

The connected writer validates target table metadata before sending rows. It does not create the target table automatically; callers can use the DDL helper when they want this crate to produce the table definition.

Diagnostics

Planning and write failures return structured diagnostics instead of requiring string parsing. Diagnostics include severity, machine-readable code, field context, row context when available, and message text.

For the complete planning surface, see Arrow to SQL Server Type Mapping.

Writer Backends

WriteBackend controls how planned Arrow rows are sent to SQL Server:

The direct raw backend is the optimized production path for currently supported mappings. The baseline backend remains useful for compatibility checks and parity tests.

Observability

arrow-tiberius emits structured spans and events through tracing for schema planning, writer initialization, batch writes, direct raw backend summaries, and writer finish. It never installs a subscriber.

Its tiberius-raw-bulk dependency also emits sanitized protocol tracing under the tiberius_raw_bulk::protocol target. Those protocol events are emitted inside active arrow-tiberius writer spans during connect, bulk-load, and finish operations.

See Observability for subscriber setup, span and event names, safe field categories, redaction guarantees, and workflow integration.

Examples

Compile-checked examples that do not require SQL Server:

cargo run --example schema_to_ddl
cargo run --example planning_diagnostics
cargo run --example backend_selection
cargo run --example policy_dependent_planning

SQL Server write example:

ARROW_TIBERIUS_EXAMPLE_MSSQL_URL='server=tcp:localhost,1433;user=sa;password=...;TrustServerCertificate=true' \
  cargo run --example sqlserver_batch_write

By default, the SQL Server example creates, writes to, and drops [dbo].[arrow_tiberius_example_write].

Compatibility

The v0.1 profile targets SQL Server 2016 with database compatibility level 100:

use arrow_tiberius::MssqlProfile;

let profile = MssqlProfile::sql_server_2016_compat_100();

arrow-tiberius depends on the published tiberius-raw-bulk package as the crate name tiberius and owns that compatibility boundary internally:

tiberius = { package = "tiberius-raw-bulk", version = "=0.12.3-raw-bulk.14", default-features = false, features = [
    "tds73",
    "winauth",
    "native-tls",
] }

Downstream crates should normally depend only on arrow-tiberius and construct SQL Server clients through connect_mssql_client_from_ado_string or ConnectedMssqlClient.

Feature Flags

Validation

Default local validation does not require SQL Server:

cargo fmt --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace

Run SQL Server integration tests through the xtask harness:

cargo xtask sqlserver-test

Documentation

See Documentation Index for the maintained user and maintainer docs.