Arrow SQL Server

Arrow SQL Server is a high-performance Apache Arrow RecordBatch bulk writer for Microsoft SQL Server, built on the Tiberius TDS driver.

The current API focuses on Arrow-to-SQL Server writes:

• plan SQL Server-compatible schemas from Arrow schemas,
• render deterministic CREATE TABLE SQL,
• report unsupported mappings as structured diagnostics,
• write Arrow RecordBatch values with a selectable SQL Server bulk writer,
• emit sanitized writer and protocol tracing through tracing.

SQL Server-to-Arrow reads are reserved for a later release.

Install

[dependencies]
arrow-sql-server = "0.3"

Quick Start

Plan an Arrow schema and render SQL Server DDL:

use arrow_schema::{DataType, Field, Schema};
use arrow_sql_server::{
    CompatibilityLevel, MssqlProfile, MssqlVersion, PlanOptions, TableName,
    create_table_sql_from_mappings,
};

fn main() -> arrow_sql_server::Result<()> {
    let schema = Schema::new(vec![
        Field::new("id", DataType::Int64, false),
        Field::new("name", DataType::Utf8, true),
    ]);

    let profile = MssqlProfile::new(
        MssqlVersion::SqlServer2022,
        CompatibilityLevel::SQL_SERVER_2022,
    )?;
    let outcome = profile.plan_arrow_schema(&schema, PlanOptions::default())?;

    let table = TableName::new("dbo", "people")?;
    let ddl = create_table_sql_from_mappings(&table, outcome.mappings());

    assert!(ddl.contains("CREATE TABLE [dbo].[people]"));
    Ok(())
}

Write a batch to an existing SQL Server table:

use arrow_array::RecordBatch;
use arrow_sql_server::{
    CompatibilityLevel, MssqlProfile, MssqlVersion, PlanOptions, TableName,
    WriteBackend, WriteOptions, connect_mssql_client_from_ado_string,
};

async fn write_batch(
    connection_string: &str,
    batch: &RecordBatch,
) -> arrow_sql_server::Result<()> {
    let mut client = connect_mssql_client_from_ado_string(connection_string).await?;
    let profile = MssqlProfile::new(
        MssqlVersion::SqlServer2022,
        CompatibilityLevel::SQL_SERVER_2022,
    )?;
    let planned_schema = profile
        .plan_arrow_schema(batch.schema().as_ref(), PlanOptions::default())?
        .into_value();

    let table = TableName::new("dbo", "people")?;
    let mut writer = client
        .bulk_writer(
            table,
            planned_schema,
            WriteOptions {
                backend: WriteBackend::DirectRawBulk,
                ..WriteOptions::default()
            },
        )
        .await?;

    writer.write_batch(batch).await?;
    writer.finish().await?;
    Ok(())
}

The connected writer validates target table metadata before sending rows. It does not create the target table automatically; callers can use the DDL helper when they want this crate to produce the table definition.

Diagnostics

Planning and write failures return structured diagnostics instead of requiring string parsing. Diagnostics include severity, machine-readable code, field context, row context when available, and message text.

For user-facing write failure reports, use Error::safe_error_info(). It exposes the write phase, inner error kind, sanitized summary, diagnostic codes, and structured diagnostics when available while keeping dependency source text out of default reports.

For the complete planning surface, see Arrow to SQL Server Type Mapping.

Writer Backends

WriteBackend controls how planned Arrow rows are sent to SQL Server:

The direct raw backend is the optimized production path for currently supported mappings. The baseline backend remains useful for compatibility checks and parity tests.

Observability

Arrow SQL Server emits structured spans and events through tracing for schema planning, writer initialization, batch writes, direct raw backend summaries, and writer finish. It never installs a subscriber.

Its tiberius-raw-bulk dependency also emits sanitized protocol tracing under the tiberius_raw_bulk::protocol target. Those protocol events are emitted inside active arrow_sql_server writer spans during connect, bulk-load, and finish operations.

See Observability for subscriber setup, span and event names, safe field categories, redaction guarantees, and workflow integration.

Examples

Compile-checked examples that do not require SQL Server:

cargo run --example schema_to_ddl
cargo run --example planning_diagnostics
cargo run --example backend_selection
cargo run --example policy_dependent_planning

SQL Server write example:

ARROW_SQL_SERVER_EXAMPLE_MSSQL_URL='server=tcp:localhost,1433;user=sa;password=...;TrustServerCertificate=true' \
  cargo run --example sqlserver_batch_write

By default, the SQL Server example creates, writes to, and drops [dbo].[arrow_sql_server_example_write].

Compatibility

Choose the MssqlProfile that matches the SQL Server version and database compatibility level you plan to write against:

use arrow_sql_server::{CompatibilityLevel, MssqlProfile, MssqlVersion};

let profile = MssqlProfile::new(
    MssqlVersion::SqlServer2022,
    CompatibilityLevel::SQL_SERVER_2022,
)?;

The profile surface models SQL Server 2016, 2017, 2019, 2022, and 2025 version/compatibility-level pairs. Legacy convenience constructors such as MssqlProfile::sql_server_2016_compat_100() and MssqlProfile::sql_server_2017_compat_100() remain available for callers that target those exact environments.

Arrow SQL Server depends on the published tiberius-raw-bulk package as the crate name tiberius and owns that compatibility boundary internally:

tiberius = { package = "tiberius-raw-bulk", version = "=0.12.3-raw-bulk.15", default-features = false, features = [
    "tds73",
    "winauth",
    "native-tls",
] }

Downstream crates should normally depend only on arrow-sql-server and construct SQL Server clients through connect_mssql_client_from_ado_string or ConnectedMssqlClient.

Feature Flags

Validation

Default local validation does not require SQL Server:

cargo fmt --check
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo test --workspace

Run SQL Server integration tests through the xtask harness:

cargo xtask sqlserver-test
cargo xtask sqlserver-compat-probe

Documentation

See Documentation Index for the maintained user and maintainer docs.