Skip to main content

Crate arrow_tiberius

Crate arrow_tiberius 

Source
Expand description

Apache Arrow and SQL Server bridge through Tiberius.

arrow-tiberius bridges Apache Arrow and Microsoft SQL Server through the Tiberius TDS driver. The crate is designed around a bidirectional boundary: Arrow schemas and RecordBatch values can be planned and written to SQL Server, and future read-side APIs can map SQL Server metadata and rows back to Arrow.

The v0.1 API implements the Arrow-to-SQL Server write path first: plan an Arrow schema for SQL Server, render deterministic DDL, inspect structured diagnostics, and bulk load one or more record batches. SQL Server-to-Arrow reads are reserved for a later release.

§Quick Start

Plan an Arrow schema and render CREATE TABLE SQL:

use arrow_schema::{DataType, Field, Schema};
use arrow_tiberius::{
    MssqlProfile, PlanOptions, TableName, create_table_sql_from_mappings,
};

let schema = Schema::new(vec![
    Field::new("id", DataType::Int64, false),
    Field::new("name", DataType::Utf8, true),
]);

let profile = MssqlProfile::sql_server_2016_compat_100();
let outcome = profile.plan_arrow_schema(&schema, PlanOptions::default())?;

let table = TableName::new("dbo", "people")?;
let ddl = create_table_sql_from_mappings(&table, outcome.mappings());
assert!(ddl.contains("CREATE TABLE [dbo].[people]"));

Connect through the crate-owned Tiberius compatibility boundary:

use arrow_tiberius::{
    ConnectedMssqlClient, connect_mssql_client_from_ado_string,
};

async fn connect(
    connection_string: &str,
) -> arrow_tiberius::Result<ConnectedMssqlClient> {
    connect_mssql_client_from_ado_string(connection_string).await
}

BulkWriter validates target table metadata before writing. It does not create tables automatically; callers can use create_table_sql_from_mappings when they want this crate to produce a table definition.

§Main Modules

  • schema plans Arrow fields into SQL Server column mappings and DDL metadata.
  • mssql contains SQL Server identifiers, profiles, types, and DDL helpers.
  • diagnostic exposes structured planning and runtime diagnostics.
  • The write module contains write policies, backend selection, and BulkWriter.

§Writer Backends

WriteBackend::Auto is the default selection and currently resolves to WriteBackend::DirectRawBulk. WriteBackend::DirectRawBulk is the optimized direct Arrow-to-TDS path for supported mappings. WriteBackend::BaselineTokenRow remains available as a compatibility and reference path through Tiberius TokenRow bulk load. WriteBackend::DirectFramedBulk uses the direct row encoder through Tiberius framed writes.

§SQL Server Compatibility

Choose the MssqlProfile that matches the SQL Server version and database compatibility level you plan to write against. The original MssqlProfile::sql_server_2016_compat_100 profile remains available. The profile surface also models SQL Server 2016, 2017, 2019, 2022, and 2025 version/compatibility-level pairs through MssqlProfile::new.

§Tiberius Dependency Model

This crate depends on the published tiberius-raw-bulk package as the crate name tiberius and owns that compatibility boundary. Downstream crates should use connect_mssql_client_from_ado_string and ConnectedMssqlClient instead of constructing a raw Tiberius client for BulkWriter.

[dependencies]
arrow-tiberius = "0.1"

Depending on upstream tiberius separately creates a distinct crate type and will not produce a client compatible with this crate’s writer internals.

§Feature Flags

  • bench-profile: benchmark-only direct write profiling hooks.
  • integration-tests: SQL Server integration tests that are normally run through cargo xtask sqlserver-test.

Docs.rs is configured to build with all features so feature-gated public items are visible in API documentation. Normal library use does not require either feature.

§More Documentation

Re-exports§

pub use arrow::ArrowFieldRef;
pub use connection::ConnectedBulkWriter;
pub use connection::ConnectedMssqlClient;
pub use connection::SqlExecutionOutcome;
pub use connection::connect_mssql_client_from_ado_string;
pub use diagnostic::Diagnostic;
pub use diagnostic::DiagnosticCode;
pub use diagnostic::DiagnosticSet;
pub use diagnostic::DiagnosticSeverity;
pub use diagnostic::FieldRef;
pub use diagnostic::PlanOutcome;
pub use error::Error;
pub use error::ErrorInfo;
pub use error::Result;
pub use mssql::CompatibilityLevel;
pub use mssql::CreateTableOptions;
pub use mssql::Identifier;
pub use mssql::IdentifierPolicy;
pub use mssql::MssqlColumn;
pub use mssql::MssqlProfile;
pub use mssql::MssqlTimePrecision;
pub use mssql::MssqlType;
pub use mssql::MssqlTypeLength;
pub use mssql::MssqlVersion;
pub use mssql::TableName;
pub use mssql::create_table_sql;
pub use schema::PlannedSchema;
pub use schema::SchemaMapping;
pub use schema::create_table_sql_from_mappings;
pub use schema::mssql_columns_from_mappings;
pub use schema::plan_arrow_schema_to_mssql_schema;
pub use write::BinaryPolicy;
pub use write::BulkWriter;
pub use write::Date64Policy;
pub use write::Decimal256Policy;
pub use write::DecimalPolicy;
pub use write::FloatPolicy;
pub use write::NanosecondPolicy;
pub use write::PlanOptions;
pub use write::SchemaCheck;
pub use write::StringPolicy;
pub use write::TimestampPolicy;
pub use write::TimezonePolicy;
pub use write::UInt64Policy;
pub use write::WriteBackend;
pub use write::WriteOptions;
pub use write::WritePhase;
pub use write::WriteStats;
pub use write::validate_arrow_schema_against_mappings;
pub use write::validate_record_batch_schema_against_mappings;

Modules§

arrow
Arrow-side schema metadata. Arrow-side schema metadata.
connection
SQL Server connection helpers. SQL Server connection helpers.
diagnostic
Structured diagnostics for planning and writing. Structured diagnostics for planning and writing.
error
Error types for arrow-tiberius. Error types for arrow-tiberius.
mssql
MSSQL-side schema metadata, identifiers, profile, and DDL helpers. MSSQL-side schema metadata, identifiers, profile, and DDL helpers.
schema
Bidirectional Arrow/MSSQL schema mapping. Bidirectional Arrow/MSSQL schema mapping.
write
Write-path options and conversion policies. Write-path options and policies.