Expand description
Apache Arrow and SQL Server bridge through Tiberius.
arrow-tiberius bridges Apache Arrow and Microsoft SQL Server through the
Tiberius TDS driver. The crate is designed around a bidirectional boundary:
Arrow schemas and RecordBatch values can be planned and written to SQL
Server, and future read-side APIs can map SQL Server metadata and rows back
to Arrow.
The v0.1 API implements the Arrow-to-SQL Server write path first: plan an Arrow schema for SQL Server, render deterministic DDL, inspect structured diagnostics, and bulk load one or more record batches. SQL Server-to-Arrow reads are reserved for a later release.
§Quick Start
Plan an Arrow schema and render CREATE TABLE SQL:
use arrow_schema::{DataType, Field, Schema};
use arrow_tiberius::{
MssqlProfile, PlanOptions, TableName, create_table_sql_from_mappings,
};
let schema = Schema::new(vec![
Field::new("id", DataType::Int64, false),
Field::new("name", DataType::Utf8, true),
]);
let profile = MssqlProfile::sql_server_2016_compat_100();
let outcome = profile.plan_arrow_schema(&schema, PlanOptions::default())?;
let table = TableName::new("dbo", "people")?;
let ddl = create_table_sql_from_mappings(&table, outcome.mappings());
assert!(ddl.contains("CREATE TABLE [dbo].[people]"));Connect through the crate-owned Tiberius compatibility boundary:
use arrow_tiberius::{
ConnectedMssqlClient, connect_mssql_client_from_ado_string,
};
async fn connect(
connection_string: &str,
) -> arrow_tiberius::Result<ConnectedMssqlClient> {
connect_mssql_client_from_ado_string(connection_string).await
}BulkWriter validates target table metadata before writing. It does not
create tables automatically; callers can use create_table_sql_from_mappings
when they want this crate to produce a table definition.
§Main Modules
schemaplans Arrow fields into SQL Server column mappings and DDL metadata.mssqlcontains SQL Server identifiers, profiles, types, and DDL helpers.diagnosticexposes structured planning and runtime diagnostics.- The
writemodule contains write policies, backend selection, andBulkWriter.
§Writer Backends
WriteBackend::Auto is the default selection and currently resolves to
WriteBackend::DirectRawBulk.
WriteBackend::DirectRawBulk is the optimized direct Arrow-to-TDS path for
supported mappings. WriteBackend::BaselineTokenRow remains available as a
compatibility and reference path through Tiberius TokenRow bulk load.
WriteBackend::DirectFramedBulk uses the direct row encoder through
Tiberius framed writes.
§SQL Server Compatibility
Choose the MssqlProfile that matches the SQL Server version and database
compatibility level you plan to write against. The original
MssqlProfile::sql_server_2016_compat_100 profile remains available. The
profile surface also models SQL Server 2016, 2017, 2019, 2022, and 2025
version/compatibility-level pairs through MssqlProfile::new.
§Tiberius Dependency Model
This crate depends on the published tiberius-raw-bulk package as the crate
name tiberius and owns that compatibility boundary. Downstream crates
should use connect_mssql_client_from_ado_string and
ConnectedMssqlClient instead of constructing a raw Tiberius client for
BulkWriter.
[dependencies]
arrow-tiberius = "0.1"Depending on upstream tiberius separately creates a distinct crate type and
will not produce a client compatible with this crate’s writer internals.
§Feature Flags
bench-profile: benchmark-only direct write profiling hooks.integration-tests: SQL Server integration tests that are normally run throughcargo xtask sqlserver-test.
Docs.rs is configured to build with all features so feature-gated public items are visible in API documentation. Normal library use does not require either feature.
§More Documentation
Re-exports§
pub use arrow::ArrowFieldRef;pub use connection::ConnectedBulkWriter;pub use connection::ConnectedMssqlClient;pub use connection::SqlExecutionOutcome;pub use connection::connect_mssql_client_from_ado_string;pub use diagnostic::Diagnostic;pub use diagnostic::DiagnosticCode;pub use diagnostic::DiagnosticSet;pub use diagnostic::DiagnosticSeverity;pub use diagnostic::FieldRef;pub use diagnostic::PlanOutcome;pub use error::Error;pub use error::ErrorInfo;pub use error::Result;pub use mssql::CompatibilityLevel;pub use mssql::CreateTableOptions;pub use mssql::Identifier;pub use mssql::IdentifierPolicy;pub use mssql::MssqlColumn;pub use mssql::MssqlProfile;pub use mssql::MssqlTimePrecision;pub use mssql::MssqlType;pub use mssql::MssqlTypeLength;pub use mssql::MssqlVersion;pub use mssql::TableName;pub use mssql::create_table_sql;pub use schema::PlannedSchema;pub use schema::SchemaMapping;pub use schema::create_table_sql_from_mappings;pub use schema::mssql_columns_from_mappings;pub use schema::plan_arrow_schema_to_mssql_schema;pub use write::BinaryPolicy;pub use write::BulkWriter;pub use write::Date64Policy;pub use write::Decimal256Policy;pub use write::DecimalPolicy;pub use write::FloatPolicy;pub use write::NanosecondPolicy;pub use write::PlanOptions;pub use write::SchemaCheck;pub use write::StringPolicy;pub use write::TimestampPolicy;pub use write::TimezonePolicy;pub use write::UInt64Policy;pub use write::WriteBackend;pub use write::WriteOptions;pub use write::WritePhase;pub use write::WriteStats;pub use write::validate_arrow_schema_against_mappings;pub use write::validate_record_batch_schema_against_mappings;
Modules§
- arrow
- Arrow-side schema metadata. Arrow-side schema metadata.
- connection
- SQL Server connection helpers. SQL Server connection helpers.
- diagnostic
- Structured diagnostics for planning and writing. Structured diagnostics for planning and writing.
- error
- Error types for
arrow-tiberius. Error types forarrow-tiberius. - mssql
- MSSQL-side schema metadata, identifiers, profile, and DDL helpers. MSSQL-side schema metadata, identifiers, profile, and DDL helpers.
- schema
- Bidirectional Arrow/MSSQL schema mapping. Bidirectional Arrow/MSSQL schema mapping.
- write
- Write-path options and conversion policies. Write-path options and policies.