Crate connector_arrow

A flexible database client that converts data into Apache Arrow format across various databases.

The API provided by each data source is described in api module.

Capabilities:

• Query: Query databases and retrieve results in Apache Arrow format.
• Query Parameters: Utilize Arrow type system for query parameters.
• Temporal and Container Types: Correctly handles temporal and container types.
• Schema Introspection: Query the database for schema of specific tables.
• Schema Migration: Basic schema migration commands.
• Append: Write arrow::record_batch::RecordBatch into database tables.

Example for SQLite:

use connector_arrow::api::{Connector, Statement, ResultReader};
use connector_arrow::arrow;

// a regular rusqlite connection
let conn = rusqlite::Connection::open_in_memory()?;

// wrap into connector_arrow connection
let mut conn = connector_arrow::sqlite::SQLiteConnection::new(conn);

let mut stmt = conn.query("SELECT 1 as a")?;

let mut reader = stmt.start([])?;

let schema: arrow::datatypes::SchemaRef = reader.get_schema()?;

// reader implements Iterator<Item = Result<RecordBatch, _>>
let batches: Vec<arrow::record_batch::RecordBatch> = reader.collect::<Result<_, _>>()?;

For a list of supported databases, refer to the crates.io page.

Re-exports

• pub use arrow;

Modules

• api
  Database client interface that uses Apache Arrow as data-transfer format and schema definition format.
• duckdb
  Provides connector_arrow traits for duckdb crate.
• mysql
• postgres
  Provides connector_arrow traits for postgres crate.
• sqlite
  Provides connector_arrow traits for rusqlite crate.
• tiberius
• types
• util
  Utilities for converting row-major tabular data into Apache Arrow. Used by database client implementations.

Macros

• impl_consume_unsupported
• impl_produce_unsupported

Enums

• ConnectorError
  Errors that can be raised from this library.
• TableCreateError
• TableDropError

Functions

• query
  Open a connection, execute a single query and return the results.