Crate arrow_odbc

Fill Apache Arrow arrays from ODBC data sources.

Usage

use arrow_odbc::{odbc_api::{Environment, ConnectionOptions}, OdbcReaderBuilder};

const CONNECTION_STRING: &str = "\
    Driver={ODBC Driver 17 for SQL Server};\
    Server=localhost;\
    UID=SA;\
    PWD=My@Test@Password1;\
";

fn main() -> Result<(), anyhow::Error> {
    // Your application is fine if you spin up only one Environment.
    let odbc_environment = Environment::new()?;
     
    // Connect with database.
    let connection = odbc_environment.connect_with_connection_string(
        CONNECTION_STRING,
        ConnectionOptions::default()
    )?;

    // This SQL statement does not require any arguments.
    let parameters = ();

    // Execute query and create result set
    let cursor = connection
        .execute("SELECT * FROM MyTable", parameters)?
        .expect("SELECT statement must produce a cursor");

    // Read result set as arrow batches. Infer Arrow types automatically using the meta
    // information of `cursor`.
    let arrow_record_batches = OdbcReaderBuilder::new().build(cursor)?;

    for batch in arrow_record_batches {
        // ... process batch ...
    }

    Ok(())
}

Re-exports

• pub use arrow;
• pub use odbc_api;

Structs

• BufferAllocationOptions
  Allows setting limits for buffers bound to the ODBC data source. Check this out if you find that you get memory allocation, or zero sized column errors. Used than constructing a reader using [crate::OdbcReader::with].
• ConcurrentOdbcReader
  Arrow ODBC reader. Implements the arrow::record_batch::RecordBatchReader trait so it can be used to fill Arrow arrays from an ODBC data source. Similar to crate::OdbcReader, yet ConcurrentOdbcReader fetches ODBC batches in a second transit buffer eagerly from the database in a dedicated system thread. This allows the allocation of the Arrow arrays and your application logic to run on the main thread, while fetching the batches from the source happens concurrently. You need twice the memory for the transit buffer for this strategy, since one is may be in use by the main thread in order to copy values into arrow arrays, while the other is used to write values from the database.
• OdbcReader
  Arrow ODBC reader. Implements the arrow::record_batch::RecordBatchReader trait so it can be used to fill Arrow arrays from an ODBC data source.
• OdbcReaderBuilder
  Creates instances of OdbcReader based on odbc_api::Cursor.
• OdbcWriter
  Inserts batches from an arrow::record_batch::RecordBatchReader into a database.
• Quirks
  Quirks of databases and their drivers we can take into account exporting data from database into Arrow arrays.

Enums

• ColumnFailure
  Read error related to a specific column
• Error
  A variation of things which can go wrong then creating an crate::OdbcReader.
• WriterError
  Emitted writing values from arror arrays into a table on the database

Functions

• arrow_schema_from
  Query the metadata to create an arrow schema. This method is invoked automatically for you by [crate::OdbcReader::new]. You may want to call this method in situtation ther you want to create an arrow schema without creating the reader yet.
• insert_into_table
  Fastest and most convinient way to stream the contents of arrow record batches into a database table. For usecase there you want to insert repeatedly into the same table from different streams it is more efficient to create an instance of self::OdbcWriter and reuse it.
• insert_statement_from_schema
  Creates an SQL insert statement from an arrow schema. The resulting statement will have one placeholer (?) for each column in the statement.