Crate arrow_odbc
source ·Expand description
Fill Apache Arrow arrays from ODBC data sources.
§Usage
use arrow_odbc::{odbc_api::{Environment, ConnectionOptions}, OdbcReaderBuilder};
const CONNECTION_STRING: &str = "\
Driver={ODBC Driver 17 for SQL Server};\
Server=localhost;\
UID=SA;\
PWD=My@Test@Password1;\
";
fn main() -> Result<(), anyhow::Error> {
// Your application is fine if you spin up only one Environment.
let odbc_environment = Environment::new()?;
// Connect with database.
let connection = odbc_environment.connect_with_connection_string(
CONNECTION_STRING,
ConnectionOptions::default()
)?;
// This SQL statement does not require any arguments.
let parameters = ();
// Execute query and create result set
let cursor = connection
.execute("SELECT * FROM MyTable", parameters)?
.expect("SELECT statement must produce a cursor");
// Read result set as arrow batches. Infer Arrow types automatically using the meta
// information of `cursor`.
let arrow_record_batches = OdbcReaderBuilder::new().build(cursor)?;
for batch in arrow_record_batches {
// ... process batch ...
}
Ok(())
}
Re-exports§
Structs§
- Allows setting limits for buffers bound to the ODBC data source. Check this out if you find that you get memory allocation, or zero sized column errors. Used than constructing a reader using [
crate::OdbcReader::with
]. - Arrow ODBC reader. Implements the
arrow::record_batch::RecordBatchReader
trait so it can be used to fill Arrow arrays from an ODBC data source. Similar tocrate::OdbcReader
, yetConcurrentOdbcReader
fetches ODBC batches in a second transit buffer eagerly from the database in a dedicated system thread. This allows the allocation of the Arrow arrays and your application logic to run on the main thread, while fetching the batches from the source happens concurrently. You need twice the memory for the transit buffer for this strategy, since one is may be in use by the main thread in order to copy values into arrow arrays, while the other is used to write values from the database. - Arrow ODBC reader. Implements the
arrow::record_batch::RecordBatchReader
trait so it can be used to fill Arrow arrays from an ODBC data source. - Creates instances of
OdbcReader
based onodbc_api::Cursor
. - Inserts batches from an
arrow::record_batch::RecordBatchReader
into a database.
Enums§
- Read error related to a specific column
- A variation of things which can go wrong then creating an
crate::OdbcReader
. - Emitted writing values from arror arrays into a table on the database
Functions§
- Query the metadata to create an arrow schema. This method is invoked automatically for you by [
crate::OdbcReader::new
]. You may want to call this method in situtation ther you want to create an arrow schema without creating the reader yet. - Fastest and most convinient way to stream the contents of arrow record batches into a database table. For usecase there you want to insert repeatedly into the same table from different streams it is more efficient to create an instance of
self::OdbcWriter
and reuse it. - Creates an SQL insert statement from an arrow schema. The resulting statement will have one placeholer (
?
) for each column in the statement.