Crate sqlx_exasol

Source

Expand description

A database driver for Exasol to be used with the Rust sqlx framework.

Based on sqlx version 0.9.0-alpha.1.

§Features flags

etl - Add support for ETL jobs.
compression - Add compression support (for both connections and ETL jobs).
any - Add support for the Any database driver, which can proxy to a database driver at runtime.
derive - Add support for the derive family macros, those are FromRow, Type, Encode, Decode.
macros - Add support for the query*! macros, which allows compile-time checked queries.
migrate - Add support for the migration management and migrate! macro, which allow compile-time embedded migrations.
uuid - Add support for UUID.
chrono - Add support for date and time types from chrono.
time - Add support for date and time types from time crate (alternative to chrono, which is preferred by query! macro, if both enabled).
bigdecimal - Add support for BigDecimal from the bigdecimal crate.
rust_decimal - Add support for Decimal from the rust_decimal crate.
geo-types - Add support for Geometry and its variants from the geo-types crate.
json - Add support for Json<T> as well as serde_json::Value and serde_json::RawValue.

§Supported types

See the types module.

§Comparison to native sqlx drivers

The driver re-exports all sqlx public API and implements the exposed traits. As a result, it can do all the drivers shipped with sqlx do, with some caveats:

Limitations
- separate CLI utility (sqlx-exasol instead of sqlx)
- compile time query macros cannot work along the ones from sqlx within the same crate
- no locking migrations support¹
- no column nullability checks²
Additions
- array-like parameter binding in queries, thanks to the columnar nature of the Exasol database
- performant & parallelizable ETL IMPORT/EXPORT jobs in CSV format through HTTP Transport

§Compile-time query checks

The driver now supports compile-time query validation and can be used alongside sqlx within the same crate. Note however that derive proc-macros from sqlx are database agnostic and thus sqlx-exasol just re-exports them as-is.

§CLI utility

The driver uses its own CLI utility (built on the same sqlx-cli library):

cargo install sqlx-exasol-cli

# Usage is exactly the same as sqlx-cli
sqlx-exasol database create
sqlx-exasol database drop
sqlx-exasol migrate add <name>
sqlx-exasol migrate run
cargo sqlx-exasol prepare

§Connection string

The connection string is expected to be an URL with the exa:// scheme, e.g: exa://sys:exasol@localhost:8563.

See ExaConnectOptions for a list of supported connection string parameters.

§HTTP Transport

Functionality that allows performant data import/export by creation of one-shot HTTP servers to which Exasol connects to (at most one per node), thus balancing the load.

The data is always in CSV format and job configuration can be done through the ImportBuilder and ExportBuilder structs. The workers implement AsyncWrite and AsyncRead respectively, providing great flexibility in terms of how the data is processed.

The general flow of an ETL job is:

build the job through ImportBuilder or ExportBuilder
concurrently wait on the query execution future (typically from the main thread) and on worker operations (async tasks can be spawned in multi-threaded runtimes to further parallelize the workload).
when all the workers are done (readers reach EOF, while writers require an explicit close()) the job ends and the query execution future returns.
an error/timeout issue results in the query execution future or a worker throwing an error, therefore consider joining the tasks and aborting them if an error is thrown somewhere.

§Examples

Using the driver for regular database interactions:

use std::env;

use sqlx_exasol::{error::*, *};

let pool = ExaPool::connect(&env::var("DATABASE_URL").unwrap()).await?;
let mut con = pool.acquire().await?;

sqlx_exasol::query("CREATE SCHEMA RUST_DOC_TEST")
    .execute(&mut *con)
    .await?;

Array-like parameter binding, also featuring the crate::types::ExaIter adapter. An important thing to note is that the parameter sets must be of equal length, otherwise an error is thrown:

use std::{collections::HashSet, env};

use sqlx_exasol::{error::*, *};

let pool = ExaPool::connect(&env::var("DATABASE_URL").unwrap()).await?;
let mut con = pool.acquire().await?;

let params1 = vec![1, 2, 3];
let params2 = HashSet::from([1, 2, 3]);

sqlx_exasol::query("INSERT INTO MY_TABLE VALUES (?, ?)")
    .bind(&params1)
    .bind(types::ExaIter::new(params2.iter()))
    .execute(&mut *con)
    .await?;

An EXPORT - IMPORT ETL data pipe.

use std::env;

use futures_util::{
    future::{try_join, try_join3, try_join_all},
    AsyncReadExt, AsyncWriteExt, TryFutureExt,
};
use sqlx_exasol::{error::*, etl::*, *};

async fn pipe(mut reader: ExaExport, mut writer: ExaImport) -> Result<(), BoxDynError> {
    let mut buf = vec![0; 5120].into_boxed_slice();
    let mut read = 1;

    while read > 0 {
        // Readers return EOF when there's no more data.
        read = reader.read(&mut buf).await?;
        // Write data to Exasol
        writer.write_all(&buf[..read]).await?;
    }

    // Writes, unlike readers, MUST be closed to signal we won't send more data to Exasol
    writer.close().await?;
    Ok(())
}

let pool = ExaPool::connect(&env::var("DATABASE_URL").unwrap()).await?;
let mut con1 = pool.acquire().await?;
let mut con2 = pool.acquire().await?;

// Build EXPORT job
let (export_fut, readers) = ExportBuilder::new_from_table("TEST_ETL", None)
    .build(&mut con1)
    .await?;

// Build IMPORT job
let (import_fut, writers) = ImportBuilder::new("TEST_ETL", None)
    .build(&mut con2)
    .await?;

// Use readers and writers in some futures
let transport_futs = std::iter::zip(readers, writers).map(|(r, w)| pipe(r, w));

// Execute the EXPORT and IMPORT query futures along with the worker futures
let (export_res, import_res, _) = try_join3(
    export_fut.map_err(From::from),
    import_fut.map_err(From::from),
    try_join_all(transport_futs),
)
.await?;

assert_eq!(export_res.rows_affected(), import_res.rows_affected());

§Footnotes

1: Exasol has no advisory or database locks and simple, unnested, transactions are unfortunately not enough to define a mechanism so that concurrent migrations do not collide. This does not pose a problem when migrations are run sequentially or do not act on the same database objects.

2: Exasol does not provide the information of whether a column is nullable or not, so the driver cannot implicitly decide whether a NULL value can go into a certain database column or not until it actually tries.

Modules§

_config_unstable-docs: (Exported for documentation only) Guide and reference for sqlx.toml files.
anyany
database: Traits to represent a database driver.
decode: Provides Decode for decoding values from the database.
encode: Provides Encode for encoding values for the database.
error: Types for working with errors produced by SQLx.
etletl: This module provides the building blocks for creating IMPORT and EXPORT jobs. These are represented by a query that gets executed concurrently with some ETL workers, both of which are obtained by building the ETL job. The data format is always CSV, but there are some customizations that can be done on the builders such as row or column separator, etc.
migratemigrate
pool: Provides the connection pool for asynchronous SQLx connections.
prelude: Convenience re-export of common traits.
query: Types and traits for the query family of functions and macros.
query_builder: Runtime query-builder API.
types: Supported types

Macros§

migratemigrate
querymacros
query_asmacros
query_as_uncheckedmacros
query_filemacros
query_file_asmacros
query_file_as_uncheckedmacros
query_file_scalarmacros
query_file_scalar_uncheckedmacros
query_file_uncheckedmacros
query_scalarmacros
query_scalar_uncheckedmacros
query_uncheckedmacros

Structs§

Anyany: Opaque database driver. Capable of being used in place of any SQLx database driver. The actual driver used will be selected at runtime, from the connection url.
AnyConnection: A connection to any SQLx database.
AssertSqlSafe: Assert that a query string is safe to execute on a database connection.
ExaArguments: Implementor of Arguments.
ExaAttributes: Struct representing attributes related to the connection with the Exasol server. These can either be returned by an explicit getAttributes call or as part of any response.
ExaColumn: Implementor of Column.
ExaConnectOptions: Options for connecting to the Exasol database. Implementor of ConnectOptions.
ExaConnectOptionsBuilder: Builder for ExaConnectOptions.
ExaConnection: A connection to the Exasol database. Implementor of Connection.
ExaDatabaseError: An error directly issued by the Exasol database. Implementor of DatabaseError.
ExaQueryResult: A query result containing the number of affected rows.
ExaRow: Struct representing a result set row. Implementor of Row.
ExaStatement: Implementor of Statement.
ExaTransactionManager: Implementor of TransactionManager.
ExaTypeInfo: Information about an Exasol data type and implementor of TypeInfo.
ExaValue: Implementor of Value.
ExaValueRef: Implementor of ValueRef.
Exasol: Implementor of Database.
Pool: An asynchronous pool of SQLx database connections.
QueryBuilder: A builder type for constructing queries at runtime.
RawSql: One or more raw SQL statements, separated by semicolons (;).
SessionInfo: Struct representing database information returned after establishing a connection.
SqlStr: A SQL string that is ready to execute on a database connection.
Transaction: An in-progress database transaction or savepoint.

Enums§

ColumnOrigin: The possible statuses for our knowledge of the origin of a Column.
Either: The enum Either with variants Left and Right is a general purpose sum type with two cases.
Error: Represents all the ways a method can fail within SQLx.
ExaCompressionMode: Options for controlling the desired compression behavior of the connection to the Exasol server.
ExaSslMode: Options for controlling the desired security state of the connection to the Exasol server.

Traits§

Acquire: Acquire connections or transactions from a database in a generic way.
AnyExecutorany: An alias for Executor<'_, Database = Any>.
Arguments: A tuple of arguments to be sent to the database.
Column
ColumnIndex: A type that can be used to index into a Row or Statement.
ConnectOptions
Connection: Represents a single database connection.
Database: A database driver.
Decode: A type that can be decoded from the database.
Encode: Encode a single value to be sent to the database.
ExaExecutor: An alias for Executor<'_, Database = Exasol>.
Execute: A type that may be executed against a database connection.
Executor: A type that contains or can provide a database connection to use for executing queries against the database.
FromRow: A record that can be built from a row returned by the database.
IntoArguments
Row: Represents a single row from the database.
SqlSafeStr: A SQL string that is safe to execute on a database connection.
Statement: An explicitly prepared statement.
Type: Indicates that a SQL type is supported for a database.
TypeInfo: Provides information about a SQL type for the database driver.
Value: An owned value from the database.
ValueRef: A reference to a single value from the database.

Functions§

query: Execute a single SQL query as a prepared statement (transparently cached).
query_as: Execute a single SQL query as a prepared statement (transparently cached). Maps rows to Rust types using FromRow.
query_as_with: Execute a single SQL query, with the given arguments as a prepared statement (transparently cached). Maps rows to Rust types using FromRow.
query_scalar: Execute a single SQL query as a prepared statement (transparently cached) and extract the first column of each row.
query_scalar_with: Execute a SQL query as a prepared statement (transparently cached), with the given arguments, and extract the first column of each row.
query_with: Execute a SQL query as a prepared statement (transparently cached), with the given arguments.
raw_sql: Execute one or more statements as raw SQL, separated by semicolons (;).

Type Aliases§

AnyPool
ExaPool: An alias for Pool, specialized for Exasol.
ExaPoolOptions: An alias for PoolOptions, specialized for Exasol.
ExaTransaction: An alias for Transaction, specialized for Exasol.
Result: A specialized Result type for SQLx.

Attribute Macros§

testmacros

Derive Macros§

Decode
Encode