Crate stately_arrow 

Arrow-based data connectivity and query execution over HTTP APIs.

This crate provides a flexible abstraction layer over DataFusion for building data query services with support for multiple backend connectors.

Terminology

• Connector: The raw configuration for a data source (credentials, endpoints, options).
• Connection: Runtime metadata about a connector’s active presence in DataFusion, including its catalog name and capabilities.
• Backend: The implementation that bridges a connector configuration to DataFusion, handling session setup and query execution.

Core Abstractions

• ConnectorRegistry: Trait for managing and retrieving connectors by ID.
• Backend: Trait for data source implementations that integrate with DataFusion.
• QueryContext: High-level query execution interface combining a session and registry.
• QuerySession: Abstraction over DataFusion’s SessionContext for custom implementations.

API Endpoints

The api::router function provides these endpoints:

Method	Path	Description
GET	/connectors	List available connectors
POST	/connectors	Get details for multiple connectors
GET	/connectors/{id}	Get connector details
GET	/register	List registered connections
GET	/register/{id}	Register a connector
GET	/catalogs	List DataFusion catalogs
POST	/query	Execute SQL query (streaming Arrow IPC)

Feature Flags

• object-store: Enables S3, GCS, Azure, and local filesystem backends.
• database: Enables base database connector types.
• clickhouse: Enables ClickHouse database backend (implies database).
• registry: Enables the generic registry implementation with stately integration.
• strum: Enables AsRefStr derives for enum types.

Example

use std::sync::Arc;
use axum::Router;
use stately_arrow::{api, QueryContext, QueryState};

// Create a registry (implement ConnectorRegistry or use generic::Registry)
let registry: Arc<dyn ConnectorRegistry> = create_registry();

// Create query context
let query_context = QueryContext::new(registry);

// Create the router
let arrow_router = api::router(QueryState::new(query_context));

// Mount under /arrow
let app = Router::new().nest("/arrow", arrow_router);

Re-exports

pub use api::openapi::OpenApiDoc;

pub use backend::Backend;

pub use backend::BackendMetadata;

pub use backend::Capability;

pub use backend::ConnectionKind;

pub use backend::ConnectionMetadata;

pub use context::QueryContext;

pub use context::QuerySession;

pub use context::SessionCapability;

pub use error::Error;

pub use error::Result;

pub use registry::ConnectorRegistry;

pub use registry::generic;

pub use request::ConnectionDetailsRequest;

pub use request::ConnectionSearchQuery;

pub use request::ConnectionSearchTerm;

pub use request::QueryRequest;

pub use response::ConnectionDetailsResponse;

pub use response::ListSummary;

pub use response::TableSummary;

pub use state::QueryState;

Modules

api

HTTP API module for stately-arrow.

backend

Backend trait and metadata types for data source connectors.

context

Query context and session abstractions for DataFusion integration.

database

error

object_store

Object store backend for cloud and local filesystem access.

registry

request

Request types for stately-arrow API endpoints.

response

Response types for stately-arrow API endpoints.

state

Application state for stately-arrow API handlers.