Struct Store 

pub struct Store { /* private fields */ }

Multi-dataset registry. Each dataset is registered in the shared SessionContext under its configured name. The per-dataset state is held behind ArcSwap so a reload can atomically replace it without blocking concurrent queries.

Implementations

impl Store

pub async fn load(cfg: &AppConfig) -> Result<Self, AppError>

Load every dataset declared in cfg.

pub fn names(&self) -> Vec<String>

Sorted list of dataset names.

pub fn dataset(&self, name: &str) -> Result<Arc<DatasetState>, AppError>

pub async fn sample(&self, name: &str) -> Result<String, AppError>

JSON for the first row of the dataset, or null if empty. Used by GET /api/datasets/{name}/schema for discoverability.

pub async fn reload(&self, name: &str) -> Result<ReloadStats, AppError>

Rebuild name from disk and atomically swap it in. Concurrent queries against the same name continue to see the old Arc<DatasetState> until they finish; the old data is dropped once the last reference goes away.

pub async fn query( &self, name: &str, req: &QueryRequest, ) -> Result<String, AppError>

Run a QueryRequest against name. Empty predicates → O(1) Arrow slice. Otherwise → DataFusion SQL on the single registered table. Lazy datasets skip the in-memory hot paths and always dispatch to SQL.

pub async fn query_sql( &self, sql: &str, max_rows: u64, ) -> Result<String, AppError>

Execute a pre-validated raw SELECT and return the JSON data array. The statement has already passed datapress_core::sql::validate; here it is wrapped in an outer LIMIT max_rows so the result is bounded regardless of the user’s own clauses, executed through the shared SessionContext, and run through the same fast JSON encoder as Self::query.

pub async fn query_sql_arrow_stream( &self, sql: &str, max_rows: u64, ) -> Result<ArrowIpcStream, AppError>

Same plan as Self::query_sql, but encode the bounded result as an Arrow IPC stream instead of JSON. Backs the Arrow content-negotiated branch of POST /api/v1/sql.

pub async fn query_arrow( &self, name: &str, req: &QueryRequest, ) -> Result<Vec<u8>, AppError>

Same plan as Self::query, but encode the result page as an Arrow IPC stream (one schema message + one batch + EOS). Empty results still produce a valid, self-describing zero-batch stream.

pub async fn query_arrow_stream( &self, name: &str, req: &QueryRequest, ) -> Result<ArrowIpcStream, AppError>

pub async fn query_arrow_stream_all( &self, name: &str, req: &QueryRequest, ) -> Result<ArrowIpcStream, AppError>

pub async fn parquet(&self, name: &str) -> Result<Bytes, AppError>

Encode the entire dataset as a single self-contained Parquet file.

Collects every row (all columns, no predicates, no paging) and runs it through a single parquet::arrow::ArrowWriter, so the result carries the row-group + footer metadata a Parquet reader needs to answer count(*) straight from the footer. Powers the cached GET /datasets/{name}/parquet HTTP endpoint.

impl Store

pub async fn count( &self, name: &str, req: &CountRequest, ) -> Result<i64, AppError>

Return the number of rows matching req.predicates. With no predicates this is a cheap metadata lookup on materialised datasets and a SELECT COUNT(*) on lazy ones.

Trait Implementations

impl Backend for Store

fn names(&self) -> Vec<String>

Sorted list of dataset names.

fn summary(&self, name: &str) -> Result<DatasetSummary, AppError>

Cheap summary for the dataset listing endpoint. Err(NotFound) on unknown name.

fn schema(&self, name: &str) -> Result<Arc<DatasetSchema>, AppError>

Full schema for name. Err(NotFound) on unknown name.

fn indexed_columns(&self, name: &str) -> Result<Vec<String>, AppError>

Names of columns the backend has built an equality index over, for inclusion in the /schema response. Default impl returns an empty vec — backends without per-column indexes (e.g. DuckDB, which relies on the embedded database engine) need not override.

fn sample<'life0, 'life1, 'async_trait>( &'life0 self, name: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<String, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

JSON for the first row of the dataset, or the literal string "null" if the dataset is empty.

fn query<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, name: &'life1 str, req: &'life2 QueryRequest, ) -> Pin<Box<dyn Future<Output = Result<String, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Execute req against name, returning the JSON-encoded data array (without the {"data": …, "page": …} envelope — that’s added by the handler).

fn query_arrow<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, name: &'life1 str, req: &'life2 QueryRequest, ) -> Pin<Box<dyn Future<Output = Result<Vec<u8>, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Execute req against name, returning the result as an Arrow IPC stream byte buffer (one schema message + zero or more RecordBatch messages + EOS). The handler ships this verbatim with Content-Type: application/vnd.apache.arrow.stream.

fn query_arrow_stream<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, name: &'life1 str, req: &'life2 QueryRequest, ) -> Pin<Box<dyn Future<Output = Result<ArrowIpcStream, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Execute req and stream the Arrow IPC bytes. The default adapter preserves compatibility for backends that only implement Backend::query_arrow, but high-throughput backends should override this to avoid building one full response buffer.

fn query_arrow_stream_all<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, name: &'life1 str, req: &'life2 QueryRequest, ) -> Pin<Box<dyn Future<Output = Result<ArrowIpcStream, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Execute req and stream all matching Arrow IPC batches in one HTTP response. Unlike Backend::query_arrow_stream, this is not page scoped; limit may still cap the total rows returned.

fn count<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, name: &'life1 str, req: &'life2 CountRequest, ) -> Pin<Box<dyn Future<Output = Result<i64, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Count rows in name matching req.predicates.

fn query_sql<'life0, 'life1, 'async_trait>( &'life0 self, sql: &'life1 str, max_rows: u64, ) -> Pin<Box<dyn Future<Output = Result<String, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Execute a pre-validated raw SELECT and return the JSON-encoded data array (same shape as Backend::query — the handler adds the {"data": …} envelope).

fn query_sql_arrow_stream<'life0, 'life1, 'async_trait>( &'life0 self, sql: &'life1 str, max_rows: u64, ) -> Pin<Box<dyn Future<Output = Result<ArrowIpcStream, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Execute a pre-validated raw SELECT and stream the result as Arrow IPC bytes (one schema message + zero or more RecordBatch messages

fn parquet<'life0, 'life1, 'async_trait>( &'life0 self, name: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Bytes, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Encode the entire dataset as a single self-contained Parquet file, returned as in-memory bytes.

fn reload<'life0, 'life1, 'async_trait>( &'life0 self, name: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<ReloadStats, AppError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Rebuild name from its configured source and atomically swap it in.