1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
//! LLKV: Arrow-Native SQL over Key-Value Storage
//!
//! This crate serves as the primary entrypoint for the LLKV database toolkit.
//! It re-exports the high-level SQL engine and storage abstractions from the
//! underlying `llkv-*` crates so downstream applications see a single surface
//! for planning, execution, and storage.
//!
//! # Why LLKV Exists
//!
//! LLKV explores what an [Apache Arrow](https://arrow.apache.org/)-first SQL stack can look like when layered
//! on top of key-value pagers instead of a purpose-built storage format. The
//! project targets columnar OLAP workloads while insisting on compatibility with
//! the reference SQLite [`sqllogictest`](https://sqlite.org/sqllogictest/doc/trunk/about.wiki) suite. Today every published SQLite test
//! runs unmodified against LLKV, giving the engine a broad SQL regression net as
//! new features land.
//!
//! LLKV also ingests a growing set of DuckDB `sqllogictest` cases. Those tests help
//! keep transaction semantics honest as dual-dialect support takes shape, but the
//! DuckDB integration is early and still expanding.
//!
//! # Story So Far
//!
//! LLKV is an experimental SQL database that layers Arrow columnar storage, a streaming execution
//! engine, and MVCC transaction management on top of key-value pagers. Every crate in this workspace
//! serves that goal, keeping [`arrow::record_batch::RecordBatch`] as the interchange format from
//! storage through execution.
//!
//! The surface begins with [llkv-sql](https://docs.rs/llkv-sql/latest/llkv_sql/), which parses statements via
//! [`sqlparser`](https://docs.rs/sqlparser) and lowers them into execution plans. Those plans feed
//! into [llkv-runtime](https://docs.rs/llkv-runtime/latest/llkv_runtime/), the orchestration layer that injects MVCC
//! metadata, coordinates transactions, and dispatches work across the execution and storage stacks.
//! Query evaluation lives in [llkv-executor](https://docs.rs/llkv-executor/latest/llkv_executor/), which streams Arrow
//! `RecordBatch` results without owning MVCC state, while [llkv-table](https://docs.rs/llkv-table/latest/llkv_table/)
//! enforces schema rules and logical field tracking on top of the column store.
//!
//! At the storage layer, [llkv-column-map](https://docs.rs/llkv-column-map/latest/llkv_column_map/) persists column chunks as
//! Arrow-serialized blobs keyed by pager-managed physical IDs. That layout lets backends such as
//! [`simd-r-drive`] provide zero-copy buffers, and it keeps higher layers working against Arrow data
//! structures end-to-end. Concurrency remains synchronous by default, leaning on [Rayon] and
//! [Crossbeam] while still embedding inside [Tokio] when async orchestration is required.
//!
//! Compatibility is measured continuously. [`llkv_slt_tester`](https://docs.rs/llkv-slt-tester/latest/llkv_slt_tester/)
//! executes SQLite and DuckDB `sqllogictest` suites via the
//! [`sqllogictest` crate](https://crates.io/crates/sqllogictest), CI spans Linux, macOS, and Windows,
//! and the
//! documentation here stays synchronized with the rest of the repository so the rustdoc narrative
//! matches the public design record.
//!
//! # Crate Topology
//!
//! LLKV ships as a layered Cargo workspace where higher crates depend on the ones below while sharing
//! Arrow as their interchange format. The main strata are:
//!
//! - **SQL Interface**: [`llkv-sql`](https://docs.rs/llkv-sql/latest/llkv_sql/) exposes the SQL entry point and delegates
//! planning and execution.
//! - **Query Planning**: [`llkv-plan`](https://docs.rs/llkv-plan/latest/llkv_plan/) and [`llkv-expr`](https://docs.rs/llkv-expr/latest/llkv_expr/)
//! define logical plans and expression ASTs.
//! - **Runtime & Transactions**: [`llkv-runtime`](https://docs.rs/llkv-runtime/latest/llkv_runtime/) orchestrates sessions and
//! MVCC, while [`llkv-transaction`](https://docs.rs/llkv-transaction/latest/llkv_transaction/) manages transaction IDs and
//! snapshot isolation.
//! - **Query Execution**: [`llkv-executor`](https://docs.rs/llkv-executor/latest/llkv_executor/) streams Arrow `RecordBatch`
//! results with help from [`llkv-aggregate`](https://docs.rs/llkv-aggregate/latest/llkv_aggregate/) and
//! [`llkv-join`](https://docs.rs/llkv-join/latest/llkv_join/).
//! - **Table & Metadata**: [`llkv-table`](https://docs.rs/llkv-table/latest/llkv_table/) enforces schemas and catalogs atop
//! [`llkv-column-map`](https://docs.rs/llkv-column-map/latest/llkv_column_map/), the columnar storage engine.
//! - **Storage & I/O**: [`llkv-storage`](https://docs.rs/llkv-storage/latest/llkv_storage/) provides the pager abstraction and
//! integrates with [`simd-r-drive`](https://crates.io/crates/simd-r-drive) backends.
//! - **Supporting crates**: [`llkv-result`](https://docs.rs/llkv-result/latest/llkv_result/) unifies error handling,
//! [`llkv-csv`](https://docs.rs/llkv-csv/latest/llkv_csv/) handles CSV ingestion, [`llkv-test-utils`](https://docs.rs/llkv-test-utils/latest/llkv_test_utils/)
//! supplies testing helpers, and [`llkv-slt-tester`](https://docs.rs/llkv-slt-tester/latest/llkv_slt_tester/) drives SQL logic tests.
//!
//! # MVCC Snapshot Isolation
//!
//! LLKV tracks visibility with MVCC metadata injected into every table: hidden `row_id`,
//! `created_by`, and `deleted_by` columns are managed by the runtime and storage layers. Transactions
//! obtain 64-bit IDs from the [`llkv_transaction`](https://docs.rs/llkv-transaction/latest/llkv_transaction/) stack, capture a
//! snapshot of the last committed transaction, and tag new or modified rows accordingly. Rows are
//! visible when `created_by` is at or below the snapshot watermark and `deleted_by` is absent or
//! greater than that watermark. `UPDATE` and `DELETE` use soft deletes (`deleted_by = txn_id`), so
//! old versions remain until future compaction work lands, and auto-commit statements reuse a fast
//! path that tags rows with the reserved auto-commit ID.
//!
//! Each transaction operates with both a base context (existing tables) and a staging context (new
//! tables created during the transaction). On commit, staged operations replay into the base pager
//! once the transaction watermark advances, preserving snapshot isolation without copying entire
//! tables during the unit of work.
//!
//! # Roadmap Signals
//!
//! Active work centers on extending the transaction lifecycle (the
//! [`TxnIdManager`](https://docs.rs/llkv-transaction/latest/llkv_transaction/mvcc/struct.TxnIdManager.html) still carries TODOs for next-ID
//! management), expanding the constraint system across primary, unique, foreign-key, and check
//! metadata, and tightening performance around Arrow batch sizing and columnar access patterns. The
//! crates in this workspace continue to evolve together, keeping documentation and implementation in
//! lockstep.
//!
//! # Dialect and Tooling Outlook
//!
//! - **SQLite compatibility**: LLKV parses SQLite-flavored SQL, batches `INSERT`
//! statements for throughput, and surfaces results in Arrow form. Passing the
//! upstream `sqllogictest` suites establishes a baseline but does not yet make
//! LLKV a drop-in SQLite replacement.
//! - **DuckDB coverage**: Early DuckDB suites exercise MVCC and typed
//! transaction flows. They chart the roadmap rather than guarantee full DuckDB
//! parity today.
//! - **Tokio-friendly, synchronous core**: Queries execute synchronously by
//! default, delegating concurrency to [Rayon] and [Crossbeam]. Embedders can still
//! tuck the engine inside [Tokio], which is how the SQL Logic Test runner drives
//! concurrent sessions.
//!
//! See [dev-docs/high-level-crate-linkage.md](../dev-docs/high-level-crate-linkage.md)
//! and the [DeepWiki documentation](https://deepwiki.com/jzombie/rust-llkv)
//! for diagrams and extended commentary.
//!
//! # Quick Start
//!
//! Create an in-memory SQL engine and execute queries:
//!
//! ```rust
//! use std::sync::Arc;
//! use llkv::{SqlEngine, storage::MemPager};
//!
//! let engine = SqlEngine::new(Arc::new(MemPager::default()));
//! let results = engine.execute("SELECT 42 AS answer").unwrap();
//! ```
//!
//! # Architecture
//!
//! LLKV is organized as a layered workspace:
//!
//! - **SQL Interface** ([llkv-sql](https://docs.rs/llkv-sql/latest/llkv_sql/)): Parses and executes SQL statements.
//! - **Query Planning** ([llkv-plan](https://docs.rs/llkv-plan/latest/llkv_plan/), [llkv-expr](https://docs.rs/llkv-expr/latest/llkv_expr/)): Defines logical plans and expression ASTs.
//! - **Runtime** ([llkv-runtime](https://docs.rs/llkv-runtime/latest/llkv_runtime/), [llkv-transaction](https://docs.rs/llkv-transaction/latest/llkv_transaction/)): Coordinates MVCC transactions and statement execution.
//! - **Execution** ([llkv-executor](https://docs.rs/llkv-executor/latest/llkv_executor/), [llkv-aggregate](https://docs.rs/llkv-aggregate/latest/llkv_aggregate/), [llkv-join](https://docs.rs/llkv-join/latest/llkv_join/)): Streams Arrow batches through operators.
//! - **Storage** ([llkv-table](https://docs.rs/llkv-table/latest/llkv_table/), [llkv-column-map](https://docs.rs/llkv-column-map/latest/llkv_column_map/), [llkv-storage](https://docs.rs/llkv-storage/latest/llkv_storage/)): Manages columnar storage and pager abstractions.
//!
//! # Re-exports
//!
//! This crate re-exports the following modules for convenient access:
//!
//! - [`SqlEngine`]: The main SQL execution engine.
//! - [`storage`]: Pager abstractions and implementations.
//!
//! [Rayon]: https://docs.rs/rayon
//! [Crossbeam]: https://docs.rs/crossbeam
//! [Tokio]: https://docs.rs/tokio
//! [`simd-r-drive`]: https://crates.io/crates/simd-r-drive
// Re-export the SQL engine as the primary user-facing API
pub use SqlEngine;
// Re-export storage pager abstractions
// Re-export result types for error handling
pub use ;
// Re-export runtime types that users might need when working with statement results
pub use RuntimeStatementResult;