linkrs-query 0.1.0

Query engine, parser, and executor for Linkrs
Documentation 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77

//! DataChunk: Basic unit of streaming execution
//!
//! A DataChunk represents a fixed-size batch of rows processed in streaming mode.
//! Typical size: 2048 rows (~8MB, OLAP vectorized batch)
//!
//! # Ownership & Memory Accounting Rules
//!
//! - **`rows`**: Owned `Vec<Vec<Value>>`. `Clone` performs an explicit deep
//!   copy (there is no shallow-clone path).
//! - **`schema`**: `Arc<Schema>` — shared reference, cheap `Arc::clone` on `Clone`.
//! - **`layout`**: `Arc<SlotLayout>` — always present; shared reference, cheap on `Clone`.
//! - **`memory_reservation`**: `Option<MemoryReservation>` / `Option<MemoryPoolReservation>` —
//!   ownership stays with the original chunk on `Clone`; the clone gets `None`.
//!   Use `take_memory_reservation()` to transfer ownership explicitly.
//!
//! ## Move-first discipline
//!
//! New code moves rows instead of cloning them:
//! - **`expand_visible_rows()`**: the single terminal move (visible rows out,
//!   expanded by `multiplicity`); used by `LocalChunkCollector` and every
//!   `collect` path.
//! - **`visible_rows()`**: the single borrow of visible rows (selection aware).
//! - **`take_indices`/`slice`**: move a subset of rows into a new chunk
//!   (efficient, uses `std::mem::take` per row), preserving `multiplicity`.
//! - **`Clone`**: the single explicit deep copy. It drops memory reservations
//!   and the deferred-build flag; the new owner must re-account memory via its
//!   own pool/tracker.
//!
//! # Construction Paths
//!
//! - `new_with_layout(rows, layout)` — **Production path**. Schema is derived from
//!   layout slot metadata. Always preferred when a `SlotLayout` is available.
//! - `new(rows, schema)` — Schema-driven path. Layout auto-created from column names.
//! - `from_rows(rows)` / `from_rows_with_col_names(rows, col_names)` — Convenience
//!   constructors for tests. Always produce a layout (auto-created).
//! - `project_columns(columns, layout)` — Designated `Project` output path; rebuilds
//!   the typed layout unconditionally because the output is chunk-bounded.
//!   `from_columns` is the crate-private row-major fixture behind it.
//!
//! # Columnar gating
//!
//! Fresh column sets from storage scans and join outputs honor
//! `use_columnar_path`; small 1:1 rebuilds (project, assign) rebuild
//! unconditionally by design. Blocking accumulation uses `ColumnarBatch`
//! (typed) for sort/topn and `MaterializedBatch` (spill form) elsewhere.

mod collector;
mod columnar_batch;
mod columnar_common;
mod core;
mod eval;
mod kind;
mod policy;
mod pool;
mod schema;
mod selection;
mod typed;

// Re-export public API
pub use collector::{LocalChunkCollector, COLLECTOR_BLOCK_ROWS};
pub use columnar_batch::{BatchColumn, ColumnarBatch};
pub use columnar_common::EdgeHeaderColumn;
pub use core::DataChunk;
pub(crate) use policy::use_columnar_path;
pub use policy::{ColumnarPolicy, QueryColumnarOverride};
pub use pool::{RowBufferPool, MAX_POOLED_ROWS, MAX_POOLED_ROW_CAPACITY};
pub use schema::{ColumnInfo, Schema};
pub(crate) use typed::gather_typed_column;
pub use typed::{TypedColumn, TypedKind};

/// OLAP vectorized batch size.
pub const VECTORIZED_BATCH_SIZE: usize = 2048;
/// Alias for the default chunk size (kept in sync with `ExecutionContext::DEFAULT_CHUNK_SIZE`).
pub const DEFAULT_CHUNK_SIZE: usize = VECTORIZED_BATCH_SIZE;

#[cfg(test)]
mod tests;