rudb_vector/lib.rs
1//! Vectors, physical forms, validity, selection vectors and the string representation.
2//!
3//! Rank 1 in the layer rule. See `xtask/layers.toml` and `spec/18-package-layout.md`.
4//!
5//! This is the interface `spec/07-execution.md` section 7.1 calls the widest one in the system.
6//! Every operator in every later crate is written against it, which is why it is specified before
7//! the first operator exists and changed by RFC afterwards rather than by whoever needs it changed.
8//!
9//! Six pieces:
10//!
11//! - [`Vector`], a typed run of at most [`VECTOR_SIZE`] values in one of four physical forms.
12//! - [`Chunk`], some vectors of the same length, which is what one operator hands the next.
13//! - [`Validity`], which is three cases rather than a bitmap, because knowing there are no nulls is
14//! worth a measurable amount and costs one branch per vector to know.
15//! - [`Selection`], which is what a filter produces instead of compacting.
16//! - [`StringView`] and [`StringColumn`], the 16 byte string with the 4 byte prefix.
17//! - [`Buffer`], the run of values behind a flat vector, which is where the buffer manager arrives.
18//!
19//! Next to them is [`for_each_layout`], which is the list of physical layouts a kernel writes its
20//! loop against. It is here rather than in the kernels because the list is a property of [`Data`],
21//! and a kernel that keeps its own copy of it is a kernel that will one day be missing a type.
22//!
23//! # What is deliberately not here
24//!
25//! Encoded vectors are M3 work, not because they are hard but because they only pay off alongside
26//! the specialization contract that decides when to decode. Borrowed buffers are M2 work, because
27//! there is no buffer manager to borrow from. Nested storage is M2 work for the same reason. Each
28//! of these is a place the interface will grow, and each one is named here so that growing it is a
29//! decision somebody makes rather than something that happens.
30//!
31//! # Unsafe
32//!
33//! This crate is on the list in `spec/16-testing.md` section 16.7 that is allowed `unsafe`, and it
34//! does not use any yet. The safe version is the baseline every unsafe version has to beat on a
35//! benchmark before it lands, so writing it first is not a detour.
36//!
37//! The lint below is `deny` rather than `forbid` for exactly that reason. Denied means an unsafe
38//! block needs an `allow` written next to it, which is a line a reviewer sees. Forbidden would mean
39//! the first genuinely faster kernel has to start by editing this file, and a rule that gets
40//! deleted the first time it is inconvenient was never a rule.
41
42#![deny(unsafe_code)]
43
44pub mod buffer;
45pub mod chunk;
46mod layout;
47pub mod selection;
48pub mod string;
49pub mod validity;
50pub mod vector;
51
52pub use buffer::{Buffer, Pin};
53pub use chunk::Chunk;
54pub use selection::Selection;
55pub use string::{INLINE_LIMIT, StringColumn, StringView};
56pub use validity::{Bitmap, Validity};
57pub use vector::{Data, Form, VECTOR_SIZE, Vector};