pub struct PackedPayload { /* private fields */ }Expand description
One payload column: all five facts of one object packed adjacently into
a single FixedSizeBinary(33) element.
This is the arm the first two were both missing. FourTables and
OneTableFourColumns differ in how many tables they use and agree in
having five payload columns, so a full-row fetch costs five buffer reads
at five unrelated addresses in both. Here it costs one:
per row, byte for byte (little-endian, no padding, no nulls):
0 u64 offset
8 u64 len
16 u8 object type code (1 commit, 2 tree, 3 blob, 4 tag, 6 ofs-delta, 7 ref-delta)
17 u64 uncompressed_size
25 u64 delta_base — archive offset of the base entry, 0 for none
33 ── next rowIt was 24 bytes narrower before delta_base (PLAN §13). A 33-byte record
spans one 64-byte cache line 49% of the time and two the rest, so a full-row
fetch is ~1.5 lines against the five the columnar arms touch — where at 25
bytes it was ~1.4 against four. The trade moved slightly, in both directions
at once, which is why the numbers below were re-taken rather than scaled.
The oid stays in its own column. It is the key the stree resolves
against, not one of the facts a resolved lookup fetches, so packing it in
would grow every payload read by 20 bytes nobody reads at lookup time.
§The trade, stated before it is measured
- Full-row access should win — one stride instead of four.
- Column scans should lose, and lose badly.
count_typereads one byte per row; the columnar arms streamnbytes to answer it, this arm drags33nthrough cache to usen.sum_uncompressedreads 8 of every 33 instead of 8 of every 8. - It is no longer typed Arrow columns. A
FixedSizeBinary(33)is opaque to DuckDB / Polars / DataFusion, which can query the other two arms directly off the IPC bytes. That is a real cost and it is not a performance one.
Implementations§
Source§impl PackedPayload
impl PackedPayload
Sourcepub fn build_with_oid_layout(
entries: &[IndexEntry],
layout: OidLayout,
) -> Result<Self>
pub fn build_with_oid_layout( entries: &[IndexEntry], layout: OidLayout, ) -> Result<Self>
ObjectIndex::build with the stree section’s OidLayout chosen
explicitly — the whole apparatus of the alignment experiment
(examples/oid_align_bench.rs).
It lives on this arm alone rather than on the trait because
ordinals_batch is the same code in all three arms (they share one
OidResolver), so a second arm would add a second Arrow payload to
keep resident and would measure nothing new. This is the arm the payload
sweep picked.
Sourcepub fn keyspace_phase(&self) -> usize
pub fn keyspace_phase(&self) -> usize
The cache-line phase of this index’s stree keyspace — 0 for
OidLayout::Aligned64. The bench asserts on it before quoting a
number, because two arms that landed on the same phase would be one arm
measured twice.
pub fn oid_column(&self) -> &FixedSizeBinaryArray
pub fn column_is_inside_ipc(&self) -> bool
Trait Implementations§
Source§impl ObjectIndex for PackedPayload
impl ObjectIndex for PackedPayload
Source§fn sum_uncompressed(&self) -> u64
fn sum_uncompressed(&self) -> u64
Strided: 8 useful bytes out of every 25 touched.
Source§fn count_type(&self, t: ObjType) -> usize
fn count_type(&self, t: ObjType) -> usize
Strided: 1 useful byte out of every 25 touched.
fn build(entries: &[IndexEntry]) -> Result<Self>
Source§fn lookup(&self, oid: &[u8]) -> Option<IndexRow>
fn lookup(&self, oid: &[u8]) -> Option<IndexRow>
None for an absent oid and for an oid of the
wrong width.Source§fn lookup_batch(&self, oids: &[&[u8]]) -> Vec<Option<IndexRow>>
fn lookup_batch(&self, oids: &[&[u8]]) -> Vec<Option<IndexRow>>
have negotiation sends up to
1000 oids at a time, the push connectivity check sends thousands.
Positional — out[i] answers oids[i]. Read moreSource§fn ordinals_batch(&self, oids: &[&[u8]]) -> Vec<Option<u32>>
fn ordinals_batch(&self, oids: &[&[u8]]) -> Vec<Option<u32>>
Source§fn extents_batch(&self, oids: &[&[u8]]) -> Vec<Option<(u64, u64)>>
fn extents_batch(&self, oids: &[&[u8]]) -> Vec<Option<(u64, u64)>>
extents(&[oid]), what the wire path actually asks for
when it is about to copy bytes out of a pack.fn name(&self) -> &'static str
fn len(&self) -> usize
Source§fn ipc_bytes(&self) -> usize
fn ipc_bytes(&self) -> usize
Source§fn resident_bytes(&self) -> usize
fn resident_bytes(&self) -> usize
stree oid section — everything the index keeps
alive, excluding the handful of Arc’d schema/metadata allocations.