pub const MAX_DIM: usize = 65_536; // 65_536usizeExpand description
The largest dimension a collection can hold.
Taken from the record format rather than picked again here, because a collection that accepted a vector the log cannot hold would be one that works until the file lands. The largest dimension this version will write or read.
Sixty five thousand is far past every published embedding family, which
stops at a few thousand, and it is small enough that a corrupt dim cannot
ask a reader for a gigabyte. The real limit is lower and it is not this: a
record has to fit a page, so at f32 a vector is bounded by the page size,
and a collection of vectors this long would want the chunked form (05
section 4.4) instead.