Struct Shape 

pub struct Shape {
    pub ksim: usize,
    pub dproj: usize,
    pub reps: usize,
}

How big an encoding is and how much accuracy it buys.

The three numbers trade the same way in every experiment in the paper: more buckets and more repetitions track Chamfer more closely and cost a longer encoding, and the encoding’s length is the product of all three.

Fields

ksim: usize

How many random hyperplanes cut the space, so there are 2^ksim buckets.

This is the one to think about against the number of tokens a document has. Buckets well past the token count means most of them are empty and filled from a neighbour, which is not wrong but is not buying anything either.

dproj: usize

How many numbers each bucket’s block is squeezed down to.

Without this a block would be the token dimension long and the encoding would be buckets times that, which at 128 dimensional tokens and sixteen buckets is two thousand numbers for one repetition.

reps: usize

How many times the whole thing is done again with fresh hyperplanes.

Whether a query token and its true best match land in the same bucket is a coin toss that this averages out, and it is the knob that buys accuracy rather than just length.

Trait Implementations

impl Clone for Shape

fn clone(&self) -> Shape

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Copy for Shape

impl Debug for Shape

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Shape

fn default() -> Shape

Sixteen buckets, sixteen numbers a block, eight repetitions, which is two thousand numbers whatever the token dimension is.

That is the middle of the range the paper measures, and it is a reasonable place to start for the hundred or so tokens a passage has. At one bit a dimension it is a 256 byte code, against the 100 token by 128 dimension set it stands in for, which would be 12 kilobytes of floats.