Struct IvfBuildParams 

pub struct IvfBuildParams {
    pub num_partitions: Option<usize>,
    pub target_partition_size: Option<usize>,
    pub max_iters: usize,
    pub centroids: Option<Arc<FixedSizeListArray>>,
    pub retrain: bool,
    pub sample_rate: usize,
    pub precomputed_partitions_file: Option<String>,
    pub precomputed_shuffle_buffers: Option<(Path, Vec<String>)>,
    pub shuffle_partition_batches: usize,
    pub shuffle_partition_concurrency: usize,
    pub storage_options: Option<HashMap<String, String>>,
}

Parameters to build IVF partitions

Fields

num_partitions: Option<usize>

Deprecated: use target_partition_size instead. Number of partitions to build.

target_partition_size: Option<usize>

Target partition size. If set, the number of partitions will be computed based on the target partition size. Otherwise, the target_partition_size will be set by index type.

max_iters: usize

Max number of iterations to train kmeans.

centroids: Option<Arc<FixedSizeListArray>>

Use provided IVF centroids.

retrain: bool

Retrain centroids. If true, the centroids will be retrained based on provided centroids.

sample_rate: usize

precomputed_partitions_file: Option<String>

Precomputed partitions file (row_id -> partition_id) mutually exclusive with precomputed_shuffle_buffers

precomputed_shuffle_buffers: Option<(Path, Vec<String>)>

Precomputed shuffle buffers (row_id -> partition_id, pq_code) mutually exclusive with precomputed_partitions_file requires centroids to be set

The input is expected to be (/dir/to/buffers, [buffer1.lance, buffer2.lance, …])

shuffle_partition_batches: usize

shuffle_partition_concurrency: usize

storage_options: Option<HashMap<String, String>>

Storage options used to load precomputed partitions.

Implementations

impl IvfBuildParams

pub fn new(num_partitions: usize) -> Self

Create a new instance of IvfBuildParams.

pub fn with_target_partition_size(target_partition_size: usize) -> Self

pub fn try_with_centroids( num_partitions: usize, centroids: Arc<FixedSizeListArray>, ) -> Result<Self>

Create a new instance of IvfBuildParams with centroids.

Trait Implementations

impl Clone for IvfBuildParams

fn clone(&self) -> IvfBuildParams

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for IvfBuildParams

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for IvfBuildParams

fn default() -> Self

Returns the “default value” for a type.