Module update 

Re-exports

pub use self::new::ChannelCongestion;

Modules

new

upgrade

Structs

AvailableIds

ClearDocuments

ClonableMmap

Wrapper around Mmap allowing to virtually clone grenad-chunks in a parallel process like the indexing.

ConcurrentAvailableIds

A concurrent ID generate that will never return the same ID twice.

DocumentAdditionResult

FacetsUpdateBulk

Algorithm to insert elememts into the facet_id_(string/f64)_docids databases by rebuilding the database “from scratch”.

FacetsUpdateIncrementalInner

Implementation of FacetsUpdateIncremental that is independent of milli’s Index type

GrenadParameters

IndexDocuments

IndexDocumentsConfig

IndexerConfig

KeepFirst

KeepLatestObkv

Only the last value associated with an id is kept.

MergeCboRoaringBitmaps

Do a union of all the CboRoaringBitmaps in the values.

MergeDeladdBtreesetString

Do a union of BtreeSet on both sides of a DelAdd obkv separately and outputs a new DelAdd with both unions.

MergeDeladdCboRoaringBitmaps

Do a union of CboRoaringBitmaps on both sides of a DelAdd obkv separately and outputs a new DelAdd with both unions.

MergeIgnoreValues

Used when trying to merge readers, but you don’t actually care about the values.

MergeRoaringBitmaps

ObkvsKeepLastAdditionMergeDeletions

Merge all the obkvs deletions from the newest to the oldest and keep only the newest additions.

ObkvsMergeAdditionsAndDeletions

Merge all the obkvs from the newest to the oldest.

Settings

Transform

Extract the external ids, deduplicate and compute the new internal documents ids and fields ids, writing all the documents under their internal ids into a final file.

TransformOutput

WordPrefixDocids

WordPrefixIntegerDocids

WordsPrefixesFst

Enums

DocumentId

A type that represents a document id that has been retrieved from a document or auto-generated.

IndexDocumentsMethod

Setting

UpdateIndexingStep

Functions

as_cloneable_grenad^⚠

Safety

create_sorter

A helper function that creates a grenad sorter with the given parameters. The max memory is clamped to something reasonable.

create_writer

extract_finite_float_from_value

Try to extract an f64 from a JSON Value and return the Value in the Err variant if it failed.

fst_stream_into_hashset

Converts an fst Stream into an HashSet of Strings.

fst_stream_into_vec

grenad_obkv_into_chunks

Returns an iterator that outputs grenad readers of obkv documents with a maximum size of approximately documents_chunks_size.

merge_deladd_cbo_roaring_bitmaps_into_cbo_roaring_bitmap

A function that merges a DelAdd of bitmao into an already existing bitmap.

merge_two_del_add_obkvs

serialize_roaring_bitmap

sorter_into_reader

try_split_array_at

Divides one slice into an array and the tail at an index, returns None if N is out of bounds.

try_split_at

Divides one slice into two at an index, returns None if mid is out of bounds.

valid_facet_value

valid_lmdb_key

validate_embedding_settings

write_sorter_into_database

Write provided sorter in database using serialize_value function. merge_values function is used if an entry already exist in the database.

writer_into_reader

Type Aliases

CursorClonableMmap

EitherObkvMerge