hotdata 0.20.0

Powerful data platform API for datasets, queries, and analytics.
Documentation
# CreateIndexRequest

## Properties

Name | Type | Description | Notes
------------ | ------------- | ------------- | -------------
**algorithm** | Option<**Algorithm**> | How a vector index organises the vectors it searches. Omit this field for `hnsw`, which is the default.  `hnsw` — builds a graph of the vectors and keeps it in memory. Searches are very fast, and the memory a search needs grows with the whole table, so a large enough table cannot be served at all.  `ivf` — groups the vectors into clusters and reads only the clusters nearest the search. Searches are considerably slower than `hnsw`, and the memory a search needs follows how much of the index it reads rather than the size of the table, so a table far too large for `hnsw` can still be searched. It keeps a copy of the table's rows beside the vectors so a search is answered without reading the table; that copy is extra storage, and how much depends on `vector_precision`, which decides how compactly the copied vectors are held. Available with the `l2` and `cosine` metrics, for columns that already hold vectors and for text columns with an embedding provider, which generates the vectors. Search an `ivf` index with `ORDER BY vector_distance(column, '<query>') LIMIT <k>` (or the matching distance function and a query vector), naming the columns to return rather than selecting `*`; `vector_search` works with `hnsw` indexes only. Generating embeddings for a very large table (tens of millions of rows) is not supported yet. (enum: hnsw, ivf) | [optional]
**r#async** | Option<**bool**> | When true, create the index as a background job and return a job ID for polling. | [optional][default to false]
**async_after_ms** | Option<**i32**> | If set (requires `async` = true), wait up to this many milliseconds for the index build to finish: if it completes in time the index is returned (201), otherwise a 202 with a job ID to poll. Must be between 1000 and the server maximum; a value out of that range, or set without `async` = true, is rejected with 400. | [optional]
**columns** | **Vec<String>** | Columns to index. Required for all index types. | 
**description** | Option<**String**> | User-facing description of the embedding (e.g., \"product descriptions\"). | [optional]
**dimensions** | Option<**i32**> | Output vector dimensions. Some models support multiple dimension sizes (e.g., OpenAI text-embedding-3-small supports 512 or 1536). If omitted, the model's default dimensions are used | [optional]
**embedding_provider_id** | Option<**String**> | Embedding provider ID. When set for a vector index, the source column is treated as text and embeddings are generated automatically. The vector index is then built on the generated embedding column (`{column}_embedding` by default), with either algorithm. For an `ivf` index the metric must be `l2` or `cosine`: a provider whose metric is `dot` is refused unless `metric` names one of those. Every later load and text search embeds with the same provider, and is refused if the provider's model or endpoint has changed since the index was created; changing its API key is fine. Dropping the index keeps the generated column as an ordinary column, which loads may leave out (its rows get no vector) and which a new embedding index can generate into again. | [optional]
**index_name** | **String** |  | 
**index_type** | Option<**IndexType**> | Index type. `sorted` supports range queries, `bm25` full-text search, and `vector` similarity search. (enum: sorted, bm25, vector) | [optional][default to Sorted]
**metric** | Option<**String**> | Distance metric for vector indexes: \"l2\", \"cosine\", or \"dot\". When omitted, defaults to \"l2\" for float array columns or the provider's preferred metric for text columns with auto-embedding. | [optional]
**nlist** | Option<**i32**> | Number of clusters an `ivf` index divides the vectors into. More clusters means each one holds fewer vectors, so a search of the same effort reads less data. Omit this to let the number be chosen from the table's size. | [optional]
**output_column** | Option<**String**> | Custom name for the generated embedding column. Defaults to `{column}_embedding`. | [optional]
**probe_fraction** | Option<**f64**> | How much of an `ivf` index a search reads, as a fraction greater than 0 and at most 1. Higher finds more of the true nearest neighbours and takes longer. This is a fraction rather than a number of clusters on purpose: the same number of clusters is a different share of the index whenever `nlist` changes, and results would quietly get worse. Omit this for the server's default. | [optional]
**vector_precision** | Option<**VectorPrecision**> | How precisely a vector index stores each number of a vector. Lower precision shrinks the index so a larger table can be indexed within the same memory, and lets searches run on a smaller instance. For an `ivf` index it also shrinks what every search reads, because a search reads part of that stored copy.  Omit this field to get each algorithm's own default: an `hnsw` index stores vectors at the same precision as the column, and an `ivf` index stores them as `int8`.  The quality figures below come from one benchmark — 1536-dimension text embeddings, cosine distance, default search settings — and are a guide, not a guarantee. Other models, dimensions, distance metrics and data distributions behave differently, so measure on your own data before moving a production index to a lower precision.  `float32` — on a `float64` column this halves the index. Widely used embedding models emit 32-bit values, so for those nothing is lost; vectors that genuinely carry more than 32 bits of precision will lose some.  `float16` — half the memory of `float32`. In that benchmark its results matched `float32` to within 0.1 percentage points.  `float8` — a quarter of the memory of `float32`. In that benchmark it scored about 4 percentage points below `float32`, and raising the search effort did not close the gap, so treat the reduction as permanent for a given index.  `float64` — accepted only for a column that already holds double-precision values; it cannot add precision the stored data does not have.  `int8` — for an `ivf` index only, and its default. A quarter of the size of `float32`, which is a quarter of the bytes every search reads. On the benchmark this index was designed against, a search reading every cluster found about 99.5% of the neighbours an exact search finds, which is as accurate as `int8` gets; a search at the default width reads fewer clusters and finds fewer (see `probe_fraction`). With `cosine` that accuracy holds however widely your vectors vary in magnitude; with `l2` it falls as they spread — around 93% of the neighbours once the largest magnitude is about 16 times the smallest, and lower beyond that. Use `float32` instead to store the column as written, at four times the size and four times the bytes per search.  An `ivf` index accepts `int8` and `float32` only: it stores its copy as a table, and the remaining values have no column type to be stored in or are no smaller than `int8`. An `hnsw` index accepts everything except `int8`; `float8` is its 8-bit option.  Changing this means dropping the index and creating it again. It affects only the index: the table's own values are never altered, and text columns indexed with a generated embedding are not re-embedded. (enum: float64, float32, float16, float8, int8) | [optional]

[[Back to Model list]](../README.md#documentation-for-models) [[Back to API list]](../README.md#documentation-for-api-endpoints) [[Back to README]](../README.md)