Module datatypes

Expand description

Lance data types, Schema and Field

Structs§

Dictionary
Field: Lance Schema Field
LogicalType: LogicalType is a string presentation of arrow type. to be serialized into protobuf.
Projection: A projection is a selection of fields in a schema
Schema: Lance Schema.
SchemaCompareOptions

BlobHandling: Specifies how to handle blob columns when projecting
BlobKind: Physical storage mode for blob v2 descriptors (one byte, stored in the packed struct column).
BlobVersion: Blob column format version.
Encoding: Encoding enum.
FieldRef: Reference to a field in a schema, either by ID or by path.
NullabilityComparison
OnMissing: What to do when a column is missing in the schema
OnTypeMismatch: What to do on a merge operation if the types of the fields don’t match

BLOB_LOGICAL_TYPE
LANCE_UNENFORCED_CLUSTERING_KEY_POSITION: Use this config key in Arrow field metadata to specify the position of a clustering key column. The value is a 1-based integer indicating the order within the composite clustering key. Clustering key fields are ordered by this position value.
LANCE_UNENFORCED_PRIMARY_KEY: Use this config key in Arrow field metadata to indicate a column is a part of the primary key. The value can be any true values like true, 1, yes (case-insensitive). A primary key column must satisfy: (1) The field, and all its ancestors must not be nullable. (2) The field must be a leaf without child (i.e. it is a primitive data type). (3) The field must not be within a list type.
LANCE_UNENFORCED_PRIMARY_KEY_POSITION: Use this config key in Arrow field metadata to specify the position of a primary key column. The value is a 1-based integer indicating the order within the composite primary key. When specified, primary key fields are ordered by this position value. When not specified, primary key fields are ordered by their schema field id.

BLOB_DESC_FIELD
BLOB_DESC_FIELDS
BLOB_DESC_LANCE_FIELD
BLOB_DESC_TYPE
BLOB_V2_DESC_FIELD
BLOB_V2_DESC_FIELDS
BLOB_V2_DESC_LANCE_FIELD
BLOB_V2_DESC_TYPE
BLOB_V2_USER_FIELDS: Blob v2 user-view struct fields used by internal rewrite paths.
BLOB_V2_USER_TYPE: Blob v2 user-view struct type used by internal rewrite paths.

escape_field_path_for_project: Escape a field path for project
format_field_path: Format a field path, quoting field names that require escaping.
format_field_path_minimal: Like format_field_path, but quotes a segment only when strictly required for the result to round-trip back through parse_field_path.
parse_field_path: Parse a field path that may contain quoted field names.
validate_fixed_size_list_dimensions: Reject FixedSizeList types whose dimension is not a positive integer.