Struct DirectoryNamespace 

pub struct DirectoryNamespace { /* private fields */ }

Directory-based implementation of Lance Namespace.

This implementation stores tables as Lance datasets in a directory structure. It supports local filesystems and cloud storage backends through Lance’s object store.

Manifest-based Listing

When manifest_enabled=true, the namespace uses a special __manifest Lance table to track tables instead of scanning the filesystem. This provides:

• Better performance for listing operations
• Ability to track table metadata
• Foundation for future features like namespaces and table renaming

When dir_listing_enabled=true, the namespace falls back to directory scanning for tables not found in the manifest, enabling gradual migration.

Implementations

impl DirectoryNamespace

pub async fn migrate(&self) -> Result<usize>

Migrate directory-based tables to the manifest.

This is a one-time migration operation that:

1. Scans the directory for existing .lance tables
2. Registers any unmigrated tables in the manifest
3. Returns the count of tables that were migrated

This method is safe to run multiple times - it will skip tables that are already registered in the manifest.

Usage

After creating tables in directory-only mode or dual mode, you can migrate them to the manifest to enable manifest-only mode:

// Create namespace with dual mode (manifest + directory listing)
let namespace = DirectoryNamespaceBuilder::new("/path/to/data")
    .manifest_enabled(true)
    .dir_listing_enabled(true)
    .build()
    .await?;

// ... tables are created and used ...

// Migrate existing directory tables to manifest
let migrated_count = namespace.migrate().await?;
println!("Migrated {} tables", migrated_count);

// Now you can disable directory listing for better performance:
// (requires rebuilding the namespace)
let namespace = DirectoryNamespaceBuilder::new("/path/to/data")
    .manifest_enabled(true)
    .dir_listing_enabled(false)  // All tables now in manifest
    .build()
    .await?;

Returns

Returns the number of tables that were migrated to the manifest.

Errors

Returns an error if:

• Manifest is not enabled
• Directory listing fails
• Manifest registration fails

Trait Implementations

impl Debug for DirectoryNamespace

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Display for DirectoryNamespace

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl LanceNamespace for DirectoryNamespace

fn list_namespaces<'life0, 'async_trait>( &'life0 self, request: ListNamespacesRequest, ) -> Pin<Box<dyn Future<Output = Result<ListNamespacesResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

List namespaces.

fn describe_namespace<'life0, 'async_trait>( &'life0 self, request: DescribeNamespaceRequest, ) -> Pin<Box<dyn Future<Output = Result<DescribeNamespaceResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Describe a namespace.

fn create_namespace<'life0, 'async_trait>( &'life0 self, request: CreateNamespaceRequest, ) -> Pin<Box<dyn Future<Output = Result<CreateNamespaceResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Create a new namespace.

fn drop_namespace<'life0, 'async_trait>( &'life0 self, request: DropNamespaceRequest, ) -> Pin<Box<dyn Future<Output = Result<DropNamespaceResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Drop a namespace.

fn namespace_exists<'life0, 'async_trait>( &'life0 self, request: NamespaceExistsRequest, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Check if a namespace exists.

fn list_tables<'life0, 'async_trait>( &'life0 self, request: ListTablesRequest, ) -> Pin<Box<dyn Future<Output = Result<ListTablesResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

List tables in a namespace.

fn describe_table<'life0, 'async_trait>( &'life0 self, request: DescribeTableRequest, ) -> Pin<Box<dyn Future<Output = Result<DescribeTableResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Describe a table.

fn table_exists<'life0, 'async_trait>( &'life0 self, request: TableExistsRequest, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Check if a table exists.

fn drop_table<'life0, 'async_trait>( &'life0 self, request: DropTableRequest, ) -> Pin<Box<dyn Future<Output = Result<DropTableResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Drop a table.

fn create_table<'life0, 'async_trait>( &'life0 self, request: CreateTableRequest, request_data: Bytes, ) -> Pin<Box<dyn Future<Output = Result<CreateTableResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Create a new table with data from Arrow IPC stream.

fn create_empty_table<'life0, 'async_trait>( &'life0 self, request: CreateEmptyTableRequest, ) -> Pin<Box<dyn Future<Output = Result<CreateEmptyTableResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Create an empty table (metadata only operation).

fn register_table<'life0, 'async_trait>( &'life0 self, request: RegisterTableRequest, ) -> Pin<Box<dyn Future<Output = Result<RegisterTableResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Register a table.

fn deregister_table<'life0, 'async_trait>( &'life0 self, request: DeregisterTableRequest, ) -> Pin<Box<dyn Future<Output = Result<DeregisterTableResponse>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Deregister a table.

fn namespace_id(&self) -> String

Return a human-readable unique identifier for this namespace instance.

fn count_table_rows<'life0, 'async_trait>( &'life0 self, _request: CountTableRowsRequest, ) -> Pin<Box<dyn Future<Output = Result<i64, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Count rows in a table.

fn insert_into_table<'life0, 'async_trait>( &'life0 self, _request: InsertIntoTableRequest, _request_data: Bytes, ) -> Pin<Box<dyn Future<Output = Result<InsertIntoTableResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Insert data into a table.

fn merge_insert_into_table<'life0, 'async_trait>( &'life0 self, _request: MergeInsertIntoTableRequest, _request_data: Bytes, ) -> Pin<Box<dyn Future<Output = Result<MergeInsertIntoTableResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Merge insert data into a table.

fn update_table<'life0, 'async_trait>( &'life0 self, _request: UpdateTableRequest, ) -> Pin<Box<dyn Future<Output = Result<UpdateTableResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Update a table.

fn delete_from_table<'life0, 'async_trait>( &'life0 self, _request: DeleteFromTableRequest, ) -> Pin<Box<dyn Future<Output = Result<DeleteFromTableResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Delete from a table.

fn query_table<'life0, 'async_trait>( &'life0 self, _request: QueryTableRequest, ) -> Pin<Box<dyn Future<Output = Result<Bytes, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Query a table.

fn create_table_index<'life0, 'async_trait>( &'life0 self, _request: CreateTableIndexRequest, ) -> Pin<Box<dyn Future<Output = Result<CreateTableIndexResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Create a table index.

fn list_table_indices<'life0, 'async_trait>( &'life0 self, _request: ListTableIndicesRequest, ) -> Pin<Box<dyn Future<Output = Result<ListTableIndicesResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

List table indices.

fn describe_table_index_stats<'life0, 'async_trait>( &'life0 self, _request: DescribeTableIndexStatsRequest, ) -> Pin<Box<dyn Future<Output = Result<DescribeTableIndexStatsResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Describe table index statistics.

fn describe_transaction<'life0, 'async_trait>( &'life0 self, _request: DescribeTransactionRequest, ) -> Pin<Box<dyn Future<Output = Result<DescribeTransactionResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Describe a transaction.

fn alter_transaction<'life0, 'async_trait>( &'life0 self, _request: AlterTransactionRequest, ) -> Pin<Box<dyn Future<Output = Result<AlterTransactionResponse, Error>> + Send + 'async_trait>>

where 'life0: 'async_trait, Self: 'async_trait,

Alter a transaction.