Struct InsertBuilder 

pub struct InsertBuilder<'a> { /* private fields */ }

Insert or create a new dataset.

There are different variants of execute() methods. Those with the _stream suffix take an iterator of data so that larger than memory data can be written out. However, this eliminates optimizations that can be made when the full data is known up-front.

Those with the _uncommitted suffix write the data files but do not commit the transactions. These changes to the dataset will not be visible until they are passed to the CommitBuilder.

Implementations

impl<'a> InsertBuilder<'a>

pub fn new(dest: impl Into<WriteDestination<'a>>) -> Self

pub fn with_params(self, params: &'a WriteParams) -> Self

pub fn progress( self, callback: impl Fn(WriteStats) + Send + Sync + 'static, ) -> Self

Register a callback that is invoked after each batch of rows is written.

The callback receives cumulative WriteStats and can be used to drive a progress bar or compute throughput. It must be cheap and non-blocking; spawn a task if you need async work.

This overrides any write_progress set in WriteParams.

pub async fn execute(&self, data: Vec<RecordBatch>) -> Result<Dataset>

Execute the insert operation with the given data.

This writes the data fragments and commits them into the dataset.

pub async fn execute_stream( &self, source: impl StreamingWriteSource, ) -> Result<Dataset>

Execute the insert operation with the given stream.

This writes the data fragments and commits them into the dataset.

pub async fn execute_uncommitted( &self, data: Vec<RecordBatch>, ) -> Result<Transaction>

Write data files, but don’t commit the transaction yet.

Use CommitBuilder to commit the transaction.

Example: Append data to a dataset

use lance::dataset::{CommitBuilder, InsertBuilder, WriteMode, WriteParams};

let transaction = InsertBuilder::new(dataset.clone())
    .with_params(&WriteParams { mode: WriteMode::Append, ..Default::default() })
    .execute_uncommitted(data)
    .await?;
CommitBuilder::new(dataset)
    .execute(transaction)
    .await?;

pub async fn execute_uncommitted_stream( &self, source: impl StreamingWriteSource, ) -> Result<Transaction>

Write data files, but don’t commit the transaction yet.

Use CommitBuilder to commit the transaction.

Trait Implementations

impl<'a> Clone for InsertBuilder<'a>

fn clone(&self) -> InsertBuilder<'a>

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl<'a> Debug for InsertBuilder<'a>

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.