Struct NdWriter 

pub struct NdWriter { /* private fields */ }

A writer that makes all its rows visible in one commit.

NdWriter::write buffers rows. When the buffer has about WriteOptions::target_fragment_bytes on disk, as estimated from the fragments written so far, a Tokio task writes it as one Lance fragment, and up to WriteOptions::write_parallelism fragments are written at the same time. Readers do not see the fragments. NdWriter::finish writes the rest and commits all of them in one new table version, in the order of the rows. The writer must run on a Tokio runtime.

An append writer adds new fields to the table before it writes the first fragment with them, so readers can see new empty columns before NdWriter::finish. finish then commits a plain append.

A writer that is aborted or dropped commits nothing, and stops its fragment writes. Its data files stay until a Lance cleanup removes them. One exception: under Casting::Promote, a write that widens the table first commits the rows written before it.

Implementations

impl NdWriter

pub async fn create(uri: &str, options: &WriteOptions) -> Result<Self>

A writer for a new table at uri.

options.mode is WriteMode::Create, which fails when the table exists, or WriteMode::Overwrite, which replaces the table at NdWriter::finish.

pub async fn append(uri: &str, options: &WriteOptions) -> Result<Self>

A writer that appends to the table at uri. Fails when the table does not exist. options.mode is ignored.

pub async fn write( &mut self, rows: impl Stream<Item = Result<NdRecordBatch>>, ) -> Result<()>

Write a stream of rows, without a commit. Each nd record batch is one row.

The writer encodes the rows in groups, see WriteOptions. A group that fails a check, for example a type change, is rejected: the writer returns the error and stays usable, and the groups before it stay written. A failed fragment write makes the writer fail: NdWriter::finish then commits nothing.

pub async fn write_batch(&mut self, batch: RecordBatch) -> Result<()>

Write the rows of an encoded record batch, without a commit.

batch holds nd columns and an optional __global_attrs__ column, as crate::encode_rows gives them. The writer checks the columns, but it does not decode the rows. A batch that fails a check is rejected as a whole, and the writer stays usable. A failed fragment write makes the writer fail.

pub async fn finish(self) -> Result<Dataset>

Commit all written rows in one new table version.

Without rows, a new table or an overwrite fails, and an append returns the table unchanged.

pub fn abort(self)

Drop all written rows without a commit. Stops the fragment writes that run. A drop of the writer does the same.

Trait Implementations

impl Drop for NdWriter

fn drop(&mut self)

Stop the fragment writes that run. Their rows are never committed.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.