Struct deltalake::operations::merge::MergeBuilder
source · pub struct MergeBuilder { /* private fields */ }
Expand description
Merge records into a Delta Table.
Implementations§
source§impl MergeBuilder
impl MergeBuilder
sourcepub fn new<E>(
log_store: Arc<dyn LogStore>,
snapshot: DeltaTableState,
predicate: E,
source: DataFrame
) -> MergeBuilderwhere
E: Into<Expression>,
pub fn new<E>(
log_store: Arc<dyn LogStore>,
snapshot: DeltaTableState,
predicate: E,
source: DataFrame
) -> MergeBuilderwhere
E: Into<Expression>,
Create a new MergeBuilder
sourcepub fn when_matched_update<F>(
self,
builder: F
) -> Result<MergeBuilder, DeltaTableError>
pub fn when_matched_update<F>( self, builder: F ) -> Result<MergeBuilder, DeltaTableError>
Update a target record when it matches with a source record
The update expressions can specify both source and target columns.
Multiple match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of match clauses matter.
#Example
let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
.merge(source, col("target.id").eq(col("source.id")))
.with_source_alias("source")
.with_target_alias("target")
.when_matched_update(|update| {
update
.predicate(col("source.value").lt(lit(0)))
.update("value", lit(0))
.update("modified", col("source.modified"))
})?
.when_matched_update(|update| {
update
.update("value", col("source.value") + lit(1))
.update("modified", col("source.modified"))
})?
.await?
sourcepub fn when_matched_delete<F>(
self,
builder: F
) -> Result<MergeBuilder, DeltaTableError>
pub fn when_matched_delete<F>( self, builder: F ) -> Result<MergeBuilder, DeltaTableError>
Delete a target record when it matches with a source record
Multiple match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of match clauses matter.
#Example
let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
.merge(source, col("target.id").eq(col("source.id")))
.with_source_alias("source")
.with_target_alias("target")
.when_matched_delete(|delete| {
delete.predicate(col("source.delete"))
})?
.await?
sourcepub fn when_not_matched_insert<F>(
self,
builder: F
) -> Result<MergeBuilder, DeltaTableError>
pub fn when_not_matched_insert<F>( self, builder: F ) -> Result<MergeBuilder, DeltaTableError>
Insert a source record when it does not match with a target record
Multiple not match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of not match clauses matter.
#Example
let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
.merge(source, col("target.id").eq(col("source.id")))
.with_source_alias("source")
.with_target_alias("target")
.when_not_matched_insert(|insert| {
insert
.set("id", col("source.id"))
.set("value", col("source.value"))
.set("modified", col("source.modified"))
})?
.await?
sourcepub fn when_not_matched_by_source_update<F>(
self,
builder: F
) -> Result<MergeBuilder, DeltaTableError>
pub fn when_not_matched_by_source_update<F>( self, builder: F ) -> Result<MergeBuilder, DeltaTableError>
Update a target record when it does not match with a source record
The update expressions can specify only target columns.
Multiple source not match clauses can be specified and their predicates are evaluated to determine if the corresponding operation are performed. Only the first clause that results in a satisfy predicate is executed. The order of source not match clauses matter.
#Example
let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
.merge(source, col("target.id").eq(col("source.id")))
.with_source_alias("source")
.with_target_alias("target")
.when_not_matched_by_source_update(|update| {
update
.update("active", lit(false))
.update("to_dt", lit("2023-07-11"))
})?
.await?
sourcepub fn when_not_matched_by_source_delete<F>(
self,
builder: F
) -> Result<MergeBuilder, DeltaTableError>
pub fn when_not_matched_by_source_delete<F>( self, builder: F ) -> Result<MergeBuilder, DeltaTableError>
Delete a target record when it does not match with a source record
Multiple source “not match” clauses can be specified and their predicates are evaluated to determine if the corresponding operations are performed. Only the first clause that results in a satisfy predicate is executed. The order of source “not match” clauses matter.
#Example
let table = open_table("../path/to/table")?;
let (table, metrics) = DeltaOps(table)
.merge(source, col("target.id").eq(col("source.id")))
.with_source_alias("source")
.with_target_alias("target")
.when_not_matched_by_source_delete(|delete| {
delete
})?
.await?
sourcepub fn with_source_alias<S>(self, alias: S) -> MergeBuilderwhere
S: ToString,
pub fn with_source_alias<S>(self, alias: S) -> MergeBuilderwhere
S: ToString,
Rename columns in the source dataset to have a prefix of alias
.original column name
sourcepub fn with_target_alias<S>(self, alias: S) -> MergeBuilderwhere
S: ToString,
pub fn with_target_alias<S>(self, alias: S) -> MergeBuilderwhere
S: ToString,
Rename columns in the target dataset to have a prefix of alias
.original column name
sourcepub fn with_session_state(self, state: SessionState) -> MergeBuilder
pub fn with_session_state(self, state: SessionState) -> MergeBuilder
The Datafusion session state to use
sourcepub fn with_commit_properties(
self,
commit_properties: CommitProperties
) -> MergeBuilder
pub fn with_commit_properties( self, commit_properties: CommitProperties ) -> MergeBuilder
Additional metadata to be added to commit info
sourcepub fn with_writer_properties(
self,
writer_properties: WriterProperties
) -> MergeBuilder
pub fn with_writer_properties( self, writer_properties: WriterProperties ) -> MergeBuilder
Writer properties passed to parquet writer for when fiiles are rewritten
sourcepub fn with_safe_cast(self, safe_cast: bool) -> MergeBuilder
pub fn with_safe_cast(self, safe_cast: bool) -> MergeBuilder
Specify the cast options to use when casting columns that do not match
the table’s schema. When cast_options.safe
is set true then any
failures to cast a datatype will use null instead of returning an error
to the user.
Example (column’s type is int): Input Output 123 -> 123 Test123 -> null
Trait Implementations§
source§impl IntoFuture for MergeBuilder
impl IntoFuture for MergeBuilder
§type Output = Result<(DeltaTable, MergeMetrics), DeltaTableError>
type Output = Result<(DeltaTable, MergeMetrics), DeltaTableError>
§type IntoFuture = Pin<Box<dyn Future<Output = <MergeBuilder as IntoFuture>::Output> + Send>>
type IntoFuture = Pin<Box<dyn Future<Output = <MergeBuilder as IntoFuture>::Output> + Send>>
source§fn into_future(self) -> <MergeBuilder as IntoFuture>::IntoFuture
fn into_future(self) -> <MergeBuilder as IntoFuture>::IntoFuture
Auto Trait Implementations§
impl Freeze for MergeBuilder
impl !RefUnwindSafe for MergeBuilder
impl Send for MergeBuilder
impl Sync for MergeBuilder
impl Unpin for MergeBuilder
impl !UnwindSafe for MergeBuilder
Blanket Implementations§
source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
source§impl<T> Instrument for T
impl<T> Instrument for T
source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
source§impl<T> IntoEither for T
impl<T> IntoEither for T
source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left
is true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read moresource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self
into a Left
variant of Either<Self, Self>
if into_left(&self)
returns true
.
Converts self
into a Right
variant of Either<Self, Self>
otherwise. Read more