Struct spark_connect_rs::spark::Deduplicate

pub struct Deduplicate {
    pub input: Option<Box<Relation>>,
    pub column_names: Vec<String>,
    pub all_columns_as_keys: Option<bool>,
    pub within_watermark: Option<bool>,
}

Relation of type [Deduplicate] which have duplicate rows removed, could consider either only the subset of columns or all the columns.

Fields

input: Option<Box<Relation>>

(Required) Input relation for a Deduplicate.

column_names: Vec<String>

(Optional) Deduplicate based on a list of column names.

This field does not co-use with all_columns_as_keys.

all_columns_as_keys: Option<bool>

(Optional) Deduplicate based on all the columns of the input relation.

This field does not co-use with column_names.

within_watermark: Option<bool>

(Optional) Deduplicate within the time range of watermark.

Implementations

impl Deduplicate

pub fn all_columns_as_keys(&self) -> bool

Returns the value of all_columns_as_keys, or the default value if all_columns_as_keys is unset.

pub fn within_watermark(&self) -> bool

Returns the value of within_watermark, or the default value if within_watermark is unset.

Trait Implementations

impl Clone for Deduplicate

fn clone(&self) -> Deduplicate

Returns a copy of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for Deduplicate

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for Deduplicate

fn default() -> Self

Returns the “default value” for a type.

impl Message for Deduplicate

fn encoded_len(&self) -> usize

Returns the encoded length of the message without a length delimiter.

fn clear(&mut self)

Clears the message, resetting all fields to their default.

fn encode<B>(&self, buf: &mut B) -> Result<(), EncodeError>where B: BufMut, Self: Sized,

Encodes the message to a buffer.

fn encode_to_vec(&self) -> Vec<u8, Global>where Self: Sized,

Encodes the message to a newly allocated buffer.

fn encode_length_delimited<B>(&self, buf: &mut B) -> Result<(), EncodeError>where B: BufMut, Self: Sized,

Encodes the message with a length-delimiter to a buffer.

fn encode_length_delimited_to_vec(&self) -> Vec<u8, Global>where Self: Sized,

Encodes the message with a length-delimiter to a newly allocated buffer.

fn decode<B>(buf: B) -> Result<Self, DecodeError>where B: Buf, Self: Default,

Decodes an instance of the message from a buffer.

fn decode_length_delimited<B>(buf: B) -> Result<Self, DecodeError>where B: Buf, Self: Default,

Decodes a length-delimited instance of the message from the buffer.

fn merge<B>(&mut self, buf: B) -> Result<(), DecodeError>where B: Buf, Self: Sized,

Decodes an instance of the message from a buffer, and merges it into self.

fn merge_length_delimited<B>(&mut self, buf: B) -> Result<(), DecodeError>where B: Buf, Self: Sized,

Decodes a length-delimited instance of the message from buffer, and merges it into self.

impl PartialEq<Deduplicate> for Deduplicate

fn eq(&self, other: &Deduplicate) -> bool

This method tests for self and other values to be equal, and is used by ==.

fn ne(&self, other: &Rhs) -> bool

This method tests for !=. The default implementation is almost always sufficient, and should not be overridden without very good reason.

impl StructuralPartialEq for Deduplicate