Struct TableMapping 

pub struct TableMapping {
    pub table: String,
    pub mode: SyncMode,
    pub pipeline: Vec<PipelineStep>,
    pub columns: Vec<String>,
    pub filter: Option<RowFilter>,
}

Fields

table: String

Fully qualified table name as Spark sees it.

mode: SyncMode

pipeline: Vec<PipelineStep>

columns: Vec<String>

Projection override. The columns the source is allowed to read. Empty (the default) means derive it from the pipeline — see projected_columns — which is what you want almost always; set it only to read a column the pipeline does not name (a custom transform’s input, say).

columns: [id, name, country]

filter: Option<RowFilter>

Predicate pushdown. An optional row filter pushed into the Iceberg scan planner, so partitions / data files / row groups that cannot match are never opened.

filter: { column: country, eq: SE }

Implementations

impl TableMapping

pub fn projected_columns(&self) -> Option<Vec<String>>

The columns this mapping actually touches — the projection a source may prune to.

Every declarative step names its inputs exactly: a vertex reads its id_column plus its properties; an edge/fk_edges reads its two endpoint columns plus its properties. A mapping that builds a Person node out of (id, name) therefore needs 2 of the table’s columns — and until the projection was wired, the Iceberg source read all of them on every snapshot change.

Returns None when the projection would be UNSOUND:

• a custom step is an opaque registry transform that may read any column, so pruning could starve it;
• an explicit columns override is absent AND the derived set is empty (nothing to prune to).

None means “read every column”, i.e. the previous behaviour — the projection is a strict optimisation that can only ever narrow a read it is provably safe to narrow.

pub fn compile( &self, registry: &TransformRegistry, ) -> Result<Vec<Arc<dyn Transform>>>

Resolve the pipeline into executable transforms.

Trait Implementations

impl Clone for TableMapping

fn clone(&self) -> Self

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for TableMapping

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl<'de> Deserialize<'de> for TableMapping

fn deserialize<__D>(__deserializer: __D) -> Result<Self, __D::Error>

where __D: Deserializer<'de>,

Deserialize this value from the given Serde deserializer.

impl Serialize for TableMapping

fn serialize<__S>(&self, __serializer: __S) -> Result<__S::Ok, __S::Error>

where __S: Serializer,

Serialize this value into the given Serde serializer.