Struct TransformationEngine 

pub struct TransformationEngine {
    pub ctx: SessionContext,
    /* private fields */
}

Fields

ctx: SessionContext

Implementations

impl TransformationEngine

pub fn new() -> Self

pub fn load_project_config( &self, project_dir: &Path, ) -> Result<Arc<RbtProjectConfig>>

Load (or reuse cached) project config for project_dir.

pub fn clear_project_cache(&self)

Clear cached project config (tests / multi-project hosts).

pub async fn register_iceberg_catalog( &self, catalog_name: &str, catalog: Arc<dyn Catalog>, ) -> Result<()>

Registers an Apache Iceberg catalog directly into the DataFusion query context.

pub async fn execute_sql(&self, sql: &str) -> Result<SendableRecordBatchStream>

Executes a SQL transform query against registered tables.

pub async fn execute_dag( &self, dag: &ModelDag, project_dir: impl AsRef<Path>, output_dir: impl AsRef<Path>, ) -> Result<DagExecutionSummary>

Executes a full pipeline DAG tier by tier.

Loads materialize: policy from rbt_project.yml when present (defaults to lake-as-truth Parquet re-read for ref()).

Before any model SQL runs, bronze sources declared in staging frontmatter are registered via register_bronze_sources_for_dag.

pub async fn execute_dag_with_materialize( &self, dag: &ModelDag, project_dir: impl AsRef<Path>, output_dir: impl AsRef<Path>, materialize: &MaterializeConfig, ) -> Result<DagExecutionSummary>

Like [execute_dag] but with an explicit MaterializeConfig (tests / library).

pub async fn preview_model( &self, full_dag: &ModelDag, project_dir: impl AsRef<Path>, output_dir: impl AsRef<Path>, model_name: &str, limit: usize, ) -> Result<PreviewResult>

Preview a single model: materialize ancestors, then run target SQL with LIMIT.

Does not write the target model to the lake. Bronze + ancestor ref() tables are registered as for a normal run. limit is clamped to 1..=10_000.

pub async fn execute_dag_with_config( &self, dag: &ModelDag, project_dir: impl AsRef<Path>, output_dir: impl AsRef<Path>, config: &RbtProjectConfig, ) -> Result<DagExecutionSummary>

Full DAG execution with a pre-loaded project config (roots, scan limits, materialize).

Trait Implementations

impl Default for TransformationEngine

fn default() -> Self

Returns the “default value” for a type.