Skip to main content

OpenLineageSqlExt

Trait OpenLineageSqlExt 

Source
pub trait OpenLineageSqlExt: Sized {
    // Required methods
    fn with_lineage(
        self,
        builder: impl Into<Option<OpenLineageBuilder>>,
    ) -> Self;
    fn try_with_lineage_from_env(self) -> Result<Self, ClientError>;
    fn sql_with_lineage<'life0, 'life1, 'async_trait>(
        &'life0 self,
        sql: &'life1 str,
    ) -> Pin<Box<dyn Future<Output = DataFusionResult<DataFrame>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait;
}
Expand description

SessionContext ergonomics for OpenLineage instrumentation.

Two conveniences over OpenLineageBuilder:

  • with_lineage installs the instrumentation on a context in one consume-and-return call, mirroring DataFusion’s own SessionContext::enable_url_table(self) -> Self. It saves the host the SessionContext::new_with_state(builder.instrument(ctx.state())) dance. It takes impl Into<Option<OpenLineageBuilder>>, so ctx.with_lineage(None) uses defaults (a no-op client, default config) and ctx.with_lineage(OpenLineage::builder()…) supplies a configured one.
  • sql_with_lineage runs SQL with lineage for statements DataFusion executes outside the query-planner hook — CREATE TABLE … AS SELECT and CREATE VIEW … AS SELECT. SessionContext::sql dispatches these DDL statements straight to the catalog before the instrumented QueryPlanner ever sees the full plan, so the created dataset is silently dropped from lineage (and a CREATE VIEW emits nothing at all). For every other statement this is exactly SessionContext::sql(...).await and the normal planner path produces the lineage.

Required Methods§

Source

fn with_lineage(self, builder: impl Into<Option<OpenLineageBuilder>>) -> Self

Instrument this context with OpenLineage and return it (consume-and-return, like SessionContext::enable_url_table).

Pass a configured OpenLineageBuilder to control the transport/client, context provider, and config — or None for defaults (a no-op client and OpenLineageConfig::for_datafusion). Because the argument is impl Into<Option<OpenLineageBuilder>>, both ctx.with_lineage(None) and ctx.with_lineage(OpenLineage::builder()…) are valid call sites. For the standard-environment path use try_with_lineage_from_env (it is fallible, so it can’t fold into this signature).

Source

fn try_with_lineage_from_env(self) -> Result<Self, ClientError>

Instrument this context with OpenLineage, deriving everything from the environment (see OpenLineageBuilder::from_env).

§Errors

Returns ClientError if the client cannot be built (e.g. a malformed OPENLINEAGE_URL, or called outside a Tokio runtime).

Source

fn sql_with_lineage<'life0, 'life1, 'async_trait>( &'life0 self, sql: &'life1 str, ) -> Pin<Box<dyn Future<Output = DataFusionResult<DataFrame>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Run sql, emitting OpenLineage events even for the CTAS / CREATE VIEW statements DataFusion executes outside the query-planner hook. Equivalent to SessionContext::sql for every other statement.

Dyn Compatibility§

This trait is not dyn compatible.

In older versions of Rust, dyn compatibility was called "object safety".

Implementations on Foreign Types§

Source§

impl OpenLineageSqlExt for SessionContext

Source§

fn with_lineage(self, builder: impl Into<Option<OpenLineageBuilder>>) -> Self

Source§

fn try_with_lineage_from_env(self) -> Result<Self, ClientError>

Source§

fn sql_with_lineage<'life0, 'life1, 'async_trait>( &'life0 self, sql: &'life1 str, ) -> Pin<Box<dyn Future<Output = DataFusionResult<DataFrame>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Implementors§