pub struct DeltaFunnelSession { /* private fields */ }Expand description
Rust backing session for lazy query-load workflows.
Implementations§
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub fn dry_run_to_mssql(
&self,
request: &OutputWritePlan,
) -> Result<MssqlDryRunOutputReport, DeltaFunnelError>
pub fn dry_run_to_mssql( &self, request: &OutputWritePlan, ) -> Result<MssqlDryRunOutputReport, DeltaFunnelError>
Dry-runs one selected lazy table as an MSSQL output.
The method reuses the same session output planner as execute mode, then stops before physical DataFusion planning, row production, SQL Server lifecycle work, bulk writer construction, or row writes.
§Errors
Returns an MSSQL planning error when the request is not in
RunMode::DryRun, or the first error from session output planning.
Sourcepub fn dry_run_all_to_mssql(
&self,
requests: &[OutputWritePlan],
) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
pub fn dry_run_all_to_mssql( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
Dry-runs multiple selected lazy tables as one MSSQL output workflow.
The method plans each selected output in caller-provided order and stops before cache materialization, physical DataFusion planning, row production, SQL Server lifecycle work, bulk writer construction, or row writes.
§Errors
Returns the first duplicate-output, run-mode, or output-planning error.
Sourcepub fn dry_run_all_to_mssql_with_tracing(
&self,
requests: &[OutputWritePlan],
) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
pub fn dry_run_all_to_mssql_with_tracing( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
Runs Self::dry_run_all_to_mssql inside a DeltaFunnel workflow tracing span.
§Errors
Returns the same error as Self::dry_run_all_to_mssql.
Sourcepub async fn dry_run_all_to_mssql_with_scan_summary(
&self,
requests: &[OutputWritePlan],
) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
pub async fn dry_run_all_to_mssql_with_scan_summary( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
Dry-runs multiple selected lazy tables and honors source scan-summary options.
This method is async because
crate::DryRunScanSummaryMode::ExhaustScanMetadata requires
DataFusion physical planning to expose provider scan metadata. It still
stops before row production, SQL Server lifecycle work, bulk writer
construction, or row writes.
§Errors
Returns the first duplicate-output, run-mode, output-planning, or DataFusion physical-planning error.
Sourcepub async fn dry_run_all_to_mssql_with_scan_summary_with_tracing(
&self,
requests: &[OutputWritePlan],
) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
pub async fn dry_run_all_to_mssql_with_scan_summary_with_tracing( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>
Runs Self::dry_run_all_to_mssql_with_scan_summary inside a workflow tracing span.
§Errors
Returns the same error as Self::dry_run_all_to_mssql_with_scan_summary.
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub async fn table_from_sql(
&mut self,
sql: &str,
) -> Result<LazyTable, DeltaFunnelError>
pub async fn table_from_sql( &mut self, sql: &str, ) -> Result<LazyTable, DeltaFunnelError>
Builds a lazy SQL-derived table without registering a query alias.
The SQL must be one read-only tabular query. Planning uses DataFusion to produce a lazy table provider and does not execute rows, contact SQL Server, or create an output target.
§Errors
Returns DeltaFunnelError::SqlTable when the SQL is empty, contains
an unsupported or non-read-only statement, or cannot be planned against
the session’s registered aliases.
Sourcepub fn register_alias(
&mut self,
name: impl Into<String>,
table: &LazyTable,
) -> Result<LazyTable, DeltaFunnelError>
pub fn register_alias( &mut self, name: impl Into<String>, table: &LazyTable, ) -> Result<LazyTable, DeltaFunnelError>
Registers a session-owned alias for a lazy SQL-derived table.
Alias names use the same unquoted identifier rules as Delta source aliases. The alias is registered into the session’s DataFusion catalog only after all local validation succeeds.
§Errors
Returns DeltaFunnelError::InvalidSourceName or
DeltaFunnelError::DuplicateSourceName for invalid or ambiguous
aliases, and DeltaFunnelError::SqlTable when the table handle is not
a pending SQL-derived table owned by this session or DataFusion rejects
the alias registration.
Sourcepub fn registered_source(&self, name: &str) -> Option<&RegisteredSessionSource>
pub fn registered_source(&self, name: &str) -> Option<&RegisteredSessionSource>
Finds a registered Delta source by alias using unquoted SQL semantics.
Sourcepub fn derived_tables(&self) -> &[RegisteredDerivedTable]
pub fn derived_tables(&self) -> &[RegisteredDerivedTable]
Returns registered SQL-derived aliases in registration order.
Sourcepub fn registered_derived_table(
&self,
name: &str,
) -> Option<&RegisteredDerivedTable>
pub fn registered_derived_table( &self, name: &str, ) -> Option<&RegisteredDerivedTable>
Finds a registered SQL-derived alias by name using unquoted SQL semantics.
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub fn delta_lake(
&mut self,
source: DeltaSourceConfig,
) -> Result<LazyTable, DeltaFunnelError>
pub fn delta_lake( &mut self, source: DeltaSourceConfig, ) -> Result<LazyTable, DeltaFunnelError>
Registers one Delta source and returns its lazy table handle.
The method performs source setup only: Delta snapshot metadata loading, protocol preflight, and DataFusion table registration. It does not scan data files for row production, parse user SQL, contact SQL Server, or execute an output action.
§Errors
Returns the first Delta source loading, protocol preflight, duplicate alias, schema conversion, or DataFusion registration error. Session source state is updated only after the DataFusion registration succeeds.
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub fn sources(&self) -> &[RegisteredSessionSource]
pub fn sources(&self) -> &[RegisteredSessionSource]
Returns registered Delta source reports in registration order.
Sourcepub fn source_reports(&self) -> Vec<DeltaSourceReport>
pub fn source_reports(&self) -> Vec<DeltaSourceReport>
Returns metadata-only Delta source readiness reports in registration order.
Sourcepub async fn source_reports_for_lazy_table_plan(
&self,
table: &LazyTable,
) -> Result<Vec<DeltaSourceReport>, DeltaFunnelError>
pub async fn source_reports_for_lazy_table_plan( &self, table: &LazyTable, ) -> Result<Vec<DeltaSourceReport>, DeltaFunnelError>
Returns Delta source reports enriched from the physical plan for table.
This method resolves the lazy table and builds a DataFusion physical plan so provider planning metadata can be reported. It does not execute row streams or contact SQL Server.
§Errors
Returns an error when the table is unknown or DataFusion cannot build the physical plan.
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub fn plan_mssql_output(
&self,
request: &OutputWritePlan,
) -> Result<PlannedMssqlOutput, DeltaFunnelError>
pub fn plan_mssql_output( &self, request: &OutputWritePlan, ) -> Result<PlannedMssqlOutput, DeltaFunnelError>
Plans one lazy table as an MSSQL output without executing the table.
The selected table must be owned by this session. The method uses the table’s logical Arrow schema, resolves the effective SQL Server connection from the output override or session default, and reuses the SQL Server schema, DDL, and lifecycle planning rules. It intentionally performs no SQL Server I/O, physical DataFusion planning, row reads, batch streaming, or writer construction.
§Errors
Returns an MSSQL planning error when the selected table is not known to this session, the target has no effective connection, the output or target config is invalid, the schema cannot be mapped to SQL Server, or the requested load mode is not supported by the current target planner.
Sourcepub async fn write_to_mssql(
&self,
request: &OutputWritePlan,
) -> Result<MssqlWriteReport, DeltaFunnelError>
pub async fn write_to_mssql( &self, request: &OutputWritePlan, ) -> Result<MssqlWriteReport, DeltaFunnelError>
Writes one selected lazy table to SQL Server.
The method reuses the session output planner, builds a DataFusion physical
plan for the selected lazy table, exposes DataFusion’s merged
RecordBatch stream, and hands that stream directly to the existing
one-output MSSQL sink. It does not implement SQL Server lifecycle,
writer, cleanup, retry, or stream buffering behavior itself.
§Errors
Returns the first planning, DataFusion stream setup, upstream stream, SQL Server connection, lifecycle, schema validation, write, or cleanup error.
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub async fn write_all(
&self,
requests: &[OutputWritePlan],
) -> Result<WriteAllReport, DeltaFunnelError>
pub async fn write_all( &self, requests: &[OutputWritePlan], ) -> Result<WriteAllReport, DeltaFunnelError>
Writes multiple selected lazy tables to SQL Server sequentially.
The default mode performs conservative automatic cache planning for shared registered derived aliases. The returned report wraps the lower-level SQL Server workflow report and includes cache selection metadata for this call.
§Errors
Returns the first pre-execution validation/planning error, or a workflow execution error from the lower-level SQL Server workflow.
Sourcepub async fn write_all_with_options(
&self,
requests: &[OutputWritePlan],
options: WriteAllOptions,
) -> Result<WriteAllReport, DeltaFunnelError>
pub async fn write_all_with_options( &self, requests: &[OutputWritePlan], options: WriteAllOptions, ) -> Result<WriteAllReport, DeltaFunnelError>
Writes multiple selected lazy tables to SQL Server sequentially with explicit options.
WriteAllCacheMode::Disabled uses the baseline no-cache path. The
default Auto mode performs conservative shared-cache planning and
reports the selected or skipped cache decision.
§Errors
Returns the first pre-execution validation/planning error, or a workflow execution error from the lower-level SQL Server workflow.
Source§impl DeltaFunnelSession
impl DeltaFunnelSession
Sourcepub fn new(options: SessionOptions) -> Result<Self, DeltaFunnelError>
pub fn new(options: SessionOptions) -> Result<Self, DeltaFunnelError>
Builds a new session with validated local options.
§Errors
Returns the first local option validation failure before any source loading, SQL planning, SQL Server connection, or row execution.
Sourcepub const fn options(&self) -> &SessionOptions
pub const fn options(&self) -> &SessionOptions
Returns the validated session options.
Sourcepub const fn context(&self) -> &SessionContext
pub const fn context(&self) -> &SessionContext
Returns the DataFusion session context owned by this orchestrator.
The session context is exposed so later planning steps can analyze SQL
against registered session aliases. Delta source registration should
still go through DeltaFunnelSession::delta_lake so the orchestrator’s
source reports stay aligned with the DataFusion catalog.
Sourcepub const fn next_table_id(&self) -> u64
pub const fn next_table_id(&self) -> u64
Returns the next unassigned session-local lazy table id.
Trait Implementations§
Auto Trait Implementations§
impl !RefUnwindSafe for DeltaFunnelSession
impl !UnwindSafe for DeltaFunnelSession
impl Freeze for DeltaFunnelSession
impl Send for DeltaFunnelSession
impl Sync for DeltaFunnelSession
impl Unpin for DeltaFunnelSession
impl UnsafeUnpin for DeltaFunnelSession
Blanket Implementations§
Source§impl<T> AsAny for T
impl<T> AsAny for T
Source§fn any_ref(&self) -> &(dyn Any + Sync + Send + 'static)
fn any_ref(&self) -> &(dyn Any + Sync + Send + 'static)
dyn Any reference to the object: Read moreSource§fn as_any(self: Arc<T>) -> Arc<dyn Any + Sync + Send> ⓘ
fn as_any(self: Arc<T>) -> Arc<dyn Any + Sync + Send> ⓘ
Arc<dyn Any> reference to the object: Read moreSource§fn into_any(self: Box<T>) -> Box<dyn Any + Sync + Send>
fn into_any(self: Box<T>) -> Box<dyn Any + Sync + Send>
Box<dyn Any>: Read moreSource§fn type_name(&self) -> &'static str
fn type_name(&self) -> &'static str
std::any::type_name, since Any does not provide it and
Any::type_id is useless as a debugging aid (its Debug is just a mess of hex digits).Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self>
fn instrument(self, span: Span) -> Instrumented<Self>
Source§fn in_current_span(self) -> Instrumented<Self>
fn in_current_span(self) -> Instrumented<Self>
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more