Skip to main content

DeltaFunnelSession

Struct DeltaFunnelSession 

Source
pub struct DeltaFunnelSession { /* private fields */ }
Expand description

Rust backing session for lazy query-load workflows.

Implementations§

Source§

impl DeltaFunnelSession

Source

pub fn dry_run_to_mssql( &self, request: &OutputWritePlan, ) -> Result<MssqlDryRunOutputReport, DeltaFunnelError>

Dry-runs one selected lazy table as an MSSQL output.

The method reuses the same session output planner as execute mode, then stops before physical DataFusion planning, row production, SQL Server lifecycle work, bulk writer construction, or row writes.

§Errors

Returns an MSSQL planning error when the request is not in RunMode::DryRun, or the first error from session output planning.

Source

pub fn dry_run_all_to_mssql( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>

Dry-runs multiple selected lazy tables as one MSSQL output workflow.

The method plans each selected output in caller-provided order and stops before cache materialization, physical DataFusion planning, row production, SQL Server lifecycle work, bulk writer construction, or row writes.

§Errors

Returns the first duplicate-output, run-mode, or output-planning error.

Source

pub fn dry_run_all_to_mssql_with_tracing( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>

Runs Self::dry_run_all_to_mssql inside a DeltaFunnel workflow tracing span.

§Errors

Returns the same error as Self::dry_run_all_to_mssql.

Source

pub async fn dry_run_all_to_mssql_with_scan_summary( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>

Dry-runs multiple selected lazy tables and honors source scan-summary options.

This method is async because crate::DryRunScanSummaryMode::ExhaustScanMetadata requires DataFusion physical planning to expose provider scan metadata. It still stops before row production, SQL Server lifecycle work, bulk writer construction, or row writes.

§Errors

Returns the first duplicate-output, run-mode, output-planning, or DataFusion physical-planning error.

Source

pub async fn dry_run_all_to_mssql_with_scan_summary_with_tracing( &self, requests: &[OutputWritePlan], ) -> Result<MssqlDryRunWorkflowReport, DeltaFunnelError>

Runs Self::dry_run_all_to_mssql_with_scan_summary inside a workflow tracing span.

§Errors

Returns the same error as Self::dry_run_all_to_mssql_with_scan_summary.

Source§

impl DeltaFunnelSession

Source

pub async fn table_from_sql( &mut self, sql: &str, ) -> Result<LazyTable, DeltaFunnelError>

Builds a lazy SQL-derived table without registering a query alias.

The SQL must be one read-only tabular query. Planning uses DataFusion to produce a lazy table provider and does not execute rows, contact SQL Server, or create an output target.

§Errors

Returns DeltaFunnelError::SqlTable when the SQL is empty, contains an unsupported or non-read-only statement, or cannot be planned against the session’s registered aliases.

Source

pub fn register_alias( &mut self, name: impl Into<String>, table: &LazyTable, ) -> Result<LazyTable, DeltaFunnelError>

Registers a session-owned alias for a lazy SQL-derived table.

Alias names use the same unquoted identifier rules as Delta source aliases. The alias is registered into the session’s DataFusion catalog only after all local validation succeeds.

§Errors

Returns DeltaFunnelError::InvalidSourceName or DeltaFunnelError::DuplicateSourceName for invalid or ambiguous aliases, and DeltaFunnelError::SqlTable when the table handle is not a pending SQL-derived table owned by this session or DataFusion rejects the alias registration.

Source

pub fn registered_source(&self, name: &str) -> Option<&RegisteredSessionSource>

Finds a registered Delta source by alias using unquoted SQL semantics.

Source

pub fn derived_tables(&self) -> &[RegisteredDerivedTable]

Returns registered SQL-derived aliases in registration order.

Source

pub fn registered_derived_table( &self, name: &str, ) -> Option<&RegisteredDerivedTable>

Finds a registered SQL-derived alias by name using unquoted SQL semantics.

Source§

impl DeltaFunnelSession

Source

pub fn delta_lake( &mut self, source: DeltaSourceConfig, ) -> Result<LazyTable, DeltaFunnelError>

Registers one Delta source and returns its lazy table handle.

The method performs source setup only: Delta snapshot metadata loading, protocol preflight, and DataFusion table registration. It does not scan data files for row production, parse user SQL, contact SQL Server, or execute an output action.

§Errors

Returns the first Delta source loading, protocol preflight, duplicate alias, schema conversion, or DataFusion registration error. Session source state is updated only after the DataFusion registration succeeds.

Source§

impl DeltaFunnelSession

Source

pub fn sources(&self) -> &[RegisteredSessionSource]

Returns registered Delta source reports in registration order.

Source

pub fn source_reports(&self) -> Vec<DeltaSourceReport>

Returns metadata-only Delta source readiness reports in registration order.

Source

pub async fn source_reports_for_lazy_table_plan( &self, table: &LazyTable, ) -> Result<Vec<DeltaSourceReport>, DeltaFunnelError>

Returns Delta source reports enriched from the physical plan for table.

This method resolves the lazy table and builds a DataFusion physical plan so provider planning metadata can be reported. It does not execute row streams or contact SQL Server.

§Errors

Returns an error when the table is unknown or DataFusion cannot build the physical plan.

Source§

impl DeltaFunnelSession

Source

pub fn plan_mssql_output( &self, request: &OutputWritePlan, ) -> Result<PlannedMssqlOutput, DeltaFunnelError>

Plans one lazy table as an MSSQL output without executing the table.

The selected table must be owned by this session. The method uses the table’s logical Arrow schema, resolves the effective SQL Server connection from the output override or session default, and reuses the SQL Server schema, DDL, and lifecycle planning rules. It intentionally performs no SQL Server I/O, physical DataFusion planning, row reads, batch streaming, or writer construction.

§Errors

Returns an MSSQL planning error when the selected table is not known to this session, the target has no effective connection, the output or target config is invalid, the schema cannot be mapped to SQL Server, or the requested load mode is not supported by the current target planner.

Source

pub async fn write_to_mssql( &self, request: &OutputWritePlan, ) -> Result<MssqlWriteReport, DeltaFunnelError>

Writes one selected lazy table to SQL Server.

The method reuses the session output planner, builds a DataFusion physical plan for the selected lazy table, exposes DataFusion’s merged RecordBatch stream, and hands that stream directly to the existing one-output MSSQL sink. It does not implement SQL Server lifecycle, writer, cleanup, retry, or stream buffering behavior itself.

§Errors

Returns the first planning, DataFusion stream setup, upstream stream, SQL Server connection, lifecycle, schema validation, write, or cleanup error.

Source§

impl DeltaFunnelSession

Source

pub async fn write_all( &self, requests: &[OutputWritePlan], ) -> Result<WriteAllReport, DeltaFunnelError>

Writes multiple selected lazy tables to SQL Server sequentially.

The default mode performs conservative automatic cache planning for shared registered derived aliases. The returned report wraps the lower-level SQL Server workflow report and includes cache selection metadata for this call.

§Errors

Returns the first pre-execution validation/planning error, or a workflow execution error from the lower-level SQL Server workflow.

Source

pub async fn write_all_with_options( &self, requests: &[OutputWritePlan], options: WriteAllOptions, ) -> Result<WriteAllReport, DeltaFunnelError>

Writes multiple selected lazy tables to SQL Server sequentially with explicit options.

WriteAllCacheMode::Disabled uses the baseline no-cache path. The default Auto mode performs conservative shared-cache planning and reports the selected or skipped cache decision.

§Errors

Returns the first pre-execution validation/planning error, or a workflow execution error from the lower-level SQL Server workflow.

Source§

impl DeltaFunnelSession

Source

pub fn new(options: SessionOptions) -> Result<Self, DeltaFunnelError>

Builds a new session with validated local options.

§Errors

Returns the first local option validation failure before any source loading, SQL planning, SQL Server connection, or row execution.

Source

pub const fn options(&self) -> &SessionOptions

Returns the validated session options.

Source

pub const fn context(&self) -> &SessionContext

Returns the DataFusion session context owned by this orchestrator.

The session context is exposed so later planning steps can analyze SQL against registered session aliases. Delta source registration should still go through DeltaFunnelSession::delta_lake so the orchestrator’s source reports stay aligned with the DataFusion catalog.

Source

pub const fn next_table_id(&self) -> u64

Returns the next unassigned session-local lazy table id.

Trait Implementations§

Source§

impl Debug for DeltaFunnelSession

Source§

fn fmt(&self, formatter: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> AsAny for T
where T: Any + Send + Sync,

Source§

fn any_ref(&self) -> &(dyn Any + Sync + Send + 'static)

Obtains a dyn Any reference to the object: Read more
Source§

fn as_any(self: Arc<T>) -> Arc<dyn Any + Sync + Send>

Obtains an Arc<dyn Any> reference to the object: Read more
Source§

fn into_any(self: Box<T>) -> Box<dyn Any + Sync + Send>

Converts the object to Box<dyn Any>: Read more
Source§

fn type_name(&self) -> &'static str

Convenient wrapper for std::any::type_name, since Any does not provide it and Any::type_id is useless as a debugging aid (its Debug is just a mess of hex digits).
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
where ST: ?Sized, DT: ?Sized,

Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T> Instrument for T

Source§

fn instrument(self, span: Span) -> Instrumented<Self>

Instruments this type with the provided Span, returning an Instrumented wrapper. Read more
Source§

fn in_current_span(self) -> Instrumented<Self>

Instruments this type with the current Span, returning an Instrumented wrapper. Read more
Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> IntoEither for T

Source§

fn into_either(self, into_left: bool) -> Either<Self, Self>

Converts self into a Left variant of Either<Self, Self> if into_left is true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
where F: FnOnce(&Self) -> bool,

Converts self into a Left variant of Either<Self, Self> if into_left(&self) returns true. Converts self into a Right variant of Either<Self, Self> otherwise. Read more
Source§

impl<T> PolicyExt for T
where T: ?Sized,

Source§

fn and<P, B, E>(self, other: P) -> And<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow only if self and other return Action::Follow. Read more
Source§

fn or<P, B, E>(self, other: P) -> Or<T, P>
where T: Sized + Policy<B, E>, P: Policy<B, E>,

Create a new Policy that returns Action::Follow if either self or other returns Action::Follow. Read more
Source§

impl<T> Read<Exclusive, BecauseExclusive> for T
where T: ?Sized,

Source§

impl<T> Same for T

Source§

type Output = T

Should always be Self
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.
Source§

impl<KernelType, ArrowType> TryIntoArrow<ArrowType> for KernelType
where ArrowType: TryFromKernel<KernelType>,

Source§

fn try_into_arrow(self) -> Result<ArrowType, ArrowError>

Source§

impl<KernelType, ArrowType> TryIntoKernel<KernelType> for ArrowType
where KernelType: TryFromArrow<ArrowType>,

Source§

fn try_into_kernel(self) -> Result<KernelType, ArrowError>

Source§

impl<V, T> VZip<V> for T
where V: MultiLane<T>,

Source§

fn vzip(self) -> V

Source§

impl<T> WithSubscriber for T

Source§

fn with_subscriber<S>(self, subscriber: S) -> WithDispatch<Self>
where S: Into<Dispatch>,

Attaches the provided Subscriber to this type, returning a WithDispatch wrapper. Read more
Source§

fn with_current_subscriber(self) -> WithDispatch<Self>

Attaches the current default Subscriber to this type, returning a WithDispatch wrapper. Read more