DataSource

Trait DataSource 

Source
pub trait DataSource:
    Debug
    + Send
    + Sync {
    // Required methods
    fn register_with_telemetry<'life0, 'life1, 'life2, 'life3, 'async_trait>(
        &'life0 self,
        ctx: &'life1 SessionContext,
        table_name: &'life2 str,
        telemetry: Option<&'life3 Arc<TermTelemetry>>,
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait,
             'life3: 'async_trait;
    fn schema(&self) -> Option<&Arc<Schema>>;
    fn description(&self) -> String;

    // Provided method
    fn register<'life0, 'life1, 'life2, 'async_trait>(
        &'life0 self,
        ctx: &'life1 SessionContext,
        table_name: &'life2 str,
    ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
       where Self: 'async_trait,
             'life0: 'async_trait,
             'life1: 'async_trait,
             'life2: 'async_trait { ... }
}
Expand description

A data source that can be registered with a DataFusion context.

This trait defines the interface for all data sources in the Term library. Implementations should handle schema inference, compression detection, and efficient data loading.

§Examples

use term_guard::sources::{DataSource, CsvSource};

let source = CsvSource::new("data/users.csv")?;
let ctx = SessionContext::new();
source.register(&ctx, "users").await?;

Required Methods§

Source

fn register_with_telemetry<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, ctx: &'life1 SessionContext, table_name: &'life2 str, telemetry: Option<&'life3 Arc<TermTelemetry>>, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Registers this data source with telemetry support.

This is the main implementation method that data sources should override. The default register method delegates to this with None telemetry.

Source

fn schema(&self) -> Option<&Arc<Schema>>

Returns the schema of this data source if known.

This may return None if schema inference hasn’t been performed yet.

Source

fn description(&self) -> String

Returns a human-readable description of this data source.

Provided Methods§

Source

fn register<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, ctx: &'life1 SessionContext, table_name: &'life2 str, ) -> Pin<Box<dyn Future<Output = Result<()>> + Send + 'async_trait>>
where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Registers this data source with the given session context.

This method should handle:

  • Schema inference if not explicitly provided
  • Compression detection and handling
  • Efficient data loading
  • Telemetry spans for data loading operations
§Arguments
  • ctx - The DataFusion session context to register with
  • table_name - The name to register the table as
  • telemetry - Optional telemetry configuration for tracing data loading
§Returns

A Result indicating success or failure

Implementors§