Docs.rs
  • datafusion-datasource-49.0.0
    • datafusion-datasource 49.0.0
    • Permalink
    • Docs.rs crate page
    • Apache-2.0
    • Links
    • Homepage
    • Repository
    • crates.io
    • Source
    • Owners
    • andygrove
    • alamb
    • xudong963
    • Dependencies
      • arrow ^55.2.0 normal
      • async-compression ^0.4.19 normal optional
      • async-trait ^0.1.88 normal
      • bytes ^1.10 normal
      • bzip2 ^0.6.0 normal optional
      • chrono ^0.4.41 normal
      • datafusion-common ^49.0.0 normal
      • datafusion-common-runtime ^49.0.0 normal
      • datafusion-execution ^49.0.0 normal
      • datafusion-expr ^49.0.0 normal
      • datafusion-physical-expr ^49.0.0 normal
      • datafusion-physical-expr-common ^49.0.0 normal
      • datafusion-physical-plan ^49.0.0 normal
      • datafusion-session ^49.0.0 normal
      • flate2 ^1.1.2 normal optional
      • futures ^0.3 normal
      • glob ^0.3.0 normal
      • itertools ^0.14 normal
      • log ^0.4 normal
      • object_store ^0.12.3 normal
      • parquet ^55.2.0 normal optional
      • rand ^0.9 normal
      • tempfile ^3 normal optional
      • tokio ^1.46 normal
      • tokio-util ^0.7.15 normal optional
      • url ^2.5.4 normal
      • xz2 ^0.1 normal optional
      • zstd ^0.13 normal optional
      • criterion ^0.5.1 dev
      • tempfile ^3 dev
    • Versions
    • 88.12% of the crate is documented
  • Platform
    • i686-unknown-linux-gnu
    • x86_64-unknown-linux-gnu
  • Feature flags
  • docs.rs
    • About docs.rs
    • Badges
    • Builds
    • Metadata
    • Shorthand URLs
    • Download
    • Rustdoc JSON
    • Build queue
    • Privacy policy
  • Rust
    • Rust website
    • The Book
    • Standard Library API Reference
    • Rust by Example
    • The Cargo Guide
    • Clippy Documentation

Crate datafusion_datasource

logo

datafusion_datasource49.0.0

  • All Items

Crate Items

  • Re-exports
  • Modules
  • Structs
  • Enums
  • Functions
  • Type Aliases

Crates

  • datafusion_datasource

Crate datafusion_datasource

Source
Expand description

A table that uses the ObjectStore listing capability to get the list of files to process.

Re-exports§

pub use self::file::as_file_source;
pub use self::url::ListingTableUrl;

Modules§

decoder
Module containing helper methods for the various file formats See write.rs for write related helper methods
display
file
Common behaviors that every file format needs to implement
file_compression_type
File Compression type abstraction
file_format
Module containing helper methods for the various file formats See write.rs for write related helper methods
file_groups
Logic for managing groups of PartitionedFiles in DataFusion
file_meta
file_scan_config
FileScanConfig to configure scanning of possibly partitioned file sources.
file_sink_config
file_stream
A generic stream over file format readers that can be used by any file format that read its files from start to end.
memory
schema_adapter
SchemaAdapter and SchemaAdapterFactory to adapt file-level record batches to a table schema.
sink
Execution plan for writing data to DataSinks
source
DataSource and DataSourceExec
url
write
Module containing helper methods/traits related to enabling write support for the various file formats

Structs§

FileRange
Only scan a subset of Row Groups from the Parquet file whose data “midpoint” lies within the [start, end) byte offsets. This option can be used to scan non-overlapping sections of a Parquet file in parallel.
PartitionedFile
A single file or part of a file that should be read, along with its schema, statistics and partition column values that need to be appended to each row.

Enums§

RangeCalculation
Represents the possible outcomes of a range calculation.

Functions§

add_row_statsDeprecated
calculate_range
Calculates an appropriate byte range for reading from an object based on the provided metadata.
compute_all_files_statistics
Computes statistics for all files across multiple file groups.
generate_test_files
Generates test files with min-max statistics in different overlap patterns.
verify_sort_integrity
Used by tests and benchmarks

Type Aliases§

PartitionedFileStream
Stream of files get listed from object store

Results

Settings
Help

Query parser error: "Unexpected - (did you mean ->?)".