Struct ScanScheduler 

pub struct ScanScheduler { /* private fields */ }

An I/O scheduler which wraps an ObjectStore and throttles the amount of parallel I/O that can be run.

The ScanScheduler will cancel any outstanding I/O requests when it is dropped. For this reason it should be kept alive until all I/O has finished.

Note: The 2.X file readers already do this so this is only a concern if you are using the ScanScheduler directly.

Implementations

impl ScanScheduler

pub fn new(object_store: Arc<ObjectStore>, config: SchedulerConfig) -> Arc<Self>

Create a new scheduler with the given I/O capacity

Arguments

• object_store - the store to wrap
• config - configuration settings for the scheduler

pub async fn open_file_with_priority( self: &Arc<Self>, path: &Path, base_priority: u64, file_size_bytes: &CachedFileSize, ) -> Result<FileScheduler>

Open a file for reading

Arguments

• path - the path to the file to open
• base_priority - the base priority for I/O requests submitted to this file scheduler this will determine the upper 64 bits of priority (the lower 64 bits come from submit_request and submit_single)

pub async fn open_file( self: &Arc<Self>, path: &Path, file_size_bytes: &CachedFileSize, ) -> Result<FileScheduler>

Open a file with a default priority of 0

See Self::open_file_with_priority for more information on the priority

pub fn open_reader(self: &Arc<Self>, reader: Arc<dyn Reader>) -> FileScheduler

Open a FileScheduler over an already-open Reader.

Unlike Self::open_file, this skips the path lookup and size probe and schedules I/O against reader directly. This is useful when the reader was produced outside the scheduler’s object store (e.g. a spill file opened via crate::spill::Spill::reader), since a bare Reader cannot otherwise drive a v2 FileReader (which needs a scheduler).

Uses a base priority of 0; chain FileScheduler::with_priority to set a different one.

pub fn submit_request( &self, reader: Arc<dyn Reader>, request: Vec<Range<u64>>, priority: u128, bypass_backpressure: bool, ) -> impl Future<Output = Result<Vec<Bytes>>> + Send + use<>

pub fn stats(&self) -> ScanStats

Trait Implementations

impl Debug for ScanScheduler

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Drop for ScanScheduler

fn drop(&mut self)

Executes the destructor for this type.

fn pin_drop(self: Pin<&mut Self>)

🔬This is a nightly-only experimental API. (pin_ergonomics)

Execute the destructor for this type, but different to Drop::drop, it requires self to be pinned.