# Read Scheduling
After planning, each scan partition contains an ordered list of whole-file or
ranged Parquet tasks. The scheduler starts those tasks gradually and passes
their Arrow batches to the caller through bounded channels.
The defaults favor bounded memory and useful overlap. Tune them only after
measuring a representative workload.
## Bound active reads
Two limits apply before a file task can start:
- The scan-wide limit covers every execution partition in one scan. By default,
it is the partition target multiplied by the per-partition limit.
- The per-partition limit prevents one partition from taking all scan-wide
capacity. Its default is three active file reads.
A task holds both permits until its file stream finishes or is dropped. A task
that cannot get both permits waits without opening the data file.
## Prefetch the next files
The direct Parquet reader can prepare a small number of future file streams
while the current one is being consumed. The default prefetch depth is two per
partition. A value of zero waits until the current file is drained before
starting the next one.
Prefetching preserves task order within each partition. Prepared streams still
hold read permits, so they count against both concurrency limits.
The Delta Kernel reader does not prefetch future files. Within each partition,
its synchronous iterator reads one admitted file at a time through a bounded
blocking handoff. Several partitions can still run at once.
## Apply dynamic partition pruning
The DataFusion adapter can receive partition-only filters after the physical
plan has been built. Before each not-yet-started task is admitted, the reader
takes the latest filter value and checks it against the task's Delta partition
values.
If the filter proves that a task cannot match, the scheduler skips it before it
acquires permits, opens the object store, loads a deletion vector, or performs
schema transforms. Tasks that have already started are not cancelled.
Dynamic pruning is deliberately conservative. Missing or invalid partition
values, incomplete filters, unsupported expressions, and evaluation failures
all fall back to reading the task. This can cost extra I/O, but it cannot remove
rows that might match.
## Choose between ranged and full-file reads
The direct Parquet reader normally asks the object store for the footer and data
ranges needed by the query. This is useful for narrow projections, especially
when the file is large.
For small files that are read broadly, several remote range requests can cost
more than one full request. Setting
`parquet_full_file_read_threshold_bytes` allows the reader to fetch an eligible
whole-file task once and serve its later range reads from an in-memory copy.
Tasks created by intra-file repartitioning keep using remote range requests even
when the physical file is below the threshold.
The buffer belongs to that one file stream and is released when the stream
finishes or is dropped. Its compressed-byte memory cost grows with the chosen
threshold and the number of active file reads. Files above the threshold keep
using normal object-store range requests.
This option does not change projection, row-group pruning, deletion-vector
masking, or Arrow decoding. Keep it disabled unless measurements show that
small-file request overhead matters for your workload.
## Apply backpressure
Each execution partition sends batches through a bounded output channel. Its
default capacity is one batch. When the caller stops polling, producers wait
instead of filling an unbounded queue.
Within a partition, batches stay in task order even when the `Direct` backend has
prefetched later files. DataFusion may still execute several partitions at the
same time.
## Cancel unfinished work
Dropping the output stream tells the scheduler to stop admitting new tasks and
releases queued streams and permits. Errors follow the same bounded path and
prevent unrelated later work from being drained.
Asynchronous reads stop at cancellation boundaries. Synchronous Delta Kernel
work that has already entered a dependency call may continue until its next
safe handoff, but it cannot start a later file after cancellation.
See [execution options](https://mag1cfrog.github.io/delta-arrow-reader/reference/execution-options/) for exact defaults and
[scan metrics](https://mag1cfrog.github.io/delta-arrow-reader/reference/metrics/) for the counters exposed by this process.