pub struct OrcFileMetrics {
pub bytes_scanned: Count,
pub file_size: Count,
pub io_requests: Count,
pub metadata_load_time: Time,
pub stripes_pruned_statistics: PruningMetrics,
pub statistics_eval_time: Time,
pub predicate_evaluation_errors: Count,
pub pushdown_rows_pruned: Count,
pub pushdown_rows_matched: Count,
pub decode_time: Time,
pub rows_decoded: Count,
pub batches_produced: Count,
}Expand description
Metrics for ORC file operations.
Tracks performance statistics for reading ORC files, including I/O, metadata loading, stripe pruning, and predicate evaluation.
Fields§
§bytes_scanned: CountTotal number of bytes scanned from the file.
This includes all data read from the object store, including metadata, stripe data, and index data.
file_size: CountTotal file size in bytes (for calculating scan efficiency).
io_requests: CountNumber of I/O requests made to the object store.
metadata_load_time: TimeTime spent reading and parsing ORC file metadata (footer, postscript).
stripes_pruned_statistics: PruningMetricsNumber of stripes pruned or matched by statistics.
This uses PruningMetrics which tracks both pruned and matched counts.
pruned: Stripes skipped due to statistics not matching the predicatematched: Stripes that were read because they might contain matching rows
statistics_eval_time: TimeTime spent evaluating stripe-level statistics for pruning.
predicate_evaluation_errors: CountNumber of times predicate evaluation encountered errors.
This can happen when statistics are malformed or when the predicate cannot be evaluated against the available metadata.
pushdown_rows_pruned: CountNumber of rows filtered out by predicates pushed into the ORC scan.
pushdown_rows_matched: CountNumber of rows that passed predicates pushed into the ORC scan.
decode_time: TimeTime spent decoding ORC data into Arrow arrays.
rows_decoded: CountTotal number of rows decoded from the file.
batches_produced: CountNumber of RecordBatches produced.
Implementations§
Source§impl OrcFileMetrics
impl OrcFileMetrics
Sourcepub fn new(
partition: usize,
filename: &str,
metrics: &ExecutionPlanMetricsSet,
) -> Self
pub fn new( partition: usize, filename: &str, metrics: &ExecutionPlanMetricsSet, ) -> Self
Create new ORC file metrics.
§Arguments
partition- The partition index for this scanfilename- The name of the ORC file being scannedmetrics- The metrics set to register metrics with
Sourcepub fn scan_efficiency(&self) -> Option<f64>
pub fn scan_efficiency(&self) -> Option<f64>
Calculate the scan efficiency ratio.
Returns the ratio of bytes scanned to total file size.
- 1.0 = entire file was read
- 0.5 = half the file was read (due to projection/filtering)
- 0.0 = no data read (all stripes pruned)
Returns None if file size is zero.
Trait Implementations§
Source§impl Clone for OrcFileMetrics
impl Clone for OrcFileMetrics
Source§fn clone(&self) -> OrcFileMetrics
fn clone(&self) -> OrcFileMetrics
1.0.0 · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read moreAuto Trait Implementations§
impl Freeze for OrcFileMetrics
impl RefUnwindSafe for OrcFileMetrics
impl Send for OrcFileMetrics
impl Sync for OrcFileMetrics
impl Unpin for OrcFileMetrics
impl UnwindSafe for OrcFileMetrics
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
Source§impl<T> CloneToUninit for Twhere
T: Clone,
impl<T> CloneToUninit for Twhere
T: Clone,
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self>
fn into_either(self, into_left: bool) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self>
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more