pub enum StopCondition {
MaxRows(u64),
MaxBytes(u64),
SchemaStable {
consecutive_stable_rows: u64,
},
ConfidenceThreshold(f64),
MemoryPressure(f64),
Any(Vec<StopCondition>),
All(Vec<StopCondition>),
Never,
}Expand description
A composable condition that can trigger early termination of profiling.
Conditions other than StopCondition::MaxRows are checked per chunk for
performance. Their observed boundary can therefore vary with chunk size.
Variants§
MaxRows(u64)
Stop after processing this many rows.
MaxBytes(u64)
Stop after consuming this many bytes from the source.
The CSV streaming engines, file and async, size each read to the
budget that remains, so they stop at the end of the first record at or
past the limit: bytes_consumed counts bytes read by the profiling pass,
including the header, and exceeds the value by less than one record.
Fraction stops or progress can require a separate CSV row-count scan
before this budget takes effect. The async JSON reader still evaluates
it after whole chunks.
SchemaStable
Stop when column types have not changed for approximately N rows after the initial schema baseline (accumulated across chunks).
Fields
ConfidenceThreshold(f64)
Stop when rows_processed / estimated_total >= threshold.
This is the fraction of the estimated row count read, rather than a statistical confidence level. Only meaningful when an estimate exists. When no estimate exists, this condition is inert.
MemoryPressure(f64)
Stop when memory usage exceeds this fraction of the configured limit.
Value in 0.0..=1.0 (e.g., 0.9 = 90% of memory limit).
Any(Vec<StopCondition>)
Stop when any sub-condition triggers.
All(Vec<StopCondition>)
Stop when all sub-conditions have triggered.
Never
Never stop early — process the entire stream (default).
Implementations§
Source§impl StopCondition
impl StopCondition
Sourcepub fn schema_inference() -> Self
pub fn schema_inference() -> Self
Preset for schema-only profiling: stop after 10K rows or when schema stabilizes (no type changes for 1,000 consecutive rows).
Sourcepub fn quality_sample() -> Self
pub fn quality_sample() -> Self
Preset for quality sampling: stop after 50K rows, 50 MB, or 95% of the estimated row count.
Sourcepub fn max_rows(&self) -> Option<u64>
pub fn max_rows(&self) -> Option<u64>
The row count at which this condition can first trigger on rows alone, if any.
Mirrors the internal evaluate method: Any fires as soon as one child
fires, so it takes
the minimum of the children that a row count can trigger. All fires
only once every child has fired, so it takes the maximum, and yields
None if any child cannot be triggered by rows at all (a byte cap,
Never, …) — such a condition can never be satisfied by rows alone.
Used two ways: parsers that can only enforce a row cap consult it after
checking is_row_limit_only, and the
incremental engine uses it as a per-row guardrail alongside the real
evaluator.
Sourcepub fn max_bytes(&self) -> Option<u64>
pub fn max_bytes(&self) -> Option<u64>
The byte count at which this condition can first trigger on bytes
alone, if any: the byte counterpart of max_rows,
with the same Any/All rules.
The incremental engine sizes its reads to the budget that remains under it, so the scan stops at a record boundary rather than at the end of a large chunk.
Sourcepub fn is_row_limit_only(&self) -> bool
pub fn is_row_limit_only(&self) -> bool
Whether this condition is expressible purely as a row limit.
Never never fires; a bare MaxRows is a row cap; a composite qualifies
when every leaf is one of those. Anything else (byte caps, schema
stability, confidence) needs a real evaluator, so a parser that can only
cap rows must not silently accept it.
Sourcepub fn needs_row_count(&self) -> bool
pub fn needs_row_count(&self) -> bool
Whether any leaf needs a total row count to evaluate its read fraction.