pub struct EtaEstimator { /* private fields */ }Expand description
Predicts how much longer an operation has left, from the Progress
events it emits.
§Why not just bytes-done over elapsed
A single bytes-per-second figure is wrong for this crate’s pipeline in three specific ways, and this type exists to correct each:
- Two cost regimes. Small files are packed into batches and are syscall-bound — their cost is essentially per-file and barely depends on size. Large files are streamed and are bandwidth-bound. A bytes/sec rate learned during the small-file phase overestimates the large-file phase badly, and vice versa, so the two are measured separately and recombined.
- The directory pre-pass isn’t in
bytes_total. It runs beforeProgress::Startedis ever emitted and can dominate a run on a slow filesystem (a real exFAT-over-USB copy spent about a minute creating ~7,700 directories). It gets its own per-directory cost term. - Default batch sort is
SortOrder::Descending. The largest entries complete first, so the mix observed early in a run is not representative of what’s left — extrapolating remaining work from observed work converges on the wrong answer.Progress::Plannedsupplies the true split up front instead.
§How wall time is attributed
A second of wall time is charged to every regime that had work in flight during it, not to a single “current” regime. Small and large files genuinely do run at the same time: the dispatcher enqueues every batch before any stream, but a workload small enough to fit inside the concurrency limit starts all of them at once, and then a streaming large file overlaps the entire small-file phase. Charging that second to only one of them leaves the other with work recorded but no elapsed time to divide it by — an infinite rate, or more precisely no usable rate at all.
The regimes are then recombined the way the pipeline actually runs them: the directory pre-pass finishes strictly before dispatch begins, so its cost adds, while small and large files overlap, so theirs is a maximum rather than a sum.
estimate = directories + max(small files, large files)§Where the numbers come from, in order of authority
EntryProgresssamples — bytes observed landing at the destination while a large file is still in flight. The most direct measurement available, and the only one that exists during a single long transfer.- Completed large files — an exact byte count over an exact duration, folded in the same way.
- Overall byte throughput — used for outstanding large bytes before either of the above has produced anything. Dominated by batched small files, which pay per-file overhead that streaming doesn’t, so it reads low and the estimate starts pessimistic.
Bytes credited by (1) are not re-counted by (2); a completing entry contributes only what sampling hadn’t already seen.
A copy the filesystem satisfies by copy-on-write (APFS clonefile,
reflinks) finishes before the first sample and produces no rate at all
— correctly, since there is nothing to wait for. Measured here at 2GB
in under a millisecond.
§Usage
use file_engine::EtaEstimator;
use tokio_stream::StreamExt;
let mut handle = engine.copy("src", "dst").start()?;
let mut eta = EtaEstimator::new();
while let Some(progress) = handle.progress().next().await {
eta.observe(&progress);
if let Some(remaining) = eta.estimate() {
println!("{}s remaining", remaining.as_secs());
}
}Purely observational: it performs no I/O, spawns nothing, and holds no reference to the running operation. Feeding it events out of order, or only some of them, degrades the estimate but never panics.
Implementations§
Source§impl EtaEstimator
impl EtaEstimator
pub fn new() -> Self
Sourcepub fn observe(&mut self, progress: &Progress)
pub fn observe(&mut self, progress: &Progress)
Feeds one event in. Call this for every event on the stream: each one either supplies work done or marks the boundary of a span of wall time, and skipping events costs accuracy in both.
Sourcepub fn estimate(&self) -> Option<Duration>
pub fn estimate(&self) -> Option<Duration>
Estimated time remaining, or None while any regime with
outstanding work has no measured rate yet — an operation that has
only just started genuinely has no basis for an estimate, and
reporting nothing is more useful than reporting a fabricated
number that collapses by an order of magnitude a second later.
Returns Duration::ZERO once no work is outstanding.
Sourcepub fn bytes_per_sec(&self) -> Option<f64>
pub fn bytes_per_sec(&self) -> Option<f64>
Observed throughput for large, streamed files, in bytes per second.
None until at least one has completed. Deliberately excludes the
batched small-file phase, whose cost is per-file rather than
per-byte — averaging the two together produces a number that
describes neither.
Trait Implementations§
Source§impl Clone for EtaEstimator
impl Clone for EtaEstimator
Source§fn clone(&self) -> EtaEstimator
fn clone(&self) -> EtaEstimator
1.0.0 (const: unstable) · Source§fn clone_from(&mut self, source: &Self)
fn clone_from(&mut self, source: &Self)
source. Read more