Skip to main content

EtaEstimator

Struct EtaEstimator 

Source
pub struct EtaEstimator { /* private fields */ }
Expand description

Predicts how much longer an operation has left, from the Progress events it emits.

§Why not just bytes-done over elapsed

A single bytes-per-second figure is wrong for this crate’s pipeline in three specific ways, and this type exists to correct each:

  1. Two cost regimes. Small files are packed into batches and are syscall-bound — their cost is essentially per-file and barely depends on size. Large files are streamed and are bandwidth-bound. A bytes/sec rate learned during the small-file phase overestimates the large-file phase badly, and vice versa, so the two are measured separately and recombined.
  2. The directory pre-pass isn’t in bytes_total. It runs before Progress::Started is ever emitted and can dominate a run on a slow filesystem (a real exFAT-over-USB copy spent about a minute creating ~7,700 directories). It gets its own per-directory cost term.
  3. Default batch sort is SortOrder::Descending. The largest entries complete first, so the mix observed early in a run is not representative of what’s left — extrapolating remaining work from observed work converges on the wrong answer. Progress::Planned supplies the true split up front instead.

§How wall time is attributed

A second of wall time is charged to every regime that had work in flight during it, not to a single “current” regime. Small and large files genuinely do run at the same time: the dispatcher enqueues every batch before any stream, but a workload small enough to fit inside the concurrency limit starts all of them at once, and then a streaming large file overlaps the entire small-file phase. Charging that second to only one of them leaves the other with work recorded but no elapsed time to divide it by — an infinite rate, or more precisely no usable rate at all.

The regimes are then recombined the way the pipeline actually runs them: the directory pre-pass finishes strictly before dispatch begins, so its cost adds, while small and large files overlap, so theirs is a maximum rather than a sum.

estimate = directories + max(small files, large files)

§Where the numbers come from, in order of authority

  1. EntryProgress samples — bytes observed landing at the destination while a large file is still in flight. The most direct measurement available, and the only one that exists during a single long transfer.
  2. Completed large files — an exact byte count over an exact duration, folded in the same way.
  3. Overall byte throughput — used for outstanding large bytes before either of the above has produced anything. Dominated by batched small files, which pay per-file overhead that streaming doesn’t, so it reads low and the estimate starts pessimistic.

Bytes credited by (1) are not re-counted by (2); a completing entry contributes only what sampling hadn’t already seen.

A copy the filesystem satisfies by copy-on-write (APFS clonefile, reflinks) finishes before the first sample and produces no rate at all — correctly, since there is nothing to wait for. Measured here at 2GB in under a millisecond.

§Usage

use file_engine::EtaEstimator;
use tokio_stream::StreamExt;

let mut handle = engine.copy("src", "dst").start()?;
let mut eta = EtaEstimator::new();

while let Some(progress) = handle.progress().next().await {
    eta.observe(&progress);
    if let Some(remaining) = eta.estimate() {
        println!("{}s remaining", remaining.as_secs());
    }
}

Purely observational: it performs no I/O, spawns nothing, and holds no reference to the running operation. Feeding it events out of order, or only some of them, degrades the estimate but never panics.

Implementations§

Source§

impl EtaEstimator

Source

pub fn new() -> Self

Source

pub fn observe(&mut self, progress: &Progress)

Feeds one event in. Call this for every event on the stream: each one either supplies work done or marks the boundary of a span of wall time, and skipping events costs accuracy in both.

Source

pub fn estimate(&self) -> Option<Duration>

Estimated time remaining, or None while any regime with outstanding work has no measured rate yet — an operation that has only just started genuinely has no basis for an estimate, and reporting nothing is more useful than reporting a fabricated number that collapses by an order of magnitude a second later.

Returns Duration::ZERO once no work is outstanding.

Source

pub fn bytes_per_sec(&self) -> Option<f64>

Observed throughput for large, streamed files, in bytes per second. None until at least one has completed. Deliberately excludes the batched small-file phase, whose cost is per-file rather than per-byte — averaging the two together produces a number that describes neither.

Trait Implementations§

Source§

impl Clone for EtaEstimator

Source§

fn clone(&self) -> EtaEstimator

Returns a duplicate of the value. Read more
1.0.0 (const: unstable) · Source§

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source. Read more
Source§

impl Debug for EtaEstimator

Source§

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter. Read more
Source§

impl Default for EtaEstimator

Source§

fn default() -> Self

Returns the “default value” for a type. Read more

Auto Trait Implementations§

Blanket Implementations§

Source§

impl<T> Any for T
where T: 'static + ?Sized,

Source§

fn type_id(&self) -> TypeId

Gets the TypeId of self. Read more
Source§

impl<T> Borrow<T> for T
where T: ?Sized,

Source§

fn borrow(&self) -> &T

Immutably borrows from an owned value. Read more
Source§

impl<T> BorrowMut<T> for T
where T: ?Sized,

Source§

fn borrow_mut(&mut self) -> &mut T

Mutably borrows from an owned value. Read more
Source§

impl<T> CloneToUninit for T
where T: Clone,

Source§

unsafe fn clone_to_uninit(&self, dest: *mut u8)

🔬This is a nightly-only experimental API. (clone_to_uninit)
Performs copy-assignment from self to dest. Read more
Source§

impl<T> From<T> for T

Source§

fn from(t: T) -> T

Returns the argument unchanged.

Source§

impl<T, U> Into<U> for T
where U: From<T>,

Source§

fn into(self) -> U

Calls U::from(self).

That is, this conversion is whatever the implementation of From<T> for U chooses to do.

Source§

impl<T> ToOwned for T
where T: Clone,

Source§

type Owned = T

The resulting type after obtaining ownership.
Source§

fn to_owned(&self) -> T

Creates owned data from borrowed data, usually by cloning. Read more
Source§

fn clone_into(&self, target: &mut T)

Uses borrowed data to replace owned data, usually by cloning. Read more
Source§

impl<T, U> TryFrom<U> for T
where U: Into<T>,

Source§

type Error = Infallible

The type returned in the event of a conversion error.
Source§

fn try_from(value: U) -> Result<T, <T as TryFrom<U>>::Error>

Performs the conversion.
Source§

impl<T, U> TryInto<U> for T
where U: TryFrom<T>,

Source§

type Error = <U as TryFrom<T>>::Error

The type returned in the event of a conversion error.
Source§

fn try_into(self) -> Result<U, <U as TryFrom<T>>::Error>

Performs the conversion.