Crate async_fetcher
source ·Expand description
Provides a high level abstraction over a reqwest async client for optionally fetching files from a remote location when required, checking that the downloaded file is valid, and processing that downloaded file if processing is required (ie: decompression).
Features
- Files will only be downloaded if the timestamp of the file is older than the timestamp on the server.
- Or if the destination checksum does not match, if a checksum is provided.
- Partial downloads will be stored at a temporary location, then renamed after completion.
- The fetched files will have their file times modified to match that of the server they fetched from.
- The checksum of the fetched file can be compared, as well as the checksum of the destination.
- The caller may optionally process the fetched file into the destination.
- Implemented as a state machine for creating requests. These can be converted to futures at any time:
AsyncFetcher->ResponseStateResponseState->FetchedStateFetchedState->CompletedState
Notes
- The generated futures are compatible with multi-threaded runtimes.
- See the examples directory in the source repository for an example of it in practice.
Structs
A future builder for creating futures to fetch files from an asynchronous reqwest client.
The state which signals that fetched file is now at the destination, and provides an optional
checksum comparison method.
An error that occurred when fetching a file with this library.
This state manages renaming to the destination, and setting the timestamp of the fetched file.
This state manages downloading a response into the temporary location.