Crate static_http_cache

Introduction

static_http_cache is a local cache for static HTTP resources.

This library maintains a cache of HTTP resources in a local directory you specify. Whenever you ask it for the contents of a URL, it will re-use a previously-downloaded copy if the resource has not changed on the server. Otherwise, it will download the new version and use that instead.

Because it only supports static resources, static_http_cache only sends HTTP GET requests.

static_http_cache uses the Reqwest crate for HTTP operations, so it should properly handle HTTPS negotiation and use the operating-system’s certificate store.

Currently, static_http_cache only uses the Last-Modified and ETag HTTP headers to determine when its cached data is out of date. Therefore, it’s not suitable for general-purpose HTTP caching; it’s best suited for static content like Amazon S3 data, or Apache or nginx serving up a filesystem directory.

First Example

To use this crate, you need to construct a Cache then call its get method:

extern crate reqwest;
extern crate static_http_cache;

use std::error::Error;
use std::fs::File;
use std::path::PathBuf;

fn get_my_resource() -> Result<File, Box<dyn Error>> {
    let mut cache = static_http_cache::Cache::new(
        PathBuf::from("my_cache_directory"),
        reqwest::blocking::Client::new(),
    )?;

    cache.get(reqwest::Url::parse("http://example.com/some-resource")?)
}

For repeated queries in the same program, you’d probably want to create the Cache once and call get repeatedly, of course.

For a complete, minimal example of how to use static_http_cache, see the included simple example.

Capabilities

Alternative HTTP backends

Although static_http_cache is designed to work with the reqwest library, it will accept any type that implements the traits in the reqwest_mock module. If you want to use it with an alternative HTTP backend, or if you need to stub out network access for testing purposes, you can do that.

Concurrent cache sharing

Cache metadata is stored in a SQLite database, so it’s safe to give different threads (or even different processes) their own Cache instance backed by the same filesystem path.

Note that while it’s safe to have multiple things managing the same cache, it’s not necessarily performant: a Cache instance that’s downloading a new or updated file is likely to stall other cache reads or writes until it’s complete.

Modules

reqwest_mock

Traits describing parts of the reqwest library, so that we can override them in tests.

Structs

Cache

Represents a local cache of HTTP resources.