Struct fetch_data::FetchData

pub struct FetchData { /* private fields */ }

Used to fetch data files from a URL, if needed. It verifies file contents via a hash.

Thread Safety

FetchData works well with multithreaded testing, It is thread safe (via a Mutex).

Implementations

impl FetchData

pub fn new<S0, S1, S3, S4, S5, S6>(
    registry_contents: S0,
    url_root: S1,
    env_key: S3,
    qualifier: S4,
    organization: S5,
    application: S6
) -> FetchData where
    S0: AsRef<str>,
    S1: AsRef<str>,
    S3: AsRef<str>,
    S4: AsRef<str>,
    S5: AsRef<str>,
    S6: AsRef<str>, 

Create a new FetchData object.

Errors

To make FetchData work well as a static global, new never fails. Instead, FetchData stores any error and returns it when the first call to fetch_file, etc., is made.

Arguments

all inputs are string-like

• registry_contents - Whitespace delimited list of files and hashes. Use Rust’s std::include_str macro to include the contents of a file.
• url_root - Base URL for remote files.
• env_key - Environment variable that may contain the path to the data directory. If not set, the data directory will be create via ProjectDirs and the next three arguments.
• qualifier - The reverse domain name notation of the application, excluding the organization or application name itself.
• organization - The name of the organization that develops this application.
• application - The name of the application itself.

Example

use fetch_data::{FetchData};

// Create a new FetchData instance.
let fetch_data = FetchData::new(
    "small.fam 36e0086c0353ff336d0533330dbacb12c75e37dc3cba174313635b98dfe86ed2
     small.bim 56b6657a3766e2e52273f89d28be6135f9424ca1d204d29f3fa1c5a90eca794e",
    "https://raw.githubusercontent.com/CarlKCarlK/fetch-data/main/tests/data/",
    "BAR_APP_DATA_DIR",
    "com",
    "Foo Corp",
    "Bar App",
    );

// If the local file exists and has the right hash, just return its path.
// Otherwise, download the file, confirm its hash, and return its path.
let local_path = fetch_data.fetch_file("small.bim")?;
assert!(local_path.exists());

pub fn fetch_file<P: AsRef<Path>>(
    &self,
    path: P
) -> Result<PathBuf, FetchDataError>

Fetch data files from a URL, but only if needed. Verify contents via a hash.

Example

use fetch_data::{FetchData};

// Create a new FetchData object.
let fetch_data = FetchData::new(
    "small.fam 36e0086c0353ff336d0533330dbacb12c75e37dc3cba174313635b98dfe86ed2
     small.bim 56b6657a3766e2e52273f89d28be6135f9424ca1d204d29f3fa1c5a90eca794e",
    "https://raw.githubusercontent.com/CarlKCarlK/fetch-data/main/tests/data/",
    "BAR_APP_DATA_DIR",
    "com",
    "Foo Corp",
    "Bar App",
    );

// If the local file exists and has the right hash, just return its path.
// Otherwise, download the file, confirm its hash, and return its path.
let local_path = fetch_data.fetch_file("small.bim")?;
assert!(local_path.exists());

pub fn fetch_files<I, P>(
    &self,
    path_list: I
) -> Result<Vec<PathBuf>, FetchDataError> where
    I: IntoIterator<Item = P>,
    P: AsRef<Path>, 

Given a list of files, returns a list of their local paths. If necessary, the files will be downloaded.

Example

use fetch_data::{FetchData};

// Create a new FetchData instance.
let fetch_data = FetchData::new(
    "small.fam 36e0086c0353ff336d0533330dbacb12c75e37dc3cba174313635b98dfe86ed2
     small.bim 56b6657a3766e2e52273f89d28be6135f9424ca1d204d29f3fa1c5a90eca794e",
    "https://raw.githubusercontent.com/CarlKCarlK/fetch-data/main/tests/data/",
    "BAR_APP_DATA_DIR",
    "com",
    "Foo Corp",
    "Bar App",
    );

// If a local file exists and has the right hash, just return its path
// in a list. Otherwise, download the file, confirm its hash, and return
//  its path in the list.
let local_path_list = fetch_data.fetch_files(["small.bim", "small.bim"])?;
assert!(local_path_list[0].exists() && local_path_list[1].exists());

pub fn gen_registry_contents<I, P>(
    &self,
    path_list: I
) -> Result<String, FetchDataError> where
    I: IntoIterator<Item = P>,
    P: AsRef<Path>, 

Compute registry contents by downloading items and hashing them.

Tips

• If you put the returned contents into a file, you can use Rust’s std::include_str macro to include the contents of that file in FetchData::new.

• Use utility function fetch_data::dir_to_file_list to create a list of files in any local directory. Note the hash is computed on download files, not any original local files.

Example

use fetch_data::{FetchData};

// Create a new FetchData object.
let fetch_data = FetchData::new(
    "", // ignored
    "https://raw.githubusercontent.com/CarlKCarlK/fetch-data/main/tests/data/",
    "BAR_APP_DATA_DIR",
    "com",
    "Foo Corp",
    "Bar App",
    );

// Even if local files exist, download each file. Hash each file. Return the results as a string.
let registry_contents = fetch_data.gen_registry_contents(["small.fam", "small.bim"])?;
println!("{registry_contents}"); // Prints:
                                 // small.fam 36e0086c0353ff336d0533330dbacb12c75e37dc3cba174313635b98dfe86ed2
                                 // small.bim 56b6657a3766e2e52273f89d28be6135f9424ca1d204d29f3fa1c5a90eca794e

pub fn cache_dir(&self) -> Result<PathBuf, FetchDataError>

Return the path to the local cache directory.