Trait ScraperCommon

pub trait ScraperCommon {
    // Required methods
    fn forced_repositories<I, S>(self, repos: I) -> Self
       where I: IntoIterator<Item = S>,
             S: Into<String>,
             Self: Sized;
    fn ignored_repositories<I, S>(self, repos: I) -> Self
       where I: IntoIterator<Item = S>,
             S: Into<String>,
             Self: Sized;
    fn geoapi_servers<I, S>(self, servers: I) -> Result<Self, HostnameError>
       where I: IntoIterator<Item = S>,
             Hostname: TryFrom<S>,
             <Hostname as TryFrom<S>>::Error: Into<HostnameError>,
             Self: Sized;
}

Required Methods

fn forced_repositories<I, S>(self, repos: I) -> Self

where I: IntoIterator<Item = S>, S: Into<String>, Self: Sized,

Add a list of forced repositories to the scraper.

Forced repositories are repositories that will be scraped even if they are not listed in repositories.json. Using this is required if the backend type of any server is S3 as S3 servers do not have a repositories.json file.

fn ignored_repositories<I, S>(self, repos: I) -> Self

where I: IntoIterator<Item = S>, S: Into<String>, Self: Sized,

Add a list of ignored repositories to the scraper.

Ignored repositories are repositories that will not be scraped even if they are listed in repositories.json or are given via the forced_repositories() method. This is useful if you want to exclude certain repositories from the scrape (e.g. dev/test repositories).

If a repository is listed in both the forced and ignored lists, it will NOT be scraped.

There is no attempt to validate the existence of any of the repositories in the ignored list. If a repository is listed in the ignored list but does not exist, it will be silently ignored.

fn geoapi_servers<I, S>(self, servers: I) -> Result<Self, HostnameError>

where I: IntoIterator<Item = S>, Hostname: TryFrom<S>, <Hostname as TryFrom<S>>::Error: Into<HostnameError>, Self: Sized,

Add a list of geoapi servers to the scraper.

Geoapi servers are used to resolve the location of a server. This list contains the servers that will be used for a GeoAPI query and they will be returned in the order of distance from the querier.

Defaults to cvmfs_server_scraper::constants::DEFAULT_GEOAPI_SERVERS.

You may pass either something that can be converted into a Hostname (str/string) or a Hostname directly. If you pass a Hostname, the conversion is infallible so it is safe to unwrap().

Example

use std::convert::TryFrom;
use cvmfs_server_scraper::{Scraper, ScraperCommon, Hostname};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    /// Using strings
    let scraper = Scraper::new()
       .geoapi_servers(vec!["cvmfs-stratum-one.cern.ch", "cvmfs-stratum-one.ihep.ac.cn"])?;

    /// Using Hostname
    let hostnames: Vec<Hostname> = vec!["cvmfs-stratum-one.cern.ch".parse()?, "cvmfs-stratum-one.ihep.ac.cn".parse()?];
    let scraper = Scraper::new()
       .geoapi_servers(hostnames).unwrap();
    Ok(())
}

Implementors

impl ScraperCommon for Scraper<WithServers>

impl ScraperCommon for Scraper<WithoutServers>