Enum criner_cli::SubCommands

pub enum SubCommands {
    Mine {
        no_gui: bool,
        fps: f32,
        progress_message_scrollback_buffer_size: usize,
        io_bound_processors: u32,
        cpu_o_bound_processors: u32,
        cpu_bound_processors: u32,
        repository: Option<PathBuf>,
        time_limit: Option<Duration>,
        fetch_every: Duration,
        fetch_at_most: Option<usize>,
        process_every: Duration,
        process_at_most: Option<usize>,
        report_every: Duration,
        report_at_most: Option<usize>,
        download_crates_io_database_every_24_hours_starting_at: Option<Time>,
        glob: Option<String>,
        db_path: PathBuf,
    },
    Export {
        input_db_path: PathBuf,
        export_db_path: PathBuf,
    },
}

Variants

Mine

Mine crates.io in an incorruptible and resumable fashion

Fields of Mine

no_gui: bool

If set, no gui will be presented. Best with RUST_LOG=info to see basic information.

fps: f32

The amount of frames to show per second

progress_message_scrollback_buffer_size: usize

The amount of progress messages to keep in a ring buffer.

io_bound_processors: u32

The amount of IO-bound processors to run concurrently.

A way to choose a value is to see which part of the I/O is actually the bottle neck. Depending on that number, one should experiment with an amount of processors that saturate either input or output. Most commonly, these are bound to the input, as it is the network.

cpu_o_bound_processors: u32

The amount of CPU- and Output-bound processors to run concurrently.

These will perform a computation followed by flushing its result to disk in the form of multiple small files. It's recommended to adjust that number to whatever can saturate the speed of writing to disk, as these processors will yield when writing, allowing other processors to compute. Computes are relatively inexpensive compared to the writes.

cpu_bound_processors: u32

The amount of CPU-bound processors to run concurrently.

One can assume that one of these can occupy one core of a CPU. However, they will not use a lot of IO, nor will they use much memory.

repository: Option<PathBuf>

Path to the possibly existing crates.io repository clone. If unset, it will be cloned to a temporary spot.

time_limit: Option<Duration>

The amount of time we can take for the computation. Specified in humantime, like 10s, 5min, or 2h, or '3h 2min 2s'

fetch_every: Duration

The time between each fetch operation, specified in humantime, like 10s, 5min, or 2h, or '3h 2min 2s'

fetch_at_most: Option<usize>

If set, the amount of times the fetch stage will run. If set to 0, it will never run.

process_every: Duration

The time between each processing run, specified in humantime, like 10s, 5min, or 2h, or '3h 2min 2s'

process_at_most: Option<usize>

If set, the amount of times the process stage will run. If set to 0, they will never run.

report_every: Duration

The time between each reporting and processing run, specified in humantime, like 10s, 5min, or 2h, or '3h 2min 2s'

report_at_most: Option<usize>

If set, the amount of times the reporting stage will run. If set to 0, they will never run.

download_crates_io_database_every_24_hours_starting_at: Option<Time>

If set, declare at which local time to download the crates.io database and digest it.

This job runs every 24h, as the database is updated that often. If unset, the job starts right away. Format is HH:MM, e.g. '14:30' for 2:30 pm or 03:15 for quarter past 3 in the morning.

glob: Option<String>

If set, the reporting stage will only iterate over crates that match the given standard unix glob.

moz* would match only crates starting with 'moz' for example.

db_path: PathBuf

Path to the possibly existing database. It's used to persist all mining results.

Export

Export all Criner data into a format friendly for exploration via SQL, best viewed with https://sqlitebrowser.org

Criner stores binary blobs internally and migrates them on the fly, which is optimized for raw performance. It's also impractical for exploring the data by hand, so the exported data will explode all types into tables with each column being a field. Foreign key relations are set accordingly to allow joins. Use this to get an overview of what's available, and possibly contribute a report generator which implements a query using raw data and writes it into reports.

Fields of Export

input_db_path: PathBuf

The path to the source database in sqlite format

export_db_path: PathBuf

Path to which to write the exported data. If it exists the operation will fail.

Trait Implementations

impl Debug for SubCommands

fn fmt(&self, f: &mut Formatter) -> Result

Formats the value using the given formatter.

impl Default for SubCommands

fn default() -> Self

Returns the "default value" for a type.

impl StructOpt for SubCommands

fn clap<'a, 'b>() -> App<'a, 'b>

Returns the corresponding clap::App.

fn from_clap(matches: &ArgMatches) -> Self

Creates the struct from clap::ArgMatches. It cannot fail with a parameter generated by clap by construction.

fn from_args() -> Self

Gets the struct from the command line arguments. Print the error message and quit the program in case of failure.

fn from_iter<I>(iter: I) -> Self where
    I: IntoIterator,
    <I as IntoIterator>::Item: Into<OsString>,
    <I as IntoIterator>::Item: Clone, 

Gets the struct from any iterator such as a Vec of your making. Print the error message and quit the program in case of failure.

fn from_iter_safe<I>(iter: I) -> Result<Self, Error> where
    I: IntoIterator,
    <I as IntoIterator>::Item: Into<OsString>,
    <I as IntoIterator>::Item: Clone, 

Gets the struct from any iterator such as a Vec of your making.

impl StructOptInternal for SubCommands

fn augment_clap<'a, 'b>(app: App<'a, 'b>) -> App<'a, 'b>

fn from_subcommand<'a, 'b>(
    sub: (&'b str, Option<&'b ArgMatches<'a>>)
) -> Option<Self>

fn is_subcommand() -> bool