Enum SubCommands

pub enum SubCommands {
    Mine {

        no_gui: bool,
        fps: f32,
        progress_message_scrollback_buffer_size: usize,
        no_db_download: bool,
        io_bound_processors: u32,
        cpu_o_bound_processors: u32,
        cpu_bound_processors: u32,
        repository: Option<PathBuf>,
        time_limit: Option<Duration>,
        fetch_every: Duration,
        fetch_at_most: Option<usize>,
        process_every: Duration,
        process_at_most: Option<usize>,
        report_every: Duration,
        report_at_most: Option<usize>,
        download_crates_io_database_every_24_hours_starting_at: Option<Time>,
        glob: Option<String>,
        db_path: PathBuf,
    
},
    Export {
        input_db_path: PathBuf,
        export_db_path: PathBuf,
    },
}

Variants

Mine

Mine crates.io in an incorruptible and resumable fashion

Fields

no_gui: bool

If set, no gui will be presented. Best with RUST_LOG=info to see basic information.

fps: f32

The amount of frames to show per second

progress_message_scrollback_buffer_size: usize

The amount of progress messages to keep in a ring buffer.

no_db_download: bool

If set, the crates-index database for additional metadata will not be downloaded.

It costs a lot of initial processing time and IO when writing changes back to the database, which isn’t helpful while on a slow disk - right now it does so unconditionally and doesn’t track that the work was already done.

io_bound_processors: u32

The amount of IO-bound processors to run concurrently.

A way to choose a value is to see which part of the I/O is actually the bottle neck. Depending on that number, one should experiment with an amount of processors that saturate either input or output. Most commonly, these are bound to the input, as it is the network.

cpu_o_bound_processors: u32

The amount of CPU- and Output-bound processors to run concurrently.

These will perform a computation followed by flushing its result to disk in the form of multiple small files. It’s recommended to adjust that number to whatever can saturate the speed of writing to disk, as these processors will yield when writing, allowing other processors to compute. Computes are relatively inexpensive compared to the writes.

cpu_bound_processors: u32

The amount of CPU-bound processors to run concurrently.

One can assume that one of these can occupy one core of a CPU. However, they will not use a lot of IO, nor will they use much memory.

repository: Option<PathBuf>

Path to the possibly existing crates.io repository clone. If unset, it will be cloned to a temporary spot.

time_limit: Option<Duration>

The amount of time we can take for the computation. Specified in humantime, like 10s, 5min, or 2h, or ‘3h 2min 2s’

fetch_every: Duration

The time between each fetch operation, specified in humantime, like 10s, 5min, or 2h, or ‘3h 2min 2s’

fetch_at_most: Option<usize>

If set, the amount of times the fetch stage will run. If set to 0, it will never run.

process_every: Duration

The time between each processing run, specified in humantime, like 10s, 5min, or 2h, or ‘3h 2min 2s’

process_at_most: Option<usize>

If set, the amount of times the process stage will run. If set to 0, they will never run.

report_every: Duration

The time between each reporting and processing run, specified in humantime, like 10s, 5min, or 2h, or ‘3h 2min 2s’

report_at_most: Option<usize>

If set, the amount of times the reporting stage will run. If set to 0, they will never run.

download_crates_io_database_every_24_hours_starting_at: Option<Time>

If set, declare at which local time to download the crates.io database and digest it.

This job runs every 24h, as the database is updated that often. If unset, the job starts right away. Format is HH:MM, e.g. ‘14:30’ for 2:30 pm or 03:15 for quarter past 3 in the morning.

glob: Option<String>

If set, the reporting stage will only iterate over crates that match the given standard unix glob.

moz* would match only crates starting with ‘moz’ for example.

db_path: PathBuf

Path to the possibly existing database. It’s used to persist all mining results.

Export

Export all Criner data into a format friendly for exploration via SQL, best viewed with https://sqlitebrowser.org

Criner stores binary blobs internally and migrates them on the fly, which is optimized for raw performance. It’s also impractical for exploring the data by hand, so the exported data will explode all types into tables with each column being a field. Foreign key relations are set accordingly to allow joins. Use this to get an overview of what’s available, and possibly contribute a report generator which implements a query using raw data and writes it into reports.

Fields

input_db_path: PathBuf

The path to the source database in sqlite format

export_db_path: PathBuf

Path to which to write the exported data. If it exists the operation will fail.

Trait Implementations

impl CommandFactory for SubCommands

fn command<'b>() -> Command

Build a Command that can instantiate Self.

fn command_for_update<'b>() -> Command

Build a Command that can update self.

impl Debug for SubCommands

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for SubCommands

fn default() -> Self

Returns the “default value” for a type.

impl FromArgMatches for SubCommands

fn from_arg_matches(__clap_arg_matches: &ArgMatches) -> Result<Self, Error>

Instantiate Self from ArgMatches, parsing the arguments as needed.

fn from_arg_matches_mut( __clap_arg_matches: &mut ArgMatches, ) -> Result<Self, Error>

Instantiate Self from ArgMatches, parsing the arguments as needed.

fn update_from_arg_matches( &mut self, __clap_arg_matches: &ArgMatches, ) -> Result<(), Error>

Assign values from ArgMatches to self.

fn update_from_arg_matches_mut<'b>( &mut self, __clap_arg_matches: &mut ArgMatches, ) -> Result<(), Error>

Assign values from ArgMatches to self.

impl Parser for SubCommands

fn parse() -> Self

Parse from std::env::args_os(), exit on error.

fn try_parse() -> Result<Self, Error>

Parse from std::env::args_os(), return Err on error.

fn parse_from<I, T>(itr: I) -> Self

where I: IntoIterator<Item = T>, T: Into<OsString> + Clone,

Parse from iterator, exit on error.

fn try_parse_from<I, T>(itr: I) -> Result<Self, Error>

where I: IntoIterator<Item = T>, T: Into<OsString> + Clone,

Parse from iterator, return Err on error.

fn update_from<I, T>(&mut self, itr: I)

where I: IntoIterator<Item = T>, T: Into<OsString> + Clone,

Update from iterator, exit on error.

fn try_update_from<I, T>(&mut self, itr: I) -> Result<(), Error>

where I: IntoIterator<Item = T>, T: Into<OsString> + Clone,

Update from iterator, return Err on error.

impl Subcommand for SubCommands

fn augment_subcommands<'b>(__clap_app: Command) -> Command

Append to Command so it can instantiate Self via FromArgMatches::from_arg_matches_mut

fn augment_subcommands_for_update<'b>(__clap_app: Command) -> Command

Append to Command so it can instantiate self via FromArgMatches::update_from_arg_matches_mut

fn has_subcommand(__clap_name: &str) -> bool

Test whether Self can parse a specific subcommand