Enum JobType 

pub enum JobType {
    EmailSync {
        identities: Option<Vec<String>>,
        local_output: Option<PathBuf>,
        remote_output: Option<String>,
        encryption_key: Option<String>,
        concurrency: Option<usize>,
        upload_concurrency: Option<usize>,
        max_connections_per_identity: Option<usize>,
        upload_only: bool,
        report_bucket: Option<String>,
        non_interactive: bool,
    },
    EmailPull {
        identities: Option<Vec<String>>,
        local_output: Option<PathBuf>,
        remote_output: Option<String>,
        concurrency: Option<usize>,
        upload_concurrency: Option<usize>,
        max_connections_per_identity: Option<usize>,
        upload_only: bool,
        report_bucket: Option<String>,
        non_interactive: bool,
    },
    Deduplicate {
        source_bucket: Vec<String>,
        local_output: Option<PathBuf>,
        destination_bucket: Option<String>,
        concurrency: Option<usize>,
        transfers: Option<usize>,
        checkers: Option<usize>,
        tpslimit: Option<usize>,
        upload_only: bool,
        report_bucket: Option<String>,
        non_interactive: bool,
    },
    Rclone(RcloneArgs),
    Transform {

        input_file_type: Option<String>,
        video_quality: Option<String>,
        source: Option<String>,
        destination: Option<String>,
        local_output: Option<PathBuf>,
        concurrency: Option<usize>,
        download_transfers: Option<usize>,
        download_checkers: Option<usize>,
        upload_transfers: Option<usize>,
        max_pending_downloads: Option<usize>,
        tpslimit: Option<usize>,
        report_bucket: Option<String>,
        non_interactive: bool,
    
},
}

Variants

EmailSync

Fetch, transform, deduplicate, and optionally upload mail for one or more authenticated email identities. Replaces pigeon email sync (ADR-0021).

Fields

identities: Option<Vec<String>>

Aliases of the identities to sync, comma-separated. Interactively selected from the authenticated identities when omitted and stdin is a terminal; required otherwise.

local_output: Option<PathBuf>

Local directory to stage and store output under, shared across every selected identity (each gets its own subdirectory underneath). Defaults to a directory under the OS temp directory when omitted.

remote_output: Option<String>

Alias of a configured bucket-config (see pigeon dataops bucket-config new) to upload each identity’s local result tree to, once its local fetch/transform/dedupe phase is complete.

encryption_key: Option<String>

Alias of a configured encryption key (see pigeon keyring add encryption-key) to use, overriding the target bucket-config’s own default (if any). Interactively selected/confirmed when omitted; falls back to the bucket’s default non-interactively (ADR-0027).

concurrency: Option<usize>

Maximum number of fetch/transform workers to run concurrently, spanning every selected identity’s every mailbox. Interactively prompted (with a rough time estimate) when omitted and stdin is a terminal; required otherwise.

upload_concurrency: Option<usize>

Maximum number of files to upload concurrently, independent of --concurrency (which sizes IMAP fetch/transform work) – the upload phase is network-round-trip-bound, not IMAP-bound, so it benefits from its own, separately-tuned concurrency (ADR-0091). Interactively prompted when omitted and stdin is a terminal; defaults to 16 otherwise. With --upload-only, this is the only concurrency flag that has any effect.

max_connections_per_identity: Option<usize>

Maximum number of simultaneous IMAP connections opened to any one identity, regardless of --concurrency (ADR-0071) – caps worker concurrency per-account rather than only globally, so a mailbox with enough pending batches can’t cause more than this many workers to log in to the same account at once and trip a provider’s simultaneous-connection limit. Defaults to 6 (well under Gmail’s documented 15-connection cap) when omitted; not interactively prompted.

upload_only: bool

Resumes uploading already-completed local runs for the selected identities instead of starting new ones: skips the IMAP connect/fetch/transform/dedup phases entirely (and the per-identity IMAP credentials they’d otherwise need) and uploads straight from each identity’s existing local result tree, picking up where a prior run’s upload phase left off via the same .staging/.uploaded index per identity (ADR-0090). An identity with no completed local run is skipped with a warning rather than failing the whole command. Requires a mandatory --remote-output.

report_bucket: Option<String>

Alias of a configured bucket-config this run’s report, the shared observability log, and a transcript of its printed output are uploaded to, always unencrypted, under a YYYY-MM-DD-job-name-{run-id}/ prefix (ADR-0100). Mandatory – interactively selected when omitted and stdin is a terminal; required otherwise.

non_interactive: bool

Suppress every wizard prompt (ADR-0114): each omitted input resolves via its non-interactive fallback (or errors if none exists) instead of asking, and the final “proceed?” confirmation is auto-answered yes.

EmailPull

Fetches raw .eml files and unpacked attachments (no Markdown/ frontmatter transform) for one or more authenticated email identities, deduplicating attachments by content, and optionally uploads the result unencrypted to a bucket-config (ADR-0081).

Fields

identities: Option<Vec<String>>

Aliases of the identities to pull, comma-separated. Interactively selected from the authenticated identities when omitted and stdin is a terminal; required otherwise.

local_output: Option<PathBuf>

Local directory to stage and store output under, shared across every selected identity. Defaults to a directory under the OS temp directory when omitted.

remote_output: Option<String>

Alias of a configured bucket-config to upload each identity’s local result tree to, once its local fetch/dedupe phase is complete. Always uploaded unencrypted – this job never offers encryption (ADR-0081).

concurrency: Option<usize>

Maximum number of fetch/extract workers to run concurrently, spanning every selected identity’s every mailbox. Interactively prompted (with a rough time estimate) when omitted and stdin is a terminal; required otherwise.

upload_concurrency: Option<usize>

Maximum number of files to upload concurrently, independent of --concurrency (which sizes IMAP fetch/extract work) – the upload phase is network-round-trip-bound, not IMAP-bound, so it benefits from its own, separately-tuned concurrency (ADR-0091). Interactively prompted when omitted and stdin is a terminal; defaults to 16 otherwise. With --upload-only, this is the only concurrency flag that has any effect.

max_connections_per_identity: Option<usize>

Maximum number of simultaneous IMAP connections opened to any one identity, regardless of --concurrency. Defaults to 6 when omitted; not interactively prompted.

upload_only: bool

Resumes uploading already-completed local runs for the selected identities instead of starting new ones: skips the IMAP connect/fetch/dedup phases entirely (and the per-identity IMAP credentials they’d otherwise need) and uploads straight from each identity’s existing local result tree, picking up where a prior run’s upload phase left off via the same .staging/.uploaded index per identity (ADR-0090). An identity with no completed local run is skipped with a warning rather than failing the whole command. Requires a mandatory --remote-output.

report_bucket: Option<String>

Alias of a configured bucket-config this run’s report, the shared observability log, and a transcript of its printed output are uploaded to, always unencrypted, under a YYYY-MM-DD-job-name-{run-id}/ prefix (ADR-0100). Mandatory – interactively selected when omitted and stdin is a terminal; required otherwise.

non_interactive: bool

Suppress every wizard prompt (ADR-0114): each omitted input resolves via its non-interactive fallback (or errors if none exists) instead of asking, and the final “proceed?” confirmation is auto-answered yes.

Deduplicate

Recursively scans a bucket, always inflates every zip found (the zip container itself is never uploaded, only its inflated contents), content-hashes every file bucket-wide to keep one byte-identical copy of each, writes a human-readable merge report, and optionally uploads the result unencrypted to a (possibly different) bucket-config (ADR-0082). Every file is always processed and every zip is always expanded – there is no file-type or zip-expansion selection, and this job never offers encryption.

Fields

source_bucket: Vec<String>

Alias of a configured bucket-config to pull from. Repeat the flag for more than one (--source-bucket a --source-bucket b) – every listed bucket is downloaded into one shared local staging tree and deduplicated across the combined set, not per-bucket (ADR-0109). Interactively multi-selected from the configured bucket-configs when omitted and stdin is a terminal; at least one is required otherwise.

local_output: Option<PathBuf>

Local directory to stage and store output under. Defaults to a directory under the OS temp directory when omitted.

destination_bucket: Option<String>

Alias of a configured bucket-config to upload the deduped result to, once local processing is complete. Always uploaded unencrypted. Single bucket only (ADR-0109).

concurrency: Option<usize>

Maximum number of files to download/hash concurrently.

transfers: Option<usize>

rclone –transfers for the upload phase, now one bulk rclone copy rather than a per-file task queue (ADR-0136). Defaults to 8 when omitted. With --upload-only, this is the only transfer concurrency flag that has any effect.

checkers: Option<usize>

rclone –checkers for the upload phase. Defaults to 16 when omitted.

tpslimit: Option<usize>

rclone –tpslimit for the upload phase. Unset by default – no rate cap.

upload_only: bool

Resumes uploading an already-completed local deduplicate run instead of starting a new one: skips the bucket listing/download/hash/ placement phases entirely (and the source bucket credentials they’d otherwise need) and uploads straight from an existing --local-output’s result/ tree, picking up where a prior run’s upload phase left off via the same .staging/.uploaded index (ADR-0089). Requires a --local-output from a completed prior run (its .staging/.processed checkpoint must exist and its result/ must be non-empty) and a mandatory --remote-output.

report_bucket: Option<String>

Alias of a configured bucket-config this run’s report, the shared observability log, and a transcript of its printed output are uploaded to, always unencrypted, under a YYYY-MM-DD-job-name-{run-id}/ prefix (ADR-0100). Mandatory – interactively selected when omitted and stdin is a terminal; required otherwise.

non_interactive: bool

Suppress every wizard prompt (ADR-0114): each omitted input resolves via its non-interactive fallback (or errors if none exists) instead of asking, and the final “proceed?” confirmation is auto-answered yes.

Rclone(RcloneArgs)

Shells out to the external rclone binary for copy/delete operations against pigeon keyring BucketConfig aliases. rclone is always invoked against a config file pigeon itself generates from the keyring (ADR-0136, reversing ADR-0101/ADR-0110’s original raw-passthrough design) – never an externally-provisioned or user-general-purpose rclone.conf. See subcommands for the available actions. Requires rclone on PATH.

Transform

Transcodes image files (png/jpeg/heic) into full-size, maximum-quality .jpg, or video files (mov/m4v/mp4, ADR-0122) into space-optimized H.265/HEVC .mp4 at a chosen --video-quality (ADR-0112, pipeline reworked by ADR-0116): shells out to rclone to pull --source into a local staging tree, transcoding/copying each file locally and pushing it individually via rclone copyto the moment it’s ready, rather than waiting for the whole batch. --source/--destination are pigeon keyring BucketConfig aliases, optionally composed with a prefix within that bucket (alias:prefix/, ADR-0138); an omitted prefix defaults to --input-file-type’s own extension (source) or output extension (destination). Deduplication is never involved – every output file’s name is derived deterministically and is unique by construction; duplicate source content is deliberately retained at separate destination names. A single file’s transcode or push failure (e.g. a HEIC file the local ffmpeg build can’t decode, retried before giving up) is recorded and reported but never stops any other file from being processed and pushed; every file that does succeed stays checkpointed for a resumed rerun. Requires both rclone and ffmpeg/ffprobe on PATH.

Fields

input_file_type: Option<String>

Which input format this run transcodes: png, jpeg, heic, mov, m4v, or mp4. Interactively selected when omitted and stdin is a terminal; required otherwise.

video_quality: Option<String>

Compression level for a video input kind (mov/m4v/mp4): low, medium (default), high, or lossless – a small vetted libx265 CRF menu (ADR-0122). Ignored for an image input kind. Interactively selected when omitted, stdin is a terminal, and the input kind is video; defaults to medium otherwise.

source: Option<String>

Where to pull from: a configured bucket-config alias, optionally composed with a prefix inside it – media is the whole bucket, media:mp4/ its mp4/ directory (ADR-0138). An omitted prefix defaults to --input-file-type’s own extension, so --source media --input-file-type mp4 means media:mp4/; an explicit prefix is always used verbatim. Interactively selected (bucket root, then the default prefix) when omitted and stdin is a terminal; required otherwise.

destination: Option<String>

Where to push to, same alias[:prefix/] form as --source (ADR-0138). An omitted prefix defaults to --input-file-type’s output extension (jpg for an image kind, mp4 for a video one), so one bucket can serve as both source and destination when the prefixes differ.

local_output: Option<PathBuf>

Local directory this run stages pulled source files, transcoded results, and bookkeeping under. Defaults to a directory under the OS temp directory when omitted.

concurrency: Option<usize>

Maximum number of files to transcode concurrently (the ffmpeg phase only – rclone manages its own internal parallelism via --download-transfers/--download-checkers). Interactively prompted (a cores-based default) when omitted and stdin is a terminal; required otherwise.

download_transfers: Option<usize>

rclone –transfers for the download (pull) phase. Defaults to 8 when omitted.

download_checkers: Option<usize>

rclone –checkers for the download (pull) phase. Defaults to 16 when omitted.

upload_transfers: Option<usize>

Maximum number of concurrent per-file push (rclone copyto) operations. Defaults to 8 when omitted. Separate from --download-transfers to allow independent tuning of pull vs. push phases (ADR-0129).

max_pending_downloads: Option<usize>

Maximum number of downloaded but not-yet-processed files (pending transcode/push) before the pull phase is paused to allow the pipeline to catch up. Defaults to concurrency * 4 when omitted. Enables automatic disk-pressure backpressure without requiring manual tuning (ADR-0129).

tpslimit: Option<usize>

rclone –tpslimit, used by the pull phase only. Unset by default – no rate cap.

report_bucket: Option<String>

Alias of a configured bucket-config this run’s report, the shared observability log, and a transcript of its printed output are uploaded to, always unencrypted, under a YYYY-MM-DD-job-name-{run-id}/ prefix (ADR-0100). Mandatory – interactively selected when omitted and stdin is a terminal; required otherwise.

non_interactive: bool

Suppress every wizard prompt (ADR-0114): each omitted input resolves via its non-interactive fallback (or errors if none exists) instead of asking, and the final “proceed?” confirmation is auto-answered yes.

Trait Implementations

impl Debug for JobType

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl FromArgMatches for JobType

fn from_arg_matches(__clap_arg_matches: &ArgMatches) -> Result<Self, Error>

Instantiate Self from ArgMatches, parsing the arguments as needed.

fn from_arg_matches_mut( __clap_arg_matches: &mut ArgMatches, ) -> Result<Self, Error>

Instantiate Self from ArgMatches, parsing the arguments as needed.

fn update_from_arg_matches( &mut self, __clap_arg_matches: &ArgMatches, ) -> Result<(), Error>

Assign values from ArgMatches to self.

fn update_from_arg_matches_mut<'b>( &mut self, __clap_arg_matches: &mut ArgMatches, ) -> Result<(), Error>

Assign values from ArgMatches to self.

impl Subcommand for JobType

fn augment_subcommands<'b>(__clap_app: Command) -> Command

Append to Command so it can instantiate Self via FromArgMatches::from_arg_matches_mut

fn augment_subcommands_for_update<'b>(__clap_app: Command) -> Command

Append to Command so it can instantiate self via FromArgMatches::update_from_arg_matches_mut

fn has_subcommand(__clap_name: &str) -> bool

Test whether Self can parse a specific subcommand