pub enum JobType {
Email(EmailArgs),
Deduplicate {
source_bucket: Vec<String>,
local_output: Option<PathBuf>,
destination_bucket: Option<String>,
concurrency: Option<usize>,
transfers: Option<usize>,
checkers: Option<usize>,
tpslimit: Option<usize>,
upload_only: bool,
report_bucket: Option<String>,
non_interactive: bool,
},
Rclone(RcloneArgs),
Transform {Show 14 fields
input_file_type: Option<String>,
video_quality: Option<String>,
audio_quality: Option<String>,
source: Option<String>,
destination: Option<String>,
local_output: Option<PathBuf>,
concurrency: Option<usize>,
download_transfers: Option<usize>,
download_checkers: Option<usize>,
upload_transfers: Option<usize>,
max_pending_downloads: Option<usize>,
tpslimit: Option<usize>,
report_bucket: Option<String>,
non_interactive: bool,
},
}Variants§
Email(EmailArgs)
Email jobs addressed through the keyring. See subcommands for the available actions.
Deduplicate
Recursively scans a bucket, always inflates every zip found (the zip container itself is never uploaded, only its inflated contents), content-hashes every file bucket-wide to keep one byte-identical copy of each, writes a human-readable merge report, and optionally uploads the result unencrypted to a (possibly different) bucket-config (ADR-0082). Every file is always processed and every zip is always expanded – there is no file-type or zip-expansion selection, and this job never offers encryption.
Fields
source_bucket: Vec<String>Alias of a configured bucket-config to pull from. Repeat the
flag for more than one (--source-bucket a --source-bucket b)
– every listed bucket is downloaded into one shared local
staging tree and deduplicated across the combined set, not
per-bucket (ADR-0109). Interactively multi-selected from the
configured bucket-configs when omitted and stdin is a terminal;
at least one is required otherwise.
local_output: Option<PathBuf>Local directory to stage and store output under. Defaults to a directory under the OS temp directory when omitted.
destination_bucket: Option<String>Alias of a configured bucket-config to upload the deduped result to, once local processing is complete. Always uploaded unencrypted. Single bucket only (ADR-0109).
transfers: Option<usize>rclone –transfers for the upload phase, now one bulk rclone copy rather than a per-file task queue (ADR-0136). Defaults to
8 when omitted. With --upload-only, this is the only transfer
concurrency flag that has any effect.
upload_only: boolResumes uploading an already-completed local deduplicate run instead
of starting a new one: skips the bucket listing/download/hash/
placement phases entirely (and the source bucket credentials
they’d otherwise need) and uploads straight from an existing
--local-output’s result/ tree, picking up where a prior
run’s upload phase left off via the same .staging/.uploaded
index (ADR-0089). Requires a --local-output from a completed
prior run (its .staging/.processed checkpoint must exist and
its result/ must be non-empty) and a mandatory
--remote-output.
report_bucket: Option<String>Alias of a configured bucket-config this run’s report, the
shared observability log, and a transcript of its printed
output are uploaded to, always unencrypted, under a
YYYY-MM-DD-job-name-{run-id}/ prefix (ADR-0100). Mandatory –
interactively selected when omitted and stdin is a terminal;
required otherwise.
Rclone(RcloneArgs)
Shells out to the external rclone binary for copy/delete
operations against pigeon keyring BucketConfig aliases. rclone
is always invoked against a config file pigeon itself generates
from the keyring (ADR-0136, reversing ADR-0101/ADR-0110’s original
raw-passthrough design) – never an externally-provisioned or
user-general-purpose rclone.conf. See subcommands for the
available actions. Requires rclone on PATH.
Transform
Transcodes image files (png/jpeg/heic) into full-size,
maximum-quality .jpg, video files (mov/m4v/mp4, ADR-0122)
into space-optimized H.265/HEVC .mp4 at a chosen --video-quality,
audio files – an audio-only recording carrying an .mp4 extension
(mp4-audio, ADR-0142) or m4a/mp3/aac/wav/opus
(ADR-0153) – into Opus .opus at a chosen --audio-quality, or
spreadsheets
(numbers/xlsx/xls/csv, ADR-0149) into one .csv per sheet,
or documents (ADR-0154) – doc/docx/pages/rtf/pptx/key/
epub into .pdf with pdf copied through,
txt/md/markdown/mkd/rst/textclipping into .txt, and
tsv into .csv
(ADR-0112, pipeline reworked by ADR-0116): shells out to rclone to
pull --source into a local staging tree, transcoding/copying each
file locally and pushing it individually via rclone copyto the
moment it’s ready, rather than waiting for the whole batch. A
multi-sheet workbook is the one input kind that yields several
outputs, each named {stem}-{hash}-{nn}of{tt}-{sheet}.csv so they
stay recognizably related and a half-pushed workbook is detectable
on a rerun (ADR-0149).
--source/--destination are pigeon keyring BucketConfig
aliases, optionally composed with a prefix within that bucket
(alias:prefix/, ADR-0138); an omitted prefix defaults to
--input-file-type’s own extension (source) or output extension
(destination). Deduplication is never involved – every output file’s
name is derived deterministically and is unique by construction;
duplicate source content is deliberately retained at separate
destination names. A single file’s transcode or push failure (e.g.
a HEIC file the local ffmpeg build can’t decode, retried before
giving up) is recorded and reported but never stops any other file
from being processed and pushed. A rerun is resumable without any
local state: “already done” is answered by listing the destination
(ADR-0120 removed the local checkpoint). Requires rclone on
PATH, plus ffmpeg/ffprobe for an image, video, or audio input
kind, soffice (LibreOffice) for numbers and for
doc/docx/pages/rtf/pptx/key, ebook-convert (Calibre)
for epub, and pandoc for md/rst;
xlsx/xls/csv/tsv/pdf/txt/textclipping need none of
them (ADR-0149, ADR-0154).
Fields
input_file_type: Option<String>Which input format this run transcodes: png, jpeg, heic,
mov, m4v, mp4, mp4-audio, m4a, mp3, aac, wav,
opus, numbers, xlsx, xls, csv, doc, docx,
pages, rtf, pdf, epub, pptx, key, txt, md
(or markdown/mkd), rst, textclipping, or tsv.
Interactively selected when omitted and stdin is a terminal;
required otherwise.
mp4-audio is for audio-only recordings carrying an .mp4
extension (voice notes), transcoded to .opus; a file that
actually has a video stream is refused rather than silently
losing its video – embedded cover art does not count as video
(ADR-0153). opus input is re-encoded rather than copied
through, so a mixed library normalizes to one bitrate. The four spreadsheet kinds all output .csv,
one file per non-empty sheet (ADR-0149); csv input is copied
through byte-for-byte, as are pdf and txt (ADR-0154).
key is sniffed before conversion and a file that is not an
iWork container is refused – that extension is also
conventional for a private key, and LibreOffice would otherwise
render one into a readable PDF. md/rst are rendered to
prose, which discards link URLs and all inline markup.
video_quality: Option<String>Compression level for a true video input kind (mov/m4v/mp4
– not mp4-audio, which has no video stream):
low, medium (default), high, or lossless – a small
vetted libx265 CRF menu (ADR-0122). Ignored for every other
input kind. Interactively selected when omitted, stdin is a
terminal, and the input kind is video; defaults to medium
otherwise.
audio_quality: Option<String>Opus bitrate for an audio input kind (mp4-audio, m4a,
mp3, aac, wav, opus): low (32k, speech), medium
(48k), high (96k), or transparent (128k) – a small vetted
menu (ADR-0153), never a raw bitrate. Ignored for every other
input kind. Interactively selected when omitted, stdin is a
terminal, and the input kind is audio; otherwise it defaults
per kind: medium for mp4-audio, preserving ADR-0142’s
choice for voice notes, and transparent for the rest, which
are music or (for wav) lossless and would be badly served by
a speech bitrate.
source: Option<String>Where to pull from: a configured bucket-config alias, optionally
composed with a prefix inside it – media is the whole bucket,
media:mp4/ its mp4/ directory (ADR-0138). An omitted prefix
defaults to --input-file-type’s own extension, so
--source media --input-file-type mp4 means media:mp4/; an
explicit prefix is always used verbatim. Interactively selected
(bucket root, then the default prefix) when omitted and stdin is
a terminal; required otherwise.
destination: Option<String>Where to push to, same alias[:prefix/] form as --source
(ADR-0138). An omitted prefix defaults to --input-file-type’s
output extension (jpg for an image kind, mp4 for a video
one, opus for every audio kind, csv for every spreadsheet
kind),
so one bucket can serve as both source and destination when the
prefixes differ.
local_output: Option<PathBuf>Local directory this run stages pulled source files, transcoded results, and bookkeeping under. Defaults to a directory under the OS temp directory when omitted.
concurrency: Option<usize>Maximum number of files to convert concurrently (the local
conversion phase only – rclone manages its own parallelism via
--download-transfers/--download-checkers). Interactively
prompted (a cores-based default) when omitted and stdin is a
terminal; required otherwise.
download_transfers: Option<usize>rclone –transfers for the download (pull) phase. Defaults to 8 when omitted.
download_checkers: Option<usize>rclone –checkers for the download (pull) phase. Defaults to 16 when omitted.
upload_transfers: Option<usize>Maximum number of concurrent per-file push (rclone copyto)
operations. Defaults to 8 when omitted. Separate from
--download-transfers to allow independent tuning of pull vs.
push phases (ADR-0129).
max_pending_downloads: Option<usize>Maximum number of downloaded but not-yet-processed files
(pending transcode/push) before the pull phase is paused to allow
the pipeline to catch up. Defaults to concurrency * 4 when
omitted. Enables automatic disk-pressure backpressure without
requiring manual tuning (ADR-0129).
tpslimit: Option<usize>rclone –tpslimit, used by the pull phase only. Unset by default – no rate cap.
report_bucket: Option<String>Alias of a configured bucket-config this run’s report, the
shared observability log, and a transcript of its printed
output are uploaded to, always unencrypted, under a
YYYY-MM-DD-job-name-{run-id}/ prefix (ADR-0100). Mandatory –
interactively selected when omitted and stdin is a terminal;
required otherwise.
Trait Implementations§
Source§impl FromArgMatches for JobType
impl FromArgMatches for JobType
Source§fn from_arg_matches(__clap_arg_matches: &ArgMatches) -> Result<Self, Error>
fn from_arg_matches(__clap_arg_matches: &ArgMatches) -> Result<Self, Error>
Source§fn from_arg_matches_mut(
__clap_arg_matches: &mut ArgMatches,
) -> Result<Self, Error>
fn from_arg_matches_mut( __clap_arg_matches: &mut ArgMatches, ) -> Result<Self, Error>
Source§fn update_from_arg_matches(
&mut self,
__clap_arg_matches: &ArgMatches,
) -> Result<(), Error>
fn update_from_arg_matches( &mut self, __clap_arg_matches: &ArgMatches, ) -> Result<(), Error>
ArgMatches to self.Source§fn update_from_arg_matches_mut<'b>(
&mut self,
__clap_arg_matches: &mut ArgMatches,
) -> Result<(), Error>
fn update_from_arg_matches_mut<'b>( &mut self, __clap_arg_matches: &mut ArgMatches, ) -> Result<(), Error>
ArgMatches to self.Source§impl Subcommand for JobType
impl Subcommand for JobType
Source§fn augment_subcommands<'b>(__clap_app: Command) -> Command
fn augment_subcommands<'b>(__clap_app: Command) -> Command
Source§fn augment_subcommands_for_update<'b>(__clap_app: Command) -> Command
fn augment_subcommands_for_update<'b>(__clap_app: Command) -> Command
Command so it can instantiate self via
FromArgMatches::update_from_arg_matches_mut Read moreSource§fn has_subcommand(__clap_name: &str) -> bool
fn has_subcommand(__clap_name: &str) -> bool
Self can parse a specific subcommandAuto Trait Implementations§
impl Freeze for JobType
impl RefUnwindSafe for JobType
impl Send for JobType
impl Sync for JobType
impl Unpin for JobType
impl UnsafeUnpin for JobType
impl UnwindSafe for JobType
Blanket Implementations§
Source§impl<T> BorrowMut<T> for Twhere
T: ?Sized,
impl<T> BorrowMut<T> for Twhere
T: ?Sized,
Source§fn borrow_mut(&mut self) -> &mut T
fn borrow_mut(&mut self) -> &mut T
impl<ST, DT> CastableFrom<ST, Initialized, Initialized> for DT
impl<ST, DT> CastableFrom<ST, Uninit, Uninit> for DT
Source§impl<T> Instrument for T
impl<T> Instrument for T
Source§fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
fn instrument(self, span: Span) -> Instrumented<Self> ⓘ
Source§fn in_current_span(self) -> Instrumented<Self> ⓘ
fn in_current_span(self) -> Instrumented<Self> ⓘ
Source§impl<T> IntoEither for T
impl<T> IntoEither for T
Source§fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
fn into_either(self, into_left: bool) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left is true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read moreSource§fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
fn into_either_with<F>(self, into_left: F) -> Either<Self, Self> ⓘ
self into a Left variant of Either<Self, Self>
if into_left(&self) returns true.
Converts self into a Right variant of Either<Self, Self>
otherwise. Read more