Struct RcloneBackend 

pub struct RcloneBackend { /* private fields */ }

Rclone CLI storage backend.

Uses inline credentials via :backend,key=val:bucket/path syntax to avoid requiring global rclone config files.

The remote string is wrapped in SecretBox to prevent accidental logging of embedded credentials.

Commands are executed via a RemoteShell, defaulting to [LocalShell].

Timeout configuration

Resolution order: explicit (with_timeout) > env (VDSL_RCLONE_TIMEOUT) > default (300s). Batch operations scale the timeout by file count.

Implementations

impl RcloneBackend

pub fn new(remote: impl Into<String>) -> Self

Create a new RcloneBackend with the given remote string.

Timeout: env VDSL_RCLONE_TIMEOUT or default 300s. Uses [LocalShell] for command execution (backward compatible).

Example

let backend = RcloneBackend::new(":b2,account=key_id,key=secret:my-bucket");

pub fn with_shell( remote: impl Into<String>, shell: Box<dyn RemoteShell>, ) -> Self

Create with a custom RemoteShell (e.g. PodShell for GPU pod execution).

pub fn with_timeout(self, timeout_secs: u64) -> Self

Set an explicit timeout (seconds), overriding env and default.

Useful when constructing from parsed config values. Floor: 10 seconds.

Trait Implementations

impl StorageBackend for RcloneBackend

fn exists<'life0, 'life1, 'async_trait>( &'life0 self, remote_path: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<bool, InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Note: returns Ok(false) on any rclone error (including network failures). This is a best-effort check — callers must not rely on false meaning “confirmed absent”. Use push/pull for authoritative operations.

fn archive_move<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, src_remote_path: &'life1 str, archive_remote_path: &'life2 str, ) -> Pin<Box<dyn Future<Output = Result<(), InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Move a file to an archive location via rclone moveto.

Used for soft-delete on cold storage: the file is relocated (not copied) to an archive prefix, preserving content with a new path/revision. rclone moveto is atomic at the object-store level for B2/S3.

fn push_batch<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, src_root: &'life1 Path, dest_root: &'life2 str, relative_paths: &'life3 [String], ) -> Pin<Box<dyn Future<Output = HashMap<String, Result<(), InfraError>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Batch push using rclone copy --files-from.

For SFTP remotes, splits into chunks of [SFTP_BATCH_CHUNK_SIZE] files with per-chunk progress logging and retry. Non-SFTP backends run as a single batch (rclone handles large batches natively for B2/S3).

Returns per-file Ok/Err.

fn pull_batch<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, src_root: &'life1 str, dest_root: &'life2 Path, relative_paths: &'life3 [String], ) -> Pin<Box<dyn Future<Output = HashMap<String, Result<(), InfraError>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Batch pull using rclone copy --files-from.

For SFTP remotes, splits into chunks with progress logging and retry.

fn delete_batch<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, remote_root: &'life1 str, relative_paths: &'life2 [String], ) -> Pin<Box<dyn Future<Output = HashMap<String, Result<(), InfraError>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Batch delete using rclone delete --files-from.

For SFTP remotes, splits into chunks with progress logging and retry.

fn archive_move_batch<'life0, 'life1, 'life2, 'life3, 'async_trait>( &'life0 self, src_root: &'life1 str, archive_dest_root: &'life2 str, relative_paths: &'life3 [String], ) -> Pin<Box<dyn Future<Output = HashMap<String, Result<(), InfraError>>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait, 'life3: 'async_trait,

Batch archive-move using rclone move --files-from.

Moves files from src_root to archive_dest_root preserving relative paths. Uses the same chunked execution as other batch operations.

fn push<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, local_path: &'life1 Path, remote_path: &'life2 str, ) -> Pin<Box<dyn Future<Output = Result<(), InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Push a local file to this remote.

fn pull<'life0, 'life1, 'life2, 'async_trait>( &'life0 self, remote_path: &'life1 str, local_path: &'life2 Path, ) -> Pin<Box<dyn Future<Output = Result<(), InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait, 'life2: 'async_trait,

Pull a file from this remote to a local path.

fn list<'life0, 'life1, 'async_trait>( &'life0 self, remote_path: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<Vec<RemoteFile>, InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

List files at a remote path.

fn delete<'life0, 'life1, 'async_trait>( &'life0 self, remote_path: &'life1 str, ) -> Pin<Box<dyn Future<Output = Result<(), InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait, 'life1: 'async_trait,

Delete a file on this remote.

fn supports_batch(&self) -> bool

Whether this backend supports efficient batch push/pull.

fn backend_type(&self) -> &str

Backend type name for display and config matching.

fn set_progress_callback(&self, callback: Option<ProgressFn>)

Set a progress callback for batch operations.

fn ensure<'life0, 'async_trait>( &'life0 self, ) -> Pin<Box<dyn Future<Output = Result<(), InfraError>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

外部ツールの到達確認 + 確保。