rusty-cat

rusty-cat is an async Rust SDK for resumable file upload and download. It gives applications a compact public facade for building transfer tasks, running those tasks in a background scheduler, receiving progress callbacks, and plugging in protocol-specific implementations such as plain HTTP, Aliyun OSS, Aliyun OSS presigned URLs, Azure Blob Storage, and Azure Blob SAS URLs.

The crate is designed for applications that need reliable large-file transfer without forcing a specific storage backend or database layer. The SDK handles scheduling, chunk dispatch, retry, pause/resume/cancel commands, and progress fan-out. Your application remains responsible for business records, credential management, user permissions, and provider-specific setup.

The recommended public import is:

use rusty_cat::api::*;

Existing module paths still work, but rusty_cat::api::* is the stable, beginner-friendly entry point. Using the facade also makes future refactoring easier because most application code can import SDK types from a single module.

Package, platform, Rust, and license

Badge Markdown

The Crates.io, Docs.rs, and License badge Markdown is shown below. These badges are safe to paste into downstream README files or generated documentation pages:

[![Crates.io](https://img.shields.io/crates/v/rusty-cat.svg)](https://crates.io/crates/rusty-cat)
[![Docs.rs](https://docs.rs/rusty-cat/badge.svg)](https://docs.rs/rusty-cat)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Feature highlights

Capability matrix

Architecture overview

Quick start

Add the crate:

[dependencies]
rusty-cat = "0.2.4"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

For OSS providers, enable only what you need. Keeping the feature list small reduces optional dependencies and makes it clearer which cloud integrations your application actually uses:

[dependencies]
rusty-cat = { version = "0.2.4", features = ["aliyun-oss-direct"] }

The aliyun-oss, azure-blob, and oss-providers umbrellas intentionally select the presigned/SAS flows, because those are the recommended model for untrusted clients (your backend holds the credentials). When you need the direct AccessKey/Shared Key flows, enable aliyun-oss-direct and/or azure-blob-direct explicitly.

For a focused comparison of direct credentials versus presigned/SAS URLs, see Provider feature flags: direct vs presigned/SAS.

Complete end-to-end example

This example starts from MeowConfig, creates a MeowClient, registers listeners, builds a task, submits it, waits for the completion/failure signal, inspects a snapshot, and closes the client. It uses an HTTP range download task because that path works without cloud credentials; the same client lifecycle applies to upload tasks and OSS/Azure provider tasks.

use std::sync::{mpsc, Arc, Mutex};
use std::time::Duration;

use rusty_cat::api::{
	DownloadPounceBuilder, FileTransferRecord, Log, MeowClient, MeowConfig, TransferStatus,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
	let config = MeowConfig::builder()
		.max_upload_concurrency(2)
		.max_download_concurrency(2)
		.http_timeout(Duration::from_secs(30))
		.tcp_keepalive(Duration::from_secs(60))
		.command_queue_capacity(256)
		.worker_event_queue_capacity(1024)
		.build()?;

	let client = MeowClient::new(config);

	client.set_debug_log_listener(Some(Arc::new(|log: Log| {
		println!("[rusty-cat] {log}");
	})))?;

	let global_listener_id = client.register_global_progress_listener(|record| {
		println!(
			"global: task={} file={} progress={:.2}% status={:?}",
			record.task_id(),
			record.file_name(),
			record.progress() * 100.0,
			record.status(),
		);
	})?;

	let task = DownloadPounceBuilder::new(
		"example.bin",
		"./downloads/example.bin",
		1024 * 1024,
		"https://example.com/example.bin",
	)
	.with_client_file_sign("business-file-id-001")
	.with_max_chunk_retries(3)
	.build();

	let (tx, rx) = mpsc::channel::<Result<(), String>>();
	let tx = Arc::new(Mutex::new(Some(tx)));
	let progress_tx = Arc::clone(&tx);
	let complete_tx = Arc::clone(&tx);

	let task_id = client
		.try_enqueue(
			task,
			move |record: FileTransferRecord| {
				println!(
					"task={} progress={:.2}% status={:?}",
					record.task_id(),
					record.progress() * 100.0,
					record.status(),
				);

				match record.status() {
					TransferStatus::Failed(err) => {
						if let Ok(mut guard) = progress_tx.lock() {
							if let Some(tx) = guard.take() {
								let _ = tx.send(Err(format!("transfer failed: {err}")));
							}
						}
					}
					TransferStatus::Canceled => {
						if let Ok(mut guard) = progress_tx.lock() {
							if let Some(tx) = guard.take() {
								let _ = tx.send(Err("transfer canceled".to_string()));
							}
						}
					}
					_ => {}
				}
			},
			move |task_id, provider_payload| {
				println!("task {task_id} complete; payload={provider_payload:?}");
				if let Ok(mut guard) = complete_tx.lock() {
					if let Some(tx) = guard.take() {
						let _ = tx.send(Ok(()));
					}
				}
			},
		)
		.await?;

	println!("enqueued task: {task_id}");

	match rx.recv_timeout(Duration::from_secs(300)) {
		Ok(Ok(())) => println!("transfer finished"),
		Ok(Err(err)) => eprintln!("transfer ended with error: {err}"),
		Err(err) => eprintln!("timed out while waiting for completion callback: {err}"),
	}

	let snapshot = client.snapshot().await?;
	println!(
		"snapshot: queued={}, active={}",
		snapshot.queued_groups, snapshot.active_groups
	);

	client.unregister_global_progress_listener(global_listener_id)?;
	client.clear_global_listener()?;
	client.set_debug_log_listener(None)?;
	client.close().await?;
	assert!(client.is_closed());
	Ok(())
}

MeowClient API guide

There is no public enqueue(...) method in the current API. Use try_enqueue(...); the name is intentional because enqueue uses fail-fast backpressure. If your application submits many tasks at once, increase command_queue_capacity or retry CommandSendFailed with your own backoff policy.

Configuration parameters

MeowConfig and MeowConfigBuilder

Start with MeowConfig::default() for a safe baseline or use MeowConfig::builder() for validated customization. The configuration is immutable after the client is created, which prevents accidental runtime changes from affecting tasks already in the scheduler.

UploadPounceBuilder

Beginner tips:

• Use a chunk_size between 1 MiB and 8 MiB for common object storage workloads unless your provider requires a different size. Very small chunks increase request overhead; very large chunks reduce retry granularity.
• Put provider protocol objects in Arc and pass them to with_breakpoint_upload(...) because the executor can move transfer work across async tasks.
• For restart recovery, persist enough business metadata in your own database to rebuild the same logical task later, including local path, remote URL/object key, direction, chunk size, and provider type.

When you do not attach a provider plugin, uploads use the built-in default protocol. Its exact request/response format — and how the fileMd5 signature is derived — is documented in Default HTTP upload protocol contract.

DownloadPounceBuilder

Download HTTP methods are intentionally not configurable. Resumable HTTP download depends on standard HEAD and GET range behavior. If a gateway or provider needs a non-standard method, implement BreakpointDownload and inject it with with_breakpoint_download(...).

Error handling and retries

Most SDK calls return Result<_, MeowError>. MeowError::code() is a stable numeric code (an i32, suitable for FFI or structured logging) and MeowError::msg() is a human-readable message. Branch on code(), never on the message text. The numeric codes come from InnerErrorCode.

The codes you will most often branch on:

See the Error codes reference for the complete list (codes 101–122), with the meaning and suggested handling of every variant.

Retry and transient errors

Two builder knobs control how many times a failed step is retried before the task fails:

• with_max_chunk_retries(n) — extra attempts after the first failed chunk transfer (default 3; 0 disables chunk retry).
• UploadPounceBuilder::with_max_upload_prepare_retries(n) — extra attempts after the first failed upload prepare (default 3).

Within those budgets the SDK only retries transient transport failures (connection reset, timeout, incomplete message), and it waits between attempts using exponential backoff with jitter, so a flaky network or a briefly overloaded server does not trigger a retry storm. Non-transient errors (an HTTP 403, a malformed response, an invalid range) fail fast without consuming the retry budget, because retrying them would not help.

To continue a task that ultimately failed (or that you paused), rebuild it and call try_enqueue again, or resume(...) a paused one; both continue from the last checkpoint. See Resuming uploads and downloads after a restart.

OSS upload/download developer guides

OSS and Blob workflows are provider-specific, so detailed beginner guides live in separate documents. The SDK does not persist your keys, secrets, account keys, tokens, presigned URLs, or SAS URLs in a built-in database or credential store. Some values are held in memory while executing tasks. You must provide them from your application or trusted backend, and you should avoid logging them in progress callbacks or debug listeners.

If you are deciding which provider feature to enable first, start with Provider feature flags: direct vs presigned/SAS.

Persistence and custom database integration

rusty-cat intentionally has no built-in database. This keeps the SDK small and lets you choose SQLite, PostgreSQL, Redis, a mobile database, or an existing business persistence layer. The SDK emits progress records and terminal states; your application decides how those records map to durable business state.

Recommended pattern:

1. Create your own transfer table with fields such as business file ID, local path, remote URL/object key, direction, chunk size, provider, status, progress, and credential reference.
2. Register per-task and/or global progress callbacks.
3. In callbacks, persist FileTransferRecord values or forward them to a persistence worker. Do not perform slow database writes directly on the callback path; prefer batching or sending records to your own worker queue.
4. On process restart, query unfinished rows and rebuild equivalent PounceTask values.
5. Call try_enqueue(...) again. Provider protocols can resume from local/remote checkpoint information when the same logical task is recreated correctly.

Never persist raw cloud secrets unless your security model explicitly allows it. Prefer storing a reference to a backend-owned credential or generating fresh short-lived presigned/SAS URLs.

New to restart recovery? For a step-by-step, beginner-friendly walkthrough of resuming after the process is killed or crashes — covering exactly what to persist for downloads, default HTTP uploads, and presigned multipart uploads, with copy-pasteable code — see Resuming uploads and downloads after a process restart or crash.

Importing tasks in the paused state (selective restore)

try_enqueue_paused(task, progress_cb, complete_cb) imports a task in the Paused state without scheduling it. Unlike try_enqueue, it performs no network or file I/O: the task is registered into the scheduler and a single Paused progress record is emitted, but no HEAD/GET/upload request is sent and no file is opened until you start it.

This is the entry point for "restore on restart, then let the user choose what to download now":

1. On restart, rebuild a PounceTask for each unfinished row in your database.
2. Import each one with try_enqueue_paused(...) and keep the returned TaskId.
3. Render your task list from your own persisted progress. The Paused record reports 0.0 progress because no prepare has run yet, so the SDK does not know the real offset until the task is resumed.
4. When the user selects which transfers to start, call resume(task_id) for those; the rest stay paused. Downloads resume from the on-disk partial file length; uploads resume from the server-reported next_byte.

use rusty_cat::api::{DownloadPounceBuilder, MeowClient, MeowConfig};

async fn restore_and_start(client: &MeowClient) -> Result<(), rusty_cat::api::MeowError> {
    // Rebuilt from your own database after a restart.
    let task = DownloadPounceBuilder::new(
        "report.bin",
        "./downloads/report.bin",
        1024 * 1024,
        "https://example.com/report.bin",
    )
    .build();

    // Imported paused: no HTTP request is sent and no file is opened here.
    let task_id = client
        .try_enqueue_paused(task, |_record| {}, |_id, _payload| {})
        .await?;

    // Later, when the user chooses to start this transfer:
    client.resume(task_id).await?;
    Ok(())
}

See examples/restore_import_paused.rs for a complete, runnable demonstration that imports several tasks paused and resumes only a selected subset.

Examples

Shutdown checklist

• Keep callbacks short and non-blocking.
• Store every returned TaskId if you plan to pause, resume, cancel, or inspect a task.
• Use snapshot() for runtime diagnostics.
• Always call close().await during shutdown.
• Recreate a new MeowClient after a successful close() if you need to submit more work.