rusty-cat

rusty-cat is an async Rust SDK for resumable file upload and download. It gives applications a compact public facade for building transfer tasks, running those tasks in a background scheduler, receiving progress callbacks, and plugging in protocol-specific implementations such as plain HTTP, Aliyun OSS, Aliyun OSS presigned URLs, Azure Blob Storage, and Azure Blob SAS URLs.

The crate is designed for applications that need reliable large-file transfer without forcing a specific storage backend or database layer. The SDK handles scheduling, chunk dispatch, retry, pause/resume/cancel commands, and progress fan-out. Your application remains responsible for business records, credential management, user permissions, and provider-specific setup.

The recommended public import is:

use rusty_cat::api::*;

Existing module paths still work, but rusty_cat::api::* is the stable, beginner-friendly entry point. Using the facade also makes future refactoring easier because most application code can import SDK types from a single module.

Package, platform, Rust, and license

Badge Markdown

The Crates.io, Docs.rs, and License badge Markdown is shown below. These badges are safe to paste into downstream README files or generated documentation pages:

[![Crates.io](https://img.shields.io/crates/v/rusty-cat.svg)](https://crates.io/crates/rusty-cat)
[![Docs.rs](https://docs.rs/rusty-cat/badge.svg)](https://docs.rs/rusty-cat)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Feature highlights

Capability matrix

Architecture overview

Quick start

Add the crate:

[dependencies]
rusty-cat = "0.1.1"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

For OSS providers, enable only what you need. Keeping the feature list small reduces optional dependencies and makes it clearer which cloud integrations your application actually uses:

[dependencies]
rusty-cat = { version = "0.1.1", features = ["aliyun-oss-direct"] }

For a focused comparison of direct credentials versus presigned/SAS URLs, see Provider feature flags: direct vs presigned/SAS.

Complete end-to-end example

This example starts from MeowConfig, creates a MeowClient, registers listeners, builds a task, submits it, waits for the completion/failure signal, inspects a snapshot, and closes the client. It uses an HTTP range download task because that path works without cloud credentials; the same client lifecycle applies to upload tasks and OSS/Azure provider tasks.

use std::sync::{mpsc, Arc, Mutex};
use std::time::Duration;

use rusty_cat::api::{
	DownloadPounceBuilder, FileTransferRecord, Log, MeowClient, MeowConfig, TransferStatus,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
	let config = MeowConfig::builder()
		.max_upload_concurrency(2)
		.max_download_concurrency(2)
		.http_timeout(Duration::from_secs(30))
		.tcp_keepalive(Duration::from_secs(60))
		.command_queue_capacity(256)
		.worker_event_queue_capacity(1024)
		.build()?;

	let client = MeowClient::new(config);

	client.set_debug_log_listener(Some(Arc::new(|log: Log| {
		println!("[rusty-cat] {log}");
	})))?;

	let global_listener_id = client.register_global_progress_listener(|record| {
		println!(
			"global: task={} file={} progress={:.2}% status={:?}",
			record.task_id(),
			record.file_name(),
			record.progress() * 100.0,
			record.status(),
		);
	})?;

	let task = DownloadPounceBuilder::new(
		"example.bin",
		"./downloads/example.bin",
		1024 * 1024,
		"https://example.com/example.bin",
	)
	.with_client_file_sign("business-file-id-001")
	.with_max_chunk_retries(3)
	.build();

	let (tx, rx) = mpsc::channel::<Result<(), String>>();
	let tx = Arc::new(Mutex::new(Some(tx)));
	let progress_tx = Arc::clone(&tx);
	let complete_tx = Arc::clone(&tx);

	let task_id = client
		.try_enqueue(
			task,
			move |record: FileTransferRecord| {
				println!(
					"task={} progress={:.2}% status={:?}",
					record.task_id(),
					record.progress() * 100.0,
					record.status(),
				);

				match record.status() {
					TransferStatus::Failed(err) => {
						if let Ok(mut guard) = progress_tx.lock() {
							if let Some(tx) = guard.take() {
								let _ = tx.send(Err(format!("transfer failed: {err}")));
							}
						}
					}
					TransferStatus::Canceled => {
						if let Ok(mut guard) = progress_tx.lock() {
							if let Some(tx) = guard.take() {
								let _ = tx.send(Err("transfer canceled".to_string()));
							}
						}
					}
					_ => {}
				}
			},
			move |task_id, provider_payload| {
				println!("task {task_id} complete; payload={provider_payload:?}");
				if let Ok(mut guard) = complete_tx.lock() {
					if let Some(tx) = guard.take() {
						let _ = tx.send(Ok(()));
					}
				}
			},
		)
		.await?;

	println!("enqueued task: {task_id}");

	match rx.recv_timeout(Duration::from_secs(300)) {
		Ok(Ok(())) => println!("transfer finished"),
		Ok(Err(err)) => eprintln!("transfer ended with error: {err}"),
		Err(err) => eprintln!("timed out while waiting for completion callback: {err}"),
	}

	let snapshot = client.snapshot().await?;
	println!(
		"snapshot: queued={}, active={}",
		snapshot.queued_groups, snapshot.active_groups
	);

	client.unregister_global_progress_listener(global_listener_id)?;
	client.clear_global_listener()?;
	client.set_debug_log_listener(None)?;
	client.close().await?;
	assert!(client.is_closed());
	Ok(())
}

MeowClient API guide

There is no public enqueue(...) method in the current API. Use try_enqueue(...); the name is intentional because enqueue uses fail-fast backpressure. If your application submits many tasks at once, increase command_queue_capacity or retry CommandSendFailed with your own backoff policy.

Configuration parameters

MeowConfig and MeowConfigBuilder

Start with MeowConfig::default() for a safe baseline or use MeowConfig::builder() for validated customization. The configuration is immutable after the client is created, which prevents accidental runtime changes from affecting tasks already in the scheduler.

UploadPounceBuilder

Beginner tips:

• Use a chunk_size between 1 MiB and 8 MiB for common object storage workloads unless your provider requires a different size. Very small chunks increase request overhead; very large chunks reduce retry granularity.
• Put provider protocol objects in Arc and pass them to with_breakpoint_upload(...) because the executor can move transfer work across async tasks.
• For restart recovery, persist enough business metadata in your own database to rebuild the same logical task later, including local path, remote URL/object key, direction, chunk size, and provider type.

DownloadPounceBuilder

Download HTTP methods are intentionally not configurable. Resumable HTTP download depends on standard HEAD and GET range behavior. If a gateway or provider needs a non-standard method, implement BreakpointDownload and inject it with with_breakpoint_download(...).

OSS upload/download developer guides

OSS and Blob workflows are provider-specific, so detailed beginner guides live in separate documents. The SDK does not persist your keys, secrets, account keys, tokens, presigned URLs, or SAS URLs in a built-in database or credential store. Some values are held in memory while executing tasks. You must provide them from your application or trusted backend, and you should avoid logging them in progress callbacks or debug listeners.

If you are deciding which provider feature to enable first, start with Provider feature flags: direct vs presigned/SAS.

Persistence and custom database integration

rusty-cat intentionally has no built-in database. This keeps the SDK small and lets you choose SQLite, PostgreSQL, Redis, a mobile database, or an existing business persistence layer. The SDK emits progress records and terminal states; your application decides how those records map to durable business state.

Recommended pattern:

1. Create your own transfer table with fields such as business file ID, local path, remote URL/object key, direction, chunk size, provider, status, progress, and credential reference.
2. Register per-task and/or global progress callbacks.
3. In callbacks, persist FileTransferRecord values or forward them to a persistence worker. Do not perform slow database writes directly on the callback path; prefer batching or sending records to your own worker queue.
4. On process restart, query unfinished rows and rebuild equivalent PounceTask values.
5. Call try_enqueue(...) again. Provider protocols can resume from local/remote checkpoint information when the same logical task is recreated correctly.

Never persist raw cloud secrets unless your security model explicitly allows it. Prefer storing a reference to a backend-owned credential or generating fresh short-lived presigned/SAS URLs.

Examples

Shutdown checklist

• Keep callbacks short and non-blocking.
• Store every returned TaskId if you plan to pause, resume, cancel, or inspect a task.
• Use snapshot() for runtime diagnostics.
• Always call close().await during shutdown.
• Recreate a new MeowClient after a successful close() if you need to submit more work.