rusty-cat

rusty-cat is an async Rust SDK for resumable file upload and download. It gives applications a compact public facade for building transfer tasks, running those tasks in a background scheduler, receiving progress callbacks, and plugging in protocol-specific implementations such as plain HTTP, Aliyun OSS, Aliyun OSS presigned URLs, Azure Blob Storage, and Azure Blob SAS URLs.

The crate is designed for applications that need reliable large-file transfer without forcing a specific storage backend or database layer. The SDK handles scheduling, chunk dispatch, retry, pause/resume/cancel commands, and progress fan-out. Your application remains responsible for business records, credential management, user permissions, and provider-specific setup.

The recommended public import is:

use rusty_cat::api::*;

Existing module paths still work, but rusty_cat::api::* is the stable, beginner-friendly entry point. Using the facade also makes future refactoring easier because most application code can import SDK types from a single module.

Package, platform, Rust, and license

Item	Value
Crate	rusty-cat
Version	0.2.4
Rust edition	2021
Runtime	Tokio-based async runtime hosted by an internal scheduler thread
HTTP stack	reqwest with rustls-tls
Platforms	Linux, macOS, and Windows targets supported by stable Rust, Tokio, and reqwest
License	MIT
Repository	https://github.com/0barman/rusty-cat

Badge Markdown

The Crates.io, Docs.rs, and License badge Markdown is shown below. These badges are safe to paste into downstream README files or generated documentation pages:

[![Crates.io](https://img.shields.io/crates/v/rusty-cat.svg)](https://crates.io/crates/rusty-cat)
[![Docs.rs](https://docs.rs/rusty-cat/badge.svg)](https://docs.rs/rusty-cat)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)

Feature highlights

Capability matrix

Capability	Supported	Notes
HTTP resumable upload	Yes	Upload tasks are split into chunks and delegated to a BreakpointUpload implementation. The default style supports multipart/form-data chunk requests, and provider plugins can replace the request logic.
HTTP resumable download	Yes	Download tasks use HEAD during preparation and GET with Range headers for chunk transfer through StandardRangeDownload.
In-memory upload source	Yes	UploadPounceBuilder::from_bytes(file_name, bytes, chunk_size) uploads from an in-memory buffer instead of a file path; chunking and sizing behave the same as file-backed uploads.
Aliyun OSS direct upload/download	Yes	Enable aliyun-oss-direct; use AliOssDirectUpload and AliOssDirectDownload when the client process is trusted to hold AccessKey credentials.
Aliyun OSS presigned upload/download	Yes	Enable aliyun-oss-presigned; use short-lived presigned part and range URLs generated by your backend.
Azure Blob direct upload/download	Yes	Enable azure-blob-direct; use Shared Key-authenticated block upload and range download when the client process is trusted to hold the storage account key.
Azure Blob SAS upload/download	Yes	Enable azure-blob-sas; use short-lived SAS URLs generated by your backend.
Provider-neutral presigned primitives	Yes	Enable presigned to use PresignedMultipartUpload/PresignedRangeDownload and their plans against any S3/OSS-style backend your server can presign, without a provider-specific feature.
Presigned plan validation	Yes	PresignedMultipartUploadPlan::validate() rejects zero-size parts, duplicate offsets, and parts outside the declared object size before any byte is sent.
Presigned completion/abort callbacks	Yes	A plan can carry complete_request/abort_request plus an optional PresignedCompletionBodyBuilder, so your backend can verify and merge parts (or clean up) when a transfer ends.
Presigned URL refresh on expiry	Yes	PresignedUploadUrlRefresher/PresignedDownloadUrlRefresher with refresh_before_secs refresh part/range URLs before they expire during long transfers.
Upload concurrency setting	Yes	MeowConfig::builder().max_upload_concurrency(n) limits the number of upload groups running at the same time.
Download concurrency setting	Yes	MeowConfig::builder().max_download_concurrency(n) limits the number of download groups running at the same time.
Upload progress	Yes	Per-task progress callbacks passed to MeowClient::try_enqueue(...) receive FileTransferRecord snapshots.
Download progress	Yes	The same callback model is used for downloads, so upload and download UI code can share one progress-record handler.
Global progress listener	Yes	register_global_progress_listener(...) observes all tasks created by the client, which is useful for dashboards and persistence workers.
Global SDK debug logs	Yes	set_debug_log_listener(...) installs a process-global SDK log listener for diagnostics and integration tests.
Application-managed persistence	Yes	The SDK intentionally does not persist transfer state in an embedded database, so it can fit server, desktop, mobile, and CLI applications.
Custom database adaptation	Yes	Persist records from callbacks/listeners in your own database and rebuild tasks after restart.
Callback panic isolation	Yes	User callbacks are isolated from scheduler execution; callbacks should still be fast, non-blocking, and panic-free.
Chunk failure retry	Yes	with_max_chunk_retries(...) on upload and download builders controls additional retries after the first failed chunk transfer.
Upload prepare retry	Yes	UploadPounceBuilder::with_max_upload_prepare_retries(...) controls additional retries after the first failed upload preparation attempt.
Intra-file parallel parts	Opt-in	UploadPounceBuilder::with_max_parts_in_flight(n) uploads up to n chunks of one file concurrently. Default 1 (serial). Honored only for out-of-order-safe upload protocols — all four provider protocols (Aliyun OSS direct & presigned multipart, Azure Blob direct & SAS block blob); the built-in default HTTP upload stays serial. Progress, resume, pause, and cancel still observe a single contiguous prefix. See Concurrent chunked transfer.
Intra-file parallel download	Opt-in	DownloadPounceBuilder::with_max_parts_in_flight(n) fetches up to n range chunks of one file concurrently and writes them at absolute offsets. Default 1 (serial). Honored only for range-safe protocols (StandardRangeDownload) when the total size is known. Peak memory n * chunk_size. Interrupted concurrent downloads resume from a <file>.rcdl sidecar (re-fetching only missing parts); the sidecar is deleted on success. The download URL must be stable for the transfer's lifetime (no per-request re-signing on the range path). See Concurrent chunked transfer.
Transport-aware retry & backoff	Yes	Beyond the retry counts above, transient transport failures (connection reset, timeout, incomplete message) are retried with exponential backoff and jitter, while non-transient errors fail fast.
Disk-full & local-file-removed detection	Yes	Local I/O failures are classified into dedicated error codes DiskFull and LocalFileRemoved, so callers can react specifically instead of treating every I/O error the same.
Pause/resume/cancel	Yes	Use pause(...), resume(...), and cancel(...) with the returned TaskId.
Paused import / selective restore	Yes	try_enqueue_paused(...) imports a task in the paused state with no network/file I/O; resume only the user-selected subset on restart.
Resume after process restart / crash	Yes	Rebuild the task and re-enqueue after a restart: downloads resume from the on-disk partial file, default uploads from the server nextByte, and presigned uploads from re-injected parts. See Resuming after a restart.
Presigned multipart resume across restart	Yes	PresignedMultipartUpload::with_resumed_parts(...) re-injects parts persisted by a previous run; prepare resumes past the longest verified contiguous prefix and re-sends the rest.
Provider multipart session id surfacing	Yes	UploadResumeInfo::provider_upload_id and AliOssDirectUpload::current_upload_id() expose the provider UploadId (not a secret) so orphaned multipart sessions can be aborted out of band.
Upload abort on cancel	Yes	Canceling an upload runs the protocol's abort_upload (for example OSS AbortMultipartUpload) so uncommitted parts/blocks stop accruing storage cost.
Snapshot diagnostics	Yes	snapshot() returns queued and active scheduler state for monitoring and troubleshooting.
Custom HTTP client	Yes	Inject a preconfigured reqwest::Client with MeowConfigBuilder::http_client(...) for proxy, TLS, default headers, or observability integration.
Custom upload protocol	Yes	Implement BreakpointUpload to integrate business-specific upload APIs.
Custom download protocol	Yes	Implement BreakpointDownload to integrate custom range-download authentication or headers.
Custom upload request method/headers	Yes	with_method(...) and with_headers(...) on UploadPounceBuilder customize the default upload request line and headers.
Per-task download HTTP override	Yes	DownloadPounceBuilder::with_breakpoint_download_http(...) overrides the range Accept header for a single task without changing global config.

Architecture overview

Layer	Main types	Responsibility
Public facade	rusty_cat::api::*	One import point for client, config, task builders, callbacks, errors, status, logs, and optional providers.
Client	MeowClient	Owns immutable config, lazily starts the executor, submits tasks, controls lifecycle, and manages listeners.
Config	MeowConfig, MeowConfigBuilder	Defines concurrency, queue capacities, HTTP timeout/keepalive, range-download behavior, and optional custom HTTP client.
Task builders	UploadPounceBuilder, DownloadPounceBuilder	Convert simple parameters into executable PounceTask values.
Scheduler	Internal executor	Runs background workers, queues tasks, dispatches chunks, retries failures, and emits events.
Protocol plugins	BreakpointUpload, BreakpointDownload	Implement provider-specific signing, presigned URLs, chunk requests, and completion behavior.
Observability	FileTransferRecord, TransferSnapshot, Log	Per-task progress, global progress events, queue snapshots, and debug logs.

Quick start

Add the crate:

[dependencies]
rusty-cat = "0.2.4"
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

For OSS providers, enable only what you need. Keeping the feature list small reduces optional dependencies and makes it clearer which cloud integrations your application actually uses:

[dependencies]
rusty-cat = { version = "0.2.4", features = ["aliyun-oss-direct"] }

Feature	Purpose
aliyun-oss-direct	Aliyun OSS direct upload/download with AccessKey credentials and OSS Signature Version 4 signing.
aliyun-oss-presigned	Aliyun OSS presigned multipart upload and range download helpers.
azure-blob-direct	Azure Blob upload/download with Shared Key authentication.
azure-blob-sas	Azure Blob SAS upload/download helpers.
presigned	Provider-neutral presigned multipart/range primitives.
aliyun-oss	Convenience umbrella that currently enables aliyun-oss-presigned only. It does not pull in aliyun-oss-direct.
azure-blob	Convenience umbrella that currently enables azure-blob-sas only. It does not pull in azure-blob-direct.
oss-providers	Both umbrellas together: aliyun-oss + azure-blob (Aliyun OSS presigned + Azure Blob SAS). It does not enable the direct/AccessKey/Shared Key features.
all	Enables all four provider features (aliyun-oss-direct, aliyun-oss-presigned, azure-blob-direct, azure-blob-sas). Use it for examples, not minimal production builds.

The aliyun-oss, azure-blob, and oss-providers umbrellas intentionally select the presigned/SAS flows, because those are the recommended model for untrusted clients (your backend holds the credentials). When you need the direct AccessKey/Shared Key flows, enable aliyun-oss-direct and/or azure-blob-direct explicitly.

For a focused comparison of direct credentials versus presigned/SAS URLs, see Provider feature flags: direct vs presigned/SAS.

Complete end-to-end example

This example starts from MeowConfig, creates a MeowClient, registers listeners, builds a task, submits it, waits for the completion/failure signal, inspects a snapshot, and closes the client. It uses an HTTP range download task because that path works without cloud credentials; the same client lifecycle applies to upload tasks and OSS/Azure provider tasks.

use std::sync::{mpsc, Arc, Mutex};
use std::time::Duration;

use rusty_cat::api::{
	DownloadPounceBuilder, FileTransferRecord, Log, MeowClient, MeowConfig, TransferStatus,
};

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
	let config = MeowConfig::builder()
		.max_upload_concurrency(2)
		.max_download_concurrency(2)
		.http_timeout(Duration::from_secs(30))
		.tcp_keepalive(Duration::from_secs(60))
		.command_queue_capacity(256)
		.worker_event_queue_capacity(1024)
		.build()?;

	let client = MeowClient::new(config);

	client.set_debug_log_listener(Some(Arc::new(|log: Log| {
		println!("[rusty-cat] {log}");
	})))?;

	let global_listener_id = client.register_global_progress_listener(|record| {
		println!(
			"global: task={} file={} progress={:.2}% status={:?}",
			record.task_id(),
			record.file_name(),
			record.progress() * 100.0,
			record.status(),
		);
	})?;

	let task = DownloadPounceBuilder::new(
		"example.bin",
		"./downloads/example.bin",
		1024 * 1024,
		"https://example.com/example.bin",
	)
	.with_client_file_sign("business-file-id-001")
	.with_max_chunk_retries(3)
	.build();

	let (tx, rx) = mpsc::channel::<Result<(), String>>();
	let tx = Arc::new(Mutex::new(Some(tx)));
	let progress_tx = Arc::clone(&tx);
	let complete_tx = Arc::clone(&tx);

	let task_id = client
		.try_enqueue(
			task,
			move |record: FileTransferRecord| {
				println!(
					"task={} progress={:.2}% status={:?}",
					record.task_id(),
					record.progress() * 100.0,
					record.status(),
				);

				match record.status() {
					TransferStatus::Failed(err) => {
						if let Ok(mut guard) = progress_tx.lock() {
							if let Some(tx) = guard.take() {
								let _ = tx.send(Err(format!("transfer failed: {err}")));
							}
						}
					}
					TransferStatus::Canceled => {
						if let Ok(mut guard) = progress_tx.lock() {
							if let Some(tx) = guard.take() {
								let _ = tx.send(Err("transfer canceled".to_string()));
							}
						}
					}
					_ => {}
				}
			},
			move |task_id, provider_payload| {
				println!("task {task_id} complete; payload={provider_payload:?}");
				if let Ok(mut guard) = complete_tx.lock() {
					if let Some(tx) = guard.take() {
						let _ = tx.send(Ok(()));
					}
				}
			},
		)
		.await?;

	println!("enqueued task: {task_id}");

	match rx.recv_timeout(Duration::from_secs(300)) {
		Ok(Ok(())) => println!("transfer finished"),
		Ok(Err(err)) => eprintln!("transfer ended with error: {err}"),
		Err(err) => eprintln!("timed out while waiting for completion callback: {err}"),
	}

	let snapshot = client.snapshot().await?;
	println!(
		"snapshot: queued={}, active={}",
		snapshot.queued_groups, snapshot.active_groups
	);

	client.unregister_global_progress_listener(global_listener_id)?;
	client.clear_global_listener()?;
	client.set_debug_log_listener(None)?;
	client.close().await?;
	assert!(client.is_closed());
	Ok(())
}

MeowClient API guide

Function	Use it when	Important notes
MeowClient::new(config)	Create the SDK entry point.	The executor starts lazily on the first task operation. MeowClient is not Clone because it owns scheduler state; wrap it in Arc<MeowClient> when multiple async tasks or threads need shared access.
http_client()	Need a reqwest::Client aligned with SDK config.	Returns the injected custom client when one was configured; otherwise builds a client from http_timeout and tcp_keepalive. This is useful when protocol code outside the executor must make compatible HTTP calls.
register_global_progress_listener(listener)	Observe all task progress records.	Returns a GlobalProgressListenerId. Use this for UI-wide progress aggregation, persistence queues, or monitoring. Keep callback work fast.
unregister_global_progress_listener(id)	Remove one global listener.	Returns Ok(false) when the ID does not exist, so cleanup code can call it safely.
clear_global_listener()	Remove every global progress listener.	Useful during shutdown, integration-test cleanup, or application logout flows.
set_debug_log_listener(Some(listener))	Receive SDK debug logs.	The listener is process-global rather than client-local. Pass None to clear it before shutdown or when tests need isolation.
try_enqueue(task, progress_cb, complete_cb).await	Submit an upload/download task.	This performs asynchronous submission, not synchronous transfer completion. It fails fast when the command queue is full. Store the returned TaskId for pause/resume/cancel operations.
try_enqueue_paused(task, progress_cb, complete_cb).await	Import a task in the paused state without scheduling it.	Performs no network or file I/O until you call resume(...). Use it for restart/restore: import many tasks, then resume only the user-selected subset. Same fail-fast back-pressure as try_enqueue. See the persistence section below.
pause(task_id).await	Pause a queued or running task.	Sends a command to the scheduler. A paused task can be resumed later with the same TaskId.
resume(task_id).await	Continue a paused task.	Keeps the same TaskId and asks the scheduler to continue from available local/remote progress.
cancel(task_id).await	Stop a task.	Cancellation is best-effort and may run provider cleanup such as aborting a multipart session. Treat canceled tasks as terminal unless your application deliberately creates a new task.
snapshot().await	Inspect queued and active groups.	Useful for dashboards, health checks, and debugging scheduler behavior under concurrency.
close().await	Shut down.	Mandatory for clean shutdown: cancels in-flight work, flushes Paused events, drains callbacks, and joins the scheduler thread. Do not rely on Drop for production shutdown.
is_closed()	Check whether the client is closed.	A successfully closed client cannot be reopened; create a new MeowClient if you need to submit more work.

There is no public enqueue(...) method in the current API. Use try_enqueue(...); the name is intentional because enqueue uses fail-fast backpressure. If your application submits many tasks at once, increase command_queue_capacity or retry CommandSendFailed with your own backoff policy.

Configuration parameters

MeowConfig and MeowConfigBuilder

Start with MeowConfig::default() for a safe baseline or use MeowConfig::builder() for validated customization. The configuration is immutable after the client is created, which prevents accidental runtime changes from affecting tasks already in the scheduler.

Parameter	Default	Constraint	Description
max_upload_concurrency	2	>= 1	Maximum upload groups processed concurrently.
max_download_concurrency	2	>= 1	Maximum download groups processed concurrently.
breakpoint_download_http.range_accept	application/octet-stream	Valid header value	Default Accept header for range download chunks.
http_client	None	Reusable reqwest::Client	Optional custom HTTP client for proxy, TLS, default headers, or observability.
http_timeout	5s	Positive duration	Per-request timeout for internally built HTTP clients.
tcp_keepalive	30s	Positive duration	TCP keepalive for internally built HTTP clients.
command_queue_capacity	128	>= 1	Queue for enqueue, pause, resume, cancel, snapshot, and close commands.
worker_event_queue_capacity	256	>= 1	Queue for progress/state events.

Builder/accessor	Description
MeowConfig::builder()	Creates a builder initialized with defaults.
max_upload_concurrency(n) / max_upload_concurrency()	Sets/reads upload concurrency. Recommended range: 1..=64.
max_download_concurrency(n) / max_download_concurrency()	Sets/reads download concurrency. Recommended range: 1..=64.
http_client(client)	Injects a custom reqwest::Client for proxy, TLS, headers, or observability.
http_timeout(duration) / http_timeout()	Sets/reads HTTP timeout. Typical range: 3s..=60s.
tcp_keepalive(duration) / tcp_keepalive()	Sets/reads TCP keepalive. Typical range: 15s..=120s.
command_queue_capacity(n) / command_queue_capacity()	Sets/reads control queue capacity.
worker_event_queue_capacity(n) / worker_event_queue_capacity()	Sets/reads worker event queue capacity.
breakpoint_download_http(config) / breakpoint_download_http()	Sets/reads range-download HTTP behavior.
build()	Validates constraints and returns MeowConfig.

UploadPounceBuilder

Method	Required?	Description
UploadPounceBuilder::new(file_name, file_path, chunk_size)	Yes	Creates a file-backed upload task. chunk_size == 0 is normalized to the SDK default.
UploadPounceBuilder::from_bytes(file_name, bytes, chunk_size)	Alternative	Creates an in-memory upload task. The Vec<u8> is moved into bytes::Bytes.
with_url(url)	Usually yes	Sets target upload URL. For direct OSS/Azure, this is the final object/blob URL. For presigned flows, it is commonly the first part URL or logical target URL.
with_file_path(path)	Optional	Replaces the local file source.
with_bytes(bytes)	Optional	Replaces the source with in-memory bytes.
with_method(method)	Optional	Sets HTTP method for default/custom upload requests. Default is POST.
with_headers(headers)	Optional	Replaces base request headers.
with_breakpoint_upload(upload)	Optional	Sets a per-task custom BreakpointUpload, such as Aliyun/Azure direct or presigned upload.
with_max_chunk_retries(retries)	Optional	Sets additional retries after the first failed chunk attempt. 0 disables chunk retry. Default is 3.
with_max_upload_prepare_retries(retries)	Optional	Sets additional retries after the first failed upload prepare attempt. Default is 3.
with_max_parts_in_flight(n)	Optional	Maximum chunks of this file uploaded concurrently (intra-file parallel parts). Default 1 (strict serial, unchanged behavior). A value > 1 is only honored when the chosen upload protocol proves out-of-order safety — all four provider protocols do (Aliyun OSS direct & presigned multipart, Azure Blob direct & SAS block blob); the built-in default HTTP upload stays serial. 0 is normalized to 1. Peak upload memory for a file source is n * chunk_size, so keep n bounded. See Concurrent chunked transfer.
build()	Yes	Reads file metadata for file-backed uploads and returns PounceTask; may return std::io::Error.

Beginner tips:

• Use a chunk_size between 1 MiB and 8 MiB for common object storage workloads unless your provider requires a different size. Very small chunks increase request overhead; very large chunks reduce retry granularity.
• Put provider protocol objects in Arc and pass them to with_breakpoint_upload(...) because the executor can move transfer work across async tasks.
• For restart recovery, persist enough business metadata in your own database to rebuild the same logical task later, including local path, remote URL/object key, direction, chunk size, and provider type.

When you do not attach a provider plugin, uploads use the built-in default protocol. Its exact request/response format — and how the fileMd5 signature is derived — is documented in Default HTTP upload protocol contract.

DownloadPounceBuilder

Method	Required?	Description
DownloadPounceBuilder::new(file_name, file_path, chunk_size, url)	Yes	Creates a range-download task. The SDK uses HEAD for prepare and GET with Range for chunks.
with_url(url)	Optional	Replaces the remote download URL.
with_file_path(path)	Optional	Replaces the local output path.
with_headers(headers)	Optional	Replaces base request headers for HEAD and range GET.
with_client_file_sign(sign)	Optional	Sets a client-defined file signature shown in progress records. Useful for database keys.
with_breakpoint_download(download)	Optional	Sets a per-task custom BreakpointDownload, such as Aliyun/Azure direct or presigned range download.
with_breakpoint_download_http(config)	Optional	Overrides per-task range download HTTP behavior.
with_max_chunk_retries(retries)	Optional	Sets additional retries after the first failed range chunk attempt. 0 disables chunk retry. Default is 3.
with_max_parts_in_flight(n)	Optional	Maximum range chunks of this file fetched concurrently (intra-file parallel download). Default 1 (strict serial, unchanged behavior). A value > 1 is honored only when the download protocol proves range safety (StandardRangeDownload does) and the total size is known up front (with_total_size or a HEAD that returns Content-Length); otherwise the download stays serial. 0 is normalized to 1. Peak download memory is n * chunk_size, so keep n bounded.
with_total_size(size)	Optional	Supplies a known total size so the executor SKIPS the HEAD probe and builds the range grid directly. 0 (default) means "discover via HEAD". Required in practice to get intra-file concurrency for a URL whose server does not support HEAD (for example a presigned GET URL).
build()	Yes	Returns PounceTask. Validation happens during enqueue/runtime.

Download HTTP methods are intentionally not configurable. Resumable HTTP download depends on standard HEAD and GET range behavior. If a gateway or provider needs a non-standard method, implement BreakpointDownload and inject it with with_breakpoint_download(...).

Range headers and query-embedded auth (presigned / SAS)

Adding a Range header to a GET does not invalidate query-embedded authentication. RFC 7233 range requests carry the range in a request header, and presigned/SAS signatures (sig, X-Amz-Signature, …) cover the URL and query parameters, not arbitrary request headers — so the same signed range URL can be fetched in as many Range slices as you like, including concurrently. The range Accept header is overridable per task with with_breakpoint_download_http(BreakpointDownloadHttpConfig { range_accept }); if a strict Azure gateway also requires an explicit x-ms-version, add it (and any other provider header) through the task's base headers with with_headers(...), which are applied to both the HEAD and the range GET — never by editing the signed URL.

Concurrent-download resume semantics (<file>.rcdl)

A concurrent download (with_max_parts_in_flight(n) with n > 1) always writes a <file>.rcdl ("resumable download log") sidecar next to the target. It records which fixed-size parts of the pre-sized file are durably on disk, so an interrupted transfer re-fetches only the missing parts. The sidecar survives cancel/failure on purpose so the next run can resume; it is deleted only on successful completion. A one-shot caller that will never resume can delete it manually.

The sidecar is bound to (URL identity, total, chunk, max_parts) and to the target's on-disk length. If any of those differs on the next run — a different URL, a resized object, a different chunking, or a target that was deleted/truncated (so its length no longer equals total) — the stale sidecar is ignored and a fresh full download runs. It is never mis-applied to a target it does not match, so a leftover .rcdl can only ever cause a safe re-download, never silent corruption.

For a step-by-step, beginner-friendly walkthrough of concurrent chunked upload and download — the two concurrency knobs, the parallel-safe protocol matrix, per-provider recipes, memory sizing, and .rcdl resume — see Concurrent chunked transfer (single-file parallel parts).

Custom-protocol migration

Existing custom BreakpointDownload implementations are unaffected: supports_parallel_parts defaults to false, so a custom protocol stays strictly serial until it explicitly overrides that to true (and only when each range_url/merge_range_get_headers result is independent of any other chunk's completion). Concurrency additionally requires the total size to be known — supply it with with_total_size(...) or via a HEAD that returns Content-Length — otherwise the transfer falls back to the serial path even for a range-safe protocol.

Error handling and retries

Most SDK calls return Result<_, MeowError>. MeowError::code() is a stable numeric code (an i32, suitable for FFI or structured logging) and MeowError::msg() is a human-readable message. Branch on code(), never on the message text. The numeric codes come from InnerErrorCode.

The codes you will most often branch on:

Code	InnerErrorCode	When it happens
102	ParameterEmpty	A required value (URL, file name, non-zero size) was empty at enqueue.
103	DuplicateTaskError	The same file/task is already queued, running, or paused.
107	ClientClosed	The client was closed; create a new MeowClient to submit more work.
108	TaskNotFound	pause/resume/cancel referenced an unknown or already-terminal TaskId.
111	CommandSendFailed	The command queue is full (fail-fast back-pressure). Retry with backoff or raise command_queue_capacity.
117	InvalidTaskState	The operation is invalid in the task's current state (for example resuming a task that is not paused).
120	TaskCanceled	The task was canceled before reaching Complete.
121	DiskFull	The local disk ran out of space while writing a download.
122	LocalFileRemoved	The local source/target file was removed or replaced mid-transfer.

See the Error codes reference for the complete list (codes 101–122), with the meaning and suggested handling of every variant.

Retry and transient errors

Two builder knobs control how many times a failed step is retried before the task fails:

• with_max_chunk_retries(n) — extra attempts after the first failed chunk transfer (default 3; 0 disables chunk retry).
• UploadPounceBuilder::with_max_upload_prepare_retries(n) — extra attempts after the first failed upload prepare (default 3).

Within those budgets the SDK only retries transient transport failures (connection reset, timeout, incomplete message), and it waits between attempts using exponential backoff with jitter, so a flaky network or a briefly overloaded server does not trigger a retry storm. Non-transient errors (an HTTP 403, a malformed response, an invalid range) fail fast without consuming the retry budget, because retrying them would not help.

To continue a task that ultimately failed (or that you paused), rebuild it and call try_enqueue again, or resume(...) a paused one; both continue from the last checkpoint. See Resuming uploads and downloads after a restart.

SDK debug logs: levels and what to persist

set_debug_log_listener(Some(listener)) installs one process-global callback that receives every Log the SDK emits. Each entry carries a LogLevel that tells you how to treat it. The levels are ordered by severity and are designed so you can split a high-volume diagnostic stream from a small, durable troubleshooting stream:

Level	What it is	Frequency	Persist?
Trace	per-chunk / per-poll / per-retry hot-loop detail	very high	No — drop or sample only; never store
Debug	low-volume internal diagnostics	a few per task	optional / short-term ring buffer only
Info	normal operational notes	low	optional
Key	task & executor lifecycle checkpoints (created, enqueued, started, prepared, resumed, completed, paused, cancelled, closed, listener changes)	a few per task	Yes — keep and forward when reporting a bug
Warn	recoverable anomaly, caller misuse, backpressure, failed cleanup	per anomaly	Yes
Error	a chunk/part upload or download failed, a task failed, an HTTP error, a signing failure, or a caught panic — with full structured context	per failure	Yes

Rule of thumb: persist everything at >= LogLevel::Key (Key | Warn | Error) and never persist the high-frequency Trace stream. The library exposes this policy so you do not have to hard-code it:

use std::sync::Arc;
use rusty_cat::api::{Log, LogLevel, MeowClient, MeowConfig};

let client = MeowClient::new(MeowConfig::default());
client.set_debug_log_listener(Some(Arc::new(|log: Log| {
    // Never store the high-frequency Trace tier — it fires per chunk/poll/retry.
    if log.level() == LogLevel::Trace {
        return;
    }
    if log.level().persist_recommended() {
        // Key | Warn | Error → keep these and ship them to your log server.
        ship_to_log_server(&log);
    } else {
        // Debug | Info → optional; a short-term in-memory ring buffer at most.
        local_ring_buffer(&log);
    }
}))).unwrap();
// where `ship_to_log_server` / `local_ring_buffer` are your own sinks.

When you hit a problem, collect the persisted Key + Warn + Error entries and send them to the library author — the Key checkpoints reconstruct what the task was doing and the Error entries pinpoint the failure.

Structured fields for triage

Besides log.level(), log.tag() and log.message(), Error and Key entries may carry structured context you can index or filter on: task_id(), object_key(), part_index(), offset(), byte_len(), http_status(), attempt(), max_retries(), error_code(), and url(). Log's Display also appends every present field as key=value, so format!("{log}") is a self-describing line.

Secret safety

The SDK never puts a raw SAS/presigned URL, signature, credential, response body, or request header into a log: URLs go through sanitize_url() (signature/credential query params are redacted) and error chains/response bodies go through redact_secrets() before they reach a Log. Both helpers are public (rusty_cat::api::{sanitize_url, redact_secrets}) so you can apply the same redaction to anything you add in a progress callback or debug listener — do not re-log a raw task URL or header map yourself.

OSS upload/download developer guides

OSS and Blob workflows are provider-specific, so detailed beginner guides live in separate documents. The SDK does not persist your keys, secrets, account keys, tokens, presigned URLs, or SAS URLs in a built-in database or credential store. Some values are held in memory while executing tasks. You must provide them from your application or trusted backend, and you should avoid logging them in progress callbacks or debug listeners.

If you are deciding which provider feature to enable first, start with Provider feature flags: direct vs presigned/SAS.

Guide	Feature flag	Example source
Aliyun OSS direct upload/download	aliyun-oss-direct	examples/aliyun_oss_direct_chunk_transfer.rs
Aliyun OSS presigned upload/download	aliyun-oss-presigned	examples/aliyun_oss_presigned_chunk_transfer.rs
Azure Blob direct upload/download	azure-blob-direct	examples/azure_blob_direct_chunk_transfer.rs
Azure Blob SAS upload/download	azure-blob-sas	examples/azure_blob_sas_chunk_transfer.rs

Persistence and custom database integration

rusty-cat intentionally has no built-in database. This keeps the SDK small and lets you choose SQLite, PostgreSQL, Redis, a mobile database, or an existing business persistence layer. The SDK emits progress records and terminal states; your application decides how those records map to durable business state.

Recommended pattern:

1. Create your own transfer table with fields such as business file ID, local path, remote URL/object key, direction, chunk size, provider, status, progress, and credential reference.
2. Register per-task and/or global progress callbacks.
3. In callbacks, persist FileTransferRecord values or forward them to a persistence worker. Do not perform slow database writes directly on the callback path; prefer batching or sending records to your own worker queue.
4. On process restart, query unfinished rows and rebuild equivalent PounceTask values.
5. Call try_enqueue(...) again. Provider protocols can resume from local/remote checkpoint information when the same logical task is recreated correctly.

Never persist raw cloud secrets unless your security model explicitly allows it. Prefer storing a reference to a backend-owned credential or generating fresh short-lived presigned/SAS URLs.

New to restart recovery? For a step-by-step, beginner-friendly walkthrough of resuming after the process is killed or crashes — covering exactly what to persist for downloads, default HTTP uploads, and presigned multipart uploads, with copy-pasteable code — see Resuming uploads and downloads after a process restart or crash.

Importing tasks in the paused state (selective restore)

try_enqueue_paused(task, progress_cb, complete_cb) imports a task in the Paused state without scheduling it. Unlike try_enqueue, it performs no network or file I/O: the task is registered into the scheduler and a single Paused progress record is emitted, but no HEAD/GET/upload request is sent and no file is opened until you start it.

This is the entry point for "restore on restart, then let the user choose what to download now":

1. On restart, rebuild a PounceTask for each unfinished row in your database.
2. Import each one with try_enqueue_paused(...) and keep the returned TaskId.
3. Render your task list from your own persisted progress. The Paused record reports 0.0 progress because no prepare has run yet, so the SDK does not know the real offset until the task is resumed.
4. When the user selects which transfers to start, call resume(task_id) for those; the rest stay paused. Downloads resume from the on-disk partial file length; uploads resume from the server-reported next_byte.

use rusty_cat::api::{DownloadPounceBuilder, MeowClient, MeowConfig};

async fn restore_and_start(client: &MeowClient) -> Result<(), rusty_cat::api::MeowError> {
    // Rebuilt from your own database after a restart.
    let task = DownloadPounceBuilder::new(
        "report.bin",
        "./downloads/report.bin",
        1024 * 1024,
        "https://example.com/report.bin",
    )
    .build();

    // Imported paused: no HTTP request is sent and no file is opened here.
    let task_id = client
        .try_enqueue_paused(task, |_record| {}, |_id, _payload| {})
        .await?;

    // Later, when the user chooses to start this transfer:
    client.resume(task_id).await?;
    Ok(())
}

See examples/restore_import_paused.rs for a complete, runnable demonstration that imports several tasks paused and resumes only a selected subset.

Examples

Example	What it demonstrates
examples/http_local_chunk_transfer.rs	Local plain HTTP range download and custom binary upload protocol.
examples/restore_import_paused.rs	Paused import + selective resume (restart/restore) against a local range server.
examples/resume_after_restart.rs	Resume a download from a partial file left by a previous (killed) run.
examples/aliyun_oss_direct_chunk_transfer.rs	Aliyun OSS direct upload/download with AccessKey signing.
examples/aliyun_oss_direct_custom_chunk_transfer.rs	Aliyun OSS direct custom chunk transfer.
examples/aliyun_oss_presigned_chunk_transfer.rs	Aliyun OSS presigned multipart upload and range download.
examples/azure_blob_direct_chunk_transfer.rs	Azure Blob direct upload/download with Shared Key.
examples/azure_blob_direct_custom_chunk_transfer.rs	Azure Blob direct custom chunk transfer.
examples/azure_blob_sas_chunk_transfer.rs	Azure Blob upload/download with SAS URLs.

Shutdown checklist

• Keep callbacks short and non-blocking.
• Store every returned TaskId if you plan to pause, resume, cancel, or inspect a task.
• Use snapshot() for runtime diagnostics.
• Always call close().await during shutdown.
• Recreate a new MeowClient after a successful close() if you need to submit more work.