Crate throttle 

Resource throttling and rate limiting for file operations

This crate provides throttling mechanisms to control resource usage during file operations. It helps prevent system overload and allows for controlled resource consumption when working with large filesets or in resource-constrained environments.

Overview

The throttle system provides three types of rate limiting:

1. Open Files Limit - Controls the maximum number of simultaneously open files
2. Operations Throttle - Limits the number of operations per second
3. I/O Operations Throttle - Limits the number of I/O operations per second based on chunk size

All throttling is implemented using token-bucket semaphores that are automatically replenished at configured intervals.

Usage Patterns

Open Files Limit

Prevents exceeding system file descriptor limits by controlling concurrent file operations:

use throttle::{set_max_open_files, open_file_permit};

// Configure max open files (typically 80% of system limit)
set_max_open_files(8000);

// Acquire permit before opening file
let _guard = open_file_permit().await;
// Open file here - permit is automatically released when guard is dropped

Operations Throttling

Limits general operations per second to reduce system load:

use throttle::{init_ops_tokens, run_ops_replenish_thread, get_ops_token};
use std::time::Duration;

// Initialize with 100 operations per second
let ops_per_interval = 10;
let interval = Duration::from_millis(100); // 10 tokens / 100ms = 100/sec

init_ops_tokens(ops_per_interval);

// Start replenishment in background
tokio::spawn(run_ops_replenish_thread(ops_per_interval, interval));

// Acquire token before each operation
get_ops_token().await;
// Perform operation here

I/O Operations Throttling

Limits I/O operations based on file size and chunk size, useful for bandwidth control:

use throttle::{init_iops_tokens, run_iops_replenish_thread, get_file_iops_tokens};
use std::time::Duration;

// Initialize with desired IOPS limit
let iops_per_interval = 100;
let interval = Duration::from_millis(100);
let chunk_size = 64 * 1024; // 64 KB chunks

init_iops_tokens(iops_per_interval);
tokio::spawn(run_iops_replenish_thread(iops_per_interval, interval));

// For a 1 MB file with 64 KB chunks: requires 16 tokens
let file_size = 1024 * 1024;
get_file_iops_tokens(chunk_size, file_size).await;
// Copy file here

Token Calculation

For I/O throttling, the number of tokens required for a file is calculated as:

tokens = ⌈file_size / chunk_size⌉

This allows throttling to be proportional to the amount of data transferred.

Replenishment Strategy

Tokens are replenished using a background task that periodically adds tokens to the semaphore. The replenishment rate can be tuned by adjusting:

• tokens_per_interval: Number of tokens added each interval
• interval: Time between replenishments

For example, to achieve 1000 ops/sec:

• Option 1: 100 tokens every 100ms
• Option 2: 10 tokens every 10ms

The implementation automatically scales down to prevent excessive granularity while maintaining the target rate.

Thread Safety

All throttling mechanisms are thread-safe and can be used across multiple async tasks and threads. The semaphores use efficient parking_lot mutexes internally.

Performance Considerations

• Open Files Limit: No replenishment needed, permits released automatically
• Ops/IOPS Throttle: Background task overhead is minimal (~1 task per throttle type)
• Token Acquisition: Async operation that parks task when no tokens available

Examples

Complete Throttled Copy Operation

use throttle::*;
use std::time::Duration;

async fn setup_throttling() {
    // Limit to 80% of 10000 max files
    set_max_open_files(8000);

    // 500 operations per second
    init_ops_tokens(50);
    tokio::spawn(run_ops_replenish_thread(50, Duration::from_millis(100)));

    // 1000 IOPS (with 64KB chunks ≈ 64 MB/s)
    init_iops_tokens(100);
    tokio::spawn(run_iops_replenish_thread(100, Duration::from_millis(100)));
}

async fn copy_file_throttled(size: u64) {
    let chunk_size = 64 * 1024;

    // Acquire all required permits
    get_ops_token().await;
    get_file_iops_tokens(chunk_size, size).await;
    let _file_guard = open_file_permit().await;

    // Perform copy operation
    // ...
}

Structs

OpenFileGuard

OpsInFlightGuard

PendingMetaGuard

Backpressure guard for in-flight metadata-only operations (rm, cmp).

Resource

Which throttled metadata resource a permit / cap belongs to.

Enums

MetadataOp

Which metadata syscall a permit / cap belongs to.

Side

Which filesystem side a metadata syscall touches.

Constants

N_META_OPS

Number of MetadataOp variants. Keep in sync when adding variants.

N_META_RESOURCES

Total number of distinct (Side, MetadataOp) controllers.

N_SIDES

Number of Side variants.

Functions

current_ops_in_flight_limit

Current in-flight concurrency cap for the given Resource, for metrics and integration tests. Returns 0 when the cap has been set to zero (disabled) or has never been configured.

disable_ops_throttle

Disable the ops-throttle, making get_ops_token a no-op. Mirrors the “unlimited on this dimension” semantics of Decision so an adaptive controller can transition a previously-set rate back to “no limit” by sending rate_per_sec: None.

enable_ops_throttle

Re-enable the ops-throttle after disable_ops_throttle — the counterpart that allows a controller to toggle rate capping on and off via Decision::rate_per_sec. Returns true if enablement took effect, false if the throttle was never initialized (i.e. --ops-throttle was not set at startup) so there is nothing to enable.

get_file_iops_tokens

get_ops_token

init_iops_tokens

init_ops_tokens

open_file_permit

ops_in_flight_permit

Acquire a permit from the ops-in-flight cap for the given Resource. No-op (returns immediately) when that resource’s cap is not configured.

pending_meta_permit

run_iops_replenish_thread

run_ops_replenish_thread

set_iops_replenish

Dynamically update the iops-throttle replenish count. See set_ops_replenish for the semantics.

set_max_open_files

Configure the spawn-time concurrency caps from a single knob.

set_max_ops_in_flight

Dynamically set the maximum number of concurrent operations in flight for the given Resource.

set_ops_replenish

Dynamically update the ops-throttle replenish count.