pdk-cache-lib 1.10.0

PDK Cache Library
Documentation

PDK Cache Library   Latest Version cache--lib msrv

pdk-cache-lib is a library that provides data caching functionality for building Flex Gateway custom policies.

You may be looking for:

cache-lib in action

[dependencies]

# This package is included as a transitive dependency of the pdk crate.
pdk = { version = "1.10"}

use anyhow::Result;
use pdk::cache::{Cache, CacheBuilder, CacheError, EvictionStrategy, LruStrategy};
use pdk::hl::*;

const MAX_CACHE_ENTRIES: usize = 100;

async fn request_filter(request_state: RequestState, cache:&dyn Cache) {
    let _headers_state = request_state.into_headers_state().await;

    // Save a value to the cache
    let _save_result: Result<(), CacheError> = cache.save("example-key", "example-value".as_bytes().to_vec());

    // Retrieve value from the cache.
    let _retrieved_value: Option<Vec<u8>> = cache.get("example-key");

    // Delete a value from the cache.
    let _deleted_value: Option<Vec<u8>> = cache.delete("example-key");

    // remove all keys from the cache.
    cache.purge();

}

async fn response_filter(response_state: ResponseState, cache:&dyn Cache) {
    let _headers_state = response_state.into_headers_state().await;

    // Save a value to the cache
    let _save_result: Result<(), CacheError> = cache.save("example-key", "example-value".as_bytes().to_vec());

    // Retrieve value from the cache.
    let _retrieved_value: Option<Vec<u8>> = cache.get("example-key");

    // Delete a value from the cache.
    let _deleted_value: Option<Vec<u8>> = cache.delete("example-key");

    // remove all keys from the cache.
    cache.purge();

}

#[entrypoint]
// Inject the CacheBuilder to the 'configure' function.
async fn configure(launcher: Launcher, cache_builder: CacheBuilder) -> Result<()> {

    // Create a FIFO cache (default eviction strategy)
    let fifo_cache = cache_builder
        .new("fifo-cache-id".to_string())
        .max_entries(MAX_CACHE_ENTRIES)
        // .shared() // if uncommented the cache won't be isolated to the current policy.
        .build();

    // Or create a (sharded, approximate) LRU cache that evicts least recently used entries
    let lru_cache = cache_builder
        .new("lru-cache-id".to_string())
        .eviction_strategy(EvictionStrategy::Lru(LruStrategy::default()))
        .max_entries(MAX_CACHE_ENTRIES)
        .build();

    let filter = on_request(|rs| request_filter(rs, &fifo_cache))
        .on_response(|rs| response_filter(rs, &lru_cache));
    launcher.launch(filter).await?;
    Ok(())
}

Eviction Strategies

FIFO (First-In-First-Out)

The default eviction strategy. When the cache reaches its maximum size, the oldest entries (by insertion time) are evicted first. The get() operation does not affect eviction order.

let cache = cache_builder
    .new("cache-id".to_string())
    .eviction_strategy(EvictionStrategy::Fifo) // Optional, this is the default
    .max_entries(100)
    .build();

LRU (Sharded, Approximate Least Recently Used)

Evicts the least recently accessed entries when the cache is full. Both get() and save() operations update the access time, so frequently accessed items are less likely to be evicted.

This is not a globally-exact LRU: the state is split across independent shards and eviction happens per shard (see LRU Sharding below). Use LruStrategy::default() for the default shard count (16), or configure it with LruStrategy::default().with_shards(n) for a custom count.

let cache = cache_builder
    .new("cache-id".to_string())
    .eviction_strategy(EvictionStrategy::Lru(LruStrategy::default()))
    .max_entries(100)
    .build();

When to use LRU:

  • When you have "hot" data that is accessed frequently
  • When recent access patterns are a good predictor of future access
  • When you want to keep frequently used items in cache longer

When to use FIFO:

  • When all entries have similar access patterns
  • When insertion order correlates with data freshness
  • When you want simpler, more predictable eviction behavior

LRU Sharding Optimization

The LRU cache uses an internal sharding strategy to optimize performance for large caches. By default, the LRU state is split across 16 shards, with each shard maintaining its own independent LRU queue.

Performance Benefits

  • 15-16x serialization reduction for large caches (1000+ entries)
  • Each operation only serializes/deserializes one shard (~4KB) instead of the entire cache (~60KB for 1000 entries)
  • Enables efficient caching for high-throughput scenarios

Configuration

You can configure the number of shards with LruStrategy::default().with_shards(n). The shard count is infallible — any count is accepted and rounded to the nearest power of two (minimum 1) for balanced hash distribution:

// Default: 16 shards (recommended for most use cases)
let cache = cache_builder
    .new("cache-id".to_string())
    .eviction_strategy(EvictionStrategy::Lru(LruStrategy::default()))
    .max_entries(1000)
    .build();

// Custom: 32 shards for very large caches
let large_cache = cache_builder
    .new("large-cache-id".to_string())
    .eviction_strategy(EvictionStrategy::Lru(
        LruStrategy::default().with_shards(32)
    ))
    .max_entries(10000)
    .build();

// Conservative: 8 shards for smaller caches
let small_cache = cache_builder
    .new("small-cache-id".to_string())
    .eviction_strategy(EvictionStrategy::Lru(
        LruStrategy::default().with_shards(8)
    ))
    .max_entries(500)
    .build();

// Exact global LRU: a single shard (no sharding)
let exact_lru = cache_builder
    .new("exact-lru-id".to_string())
    .eviction_strategy(EvictionStrategy::Lru(
        LruStrategy::default().with_shards(1)
    ))
    .max_entries(100)
    .build();

LRU Accuracy Trade-off

With sharding, LRU eviction happens independently within each shard rather than globally across the entire cache:

  • Per-shard LRU: Each shard evicts its own least recently used entries when full
  • Approximate global LRU: With uniform key distribution, per-shard LRU approximates global LRU behavior
  • Acceptable for most use cases: The performance gain (15-16x) far outweighs the slight loss in LRU accuracy

Shard Count Handling

  • Any count is accepted and rounded to the nearest power of two (minimum 1) for efficient hash distribution
  • LruStrategy::default().with_shards(1) yields exact global LRU (single queue, no sharding)
  • For caches smaller than the shard count, only some shards will be used

Breaking Change Notice

Version 1.8.0 introduces a new internal storage format for LRU caches using sharding. Existing LRU caches from previous versions will be treated as empty on first access after upgrade. If you need to preserve cache data across upgrades, consider:

  1. Purging caches before upgrading
  2. Re-warming caches after upgrade
  3. Using time-based cache invalidation strategies

FIFO caches are unaffected by this change.

Support

For assistance with this project, you can access support resources or submit a support case through the official support channel at https://help.mulesoft.com/s/support.

License

Copyright 2026 Salesforce, Inc. All rights reserved. 

These Terms of Use govern the download, installation, and/or use of this software provided by Salesforce, Inc. (“Salesforce”) (the “Software”), were last updated on April 15, 2022, and constitute a legally binding agreement between you and Salesforce. If you do not agree to these Terms of Use, do not install or use the Software.  << The Software may link to third-party software components licensed under various open source licenses (“Open Source Components”). These Terms of Use pertain solely to Salesforce's proprietary code in the Software. It does not alter or extend any rights or obligations regarding the Open Source Components. For clarity, your use of the Open Source Components is governed by the terms of the applicable open source license(s). You are solely responsible for complying with the terms and conditions of those open source licenses.>>
 
Salesforce grants you a worldwide, non-exclusive, no-charge, royalty-free copyright license to reproduce, prepare derivative works of, publicly display, publicly perform, sublicense, and distribute the Software and derivative works subject to these Terms. These Terms shall be included in all copies or substantial portions of the Software. 

Subject to the limited rights expressly granted hereunder, Salesforce reserves all rights, title, and interest in and to all intellectual property subsisting in the Software. No rights are granted to you hereunder other than as expressly set forth herein. Users residing in countries on the United States Office of Foreign Assets Control sanction list, or which are otherwise subject to a US export embargo, may not use the Software.
 
Implementation of the Software may require development work, for which you are responsible. The Software may contain bugs, errors and incompatibilities and is made available on an AS IS basis without support, updates, or service level commitments. 
 
Salesforce reserves the right at any time to modify, suspend, or discontinue, the Software (or any part thereof) with or without notice. You agree that Salesforce shall not be liable to you or to any third party for any modification, suspension, or discontinuance.

You agree to defend Salesforce against any claim, demand, suit or proceeding made or brought against Salesforce by a third party arising out of or accruing from (a) your use of the Software, and (b) any application you develop with the Software that infringes any copyright, trademark, trade secret, trade dress, patent, or other intellectual property right of any person or defames any person or violates their rights of publicity or privacy (each a “Claim Against Salesforce”), and will indemnify Salesforce from any damages, attorney fees, and costs finally awarded against Salesforce as a result of, or for any amounts paid by Salesforce under a settlement approved by you in writing of, a Claim Against Salesforce, provided Salesforce (x) promptly gives you written notice of the Claim Against Salesforce, (y) gives you sole control of the defense and settlement of the Claim Against Salesforce (except that you may not settle any Claim Against Salesforce unless it unconditionally releases Salesforce of all liability), and (z) gives you all reasonable assistance, at your expense.

WITHOUT LIMITING THE GENERALITY OF THE FOREGOING, THE SOFTWARE IS NOT SUPPORTED AND IS PROVIDED “AS IS,” WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. IN NO EVENT SHALL SALESFORCE HAVE ANY LIABILITY FOR ANY DAMAGES, INCLUDING, BUT NOT LIMITED TO, DIRECT, INDIRECT, SPECIAL, INCIDENTAL, PUNITIVE, OR CONSEQUENTIAL DAMAGES, OR DAMAGES BASED ON LOST PROFITS, DATA, OR USE, IN CONNECTION WITH THE SOFTWARE, HOWEVER CAUSED AND WHETHER IN CONTRACT, TORT, OR UNDER ANY OTHER THEORY OF LIABILITY, WHETHER OR NOT YOU HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
 
These Terms of Use shall be governed exclusively by the internal laws of the State of California, without regard to its conflicts of laws rules. Each party hereby consents to the exclusive jurisdiction of the state and federal courts located in San Francisco County, California to adjudicate any dispute arising out of or relating to these Terms of Use and the download, installation, and/or use of the Software. Except as expressly stated herein, these Terms of Use constitute the entire agreement between the parties, and supersede all prior and contemporaneous agreements, proposals, or representations, written or oral, concerning their subject matter. No modification, amendment, or waiver of any provision of these Terms of Use shall be effective unless it is by an update to these Terms of Use that Salesforce makes available, or is in writing and signed by the party against whom the modification, amendment, or waiver is to be asserted.

Data Privacy: Salesforce may collect, process, and store device, system, and other information related to your use of the Software. This information includes, but is not limited to, IP address, user metrics, and other data (“Usage Data”). Salesforce may use Usage Data for analytics, product development, and marketing purposes. You acknowledge that files generated in conjunction with the Software may contain sensitive or confidential data, and you are solely responsible for anonymizing and protecting such data.