Crate tool_result_cache 

tool-result-cache

Content-addressable LRU cache for LLM agent tool calls.

When an agent calls search_web(q="anthropic prompt cache") three times in a row, it is wasting tokens, money, and time. ToolCache is a small LRU cache keyed on a stable hash of (tool_name, args) with optional TTL per entry and optional max size.

Quick example

use std::time::Duration;
use serde_json::json;
use tool_result_cache::ToolCache;

let mut cache: ToolCache<String> = ToolCache::new()
    .with_capacity(128)
    .with_ttl(Duration::from_secs(300));

let args = json!({"q": "anthropic prompt cache"});
let value = cache
    .get_or_set("search_web", &args, || "result-a".to_string())
    .clone();
assert_eq!(value, "result-a");

// Second call hits the cache and never invokes the closure.
let again = cache
    .get_or_set("search_web", &args, || panic!("would not run"))
    .clone();
assert_eq!(again, "result-a");

Key canonicalization

Arguments are JSON values (via serde_json::Value). Object keys are sorted recursively before hashing, so {"a": 1, "b": 2} and {"b": 2, "a": 1} hit the same entry.

LRU implementation

The cache stores entries in a HashMap<String, Entry> and tracks recency in a Vec<String> ordered from oldest at index 0 to most-recently used at the tail. Capacity-based eviction pops index 0; _touch removes the key from its current position and pushes it onto the tail. The internal ordering vector keeps this dependency-light (only serde_json is pulled in), at the cost of O(n) Vec::remove on touch where n is the number of cached entries. For the typical agent-side cache size (a few hundred to a few thousand entries) this is fine.

Structs

CacheStats

Snapshot of ToolCache counters at a point in time.

ToolCache

LRU cache for tool results, keyed by (tool_name, args).

Functions

cached_call

Convenience: look up (tool_name, args) in cache or compute and store compute(). Returns an owned clone of the cached value.

make_key

Build a stable cache key from a tool name and JSON arguments.