Expand description
Stretto
Stretto is a pure Rust implementation for https://github.com/dgraph-io/ristretto.
A high performance thread-safe memory-bound Rust cache.
English | 简体中文
Features
- Internal Mutability - Do not need to use
Arc<RwLock<Cache<...>>
for concurrent code, you just needCache<...>
orAsyncCache<...>
- Sync and Async - Stretto support sync and runtime agnostic async.
- In sync,
Cache
starts two extra OS level threads. One is policy thread, the other is writing thread. - In async,
AsyncCache
starts two extra green threads. One is policy thread, the other is writing thread.
- In sync,
- Store policy Stretto only store the value, which means the cache does not store the key.
- High Hit Ratios - with Dgrpah’s developers unique admission/eviction policy pairing, Stretto’s performance is best in class.
- Eviction: SampledLFU - on par with exact LRU and better performance on Search and Database traces.
- Admission: TinyLFU - extra performance with little memory overhead (12 bits per counter).
- Fast Throughput - use a variety of techniques for managing contention and the result is excellent throughput.
- Cost-Based Eviction - any large new item deemed valuable can evict multiple smaller items (cost could be anything).
- Fully Concurrent - you can use as many threads as you want with little throughput degradation.
- Metrics - optional performance metrics for throughput, hit ratios, and other stats.
- Simple API - just figure out your ideal
CacheBuilder
values and you’re off and running.
Table of Contents
Usage
Example
Please see examples on github.
Config
The CacheBuilder
or AsyncCacheBuilder
struct is used when creating Cache
/AsyncCache
instances if you want to customize the Cache
/AsyncCache
settings.
num_counters
num_counters
is the number of 4-bit access counters to keep for admission and eviction.
Dgraph’s developers have seen good performance in setting this to 10x the number of
items you expect to keep in the cache when full.
For example, if you expect each item to have a cost of 1 and max_cost
is 100, set num_counters
to 1,000.
Or, if you use variable cost values but expect the cache to hold around 10,000 items when full,
set num_counters to 100,000. The important thing is the number of unique items in the full cache,
not necessarily the max_cost
value.
max_cost
max_cost
is how eviction decisions are made. For example,
if max_cost is 100 and a new item with a cost of 1 increases total cache cost to 101,
1 item will be evicted.
max_cost
can also be used to denote the max size in bytes. For example,
if max_cost is 1,000,000 (1MB) and the cache is full with 1,000 1KB items,
a new item (that’s accepted) would cause 5 1KB items to be evicted.
max_cost
could be anything as long as it matches how you’re using the cost values when calling insert
.
key_builder
KeyBuilder
is the hashing algorithm used for every key. In Stretto, the Cache will never store the real key.
The key will be processed by KeyBuilder
. Stretto has two default built-in key builder,
one is TransparentKeyBuilder
, the other is DefaultKeyBuilder
. If your key implements TransparentKey
trait,
you can use TransparentKeyBuilder
which is faster than DefaultKeyBuilder
. Otherwise, you should use DefaultKeyBuilder
You can also write your own key builder for the Cache, by implementing KeyBuilder
trait.
Note that if you want 128bit hashes you should use the full (u64, u64)
,
otherwise just fill the u64
at the 0
position, and it will behave like
any 64bit hash.
buffer_size
buffer_size
is the size of the insert buffers. The Dgraph’s developers find that 32 * 1024 gives a good performance.
If for some reason you see insert performance decreasing with lots of contention (you shouldn’t), try increasing this value in increments of 32 * 1024. This is a fine-tuning mechanism and you probably won’t have to touch this.
metrics
Metrics is true when you want real-time logging of a variety of stats. The reason this is a CacheBuilder flag is because there’s a 10% throughput performance overhead.
ignore_internal_cost
Set to true indicates to the cache that the cost of internally storing the value should be ignored. This is useful when the cost passed to set is not using bytes as units. Keep in mind that setting this to true will increase the memory usage.
cleanup_duration
The Cache will cleanup the expired values every 500ms by default.
update_validator
By default, the Cache will always update the value if the value already exists in the cache.
UpdateValidator
is a trait to support customized update policy (check if the value should be updated
if the value already exists in the cache).
callback
CacheCallback
is for customize some extra operations on values when related event happens.
coster
Coster
is a trait you can pass to the CacheBuilder
in order to evaluate
item cost at runtime, and only for the insert
calls that aren’t dropped (this is
useful if calculating item cost is particularly expensive, and you don’t want to
waste time on items that will be dropped anyways).
To signal to Stretto that you’d like to use this Coster
trait:
- Set the
Coster
field to your ownCoster
implementation. - When calling
insert
for new items or item updates, use a cost of 0.
hasher
The hasher for the Cache, default is SipHasher.
Structs
async
async
AsyncCacheBuilder
struct is used when creating AsyncCache
instances if you want to customize the AsyncCache
settings.sync
sync
KeyBuilder
for the Cache.Histogram
stores the information needed to represent the sizes of the keys and values
as a histogram.u64
.
If the key does not implement the trait TransparentKey
, please use DefaultKeyBuilder
or write a custom key builder.get
method of the Cache.
It contains a RwLockReadGuard
and a value reference.get_mut
method of the Cache.
It contains a RwLockWriteGuard
and a mutable value reference.Enums
Traits
insert
calls that aren’t dropped (this is
useful if calculating item cost is particularly expensive, and you don’t want to
waste time on items that will be dropped anyways).KeyBuilder
is the hashing algorithm used for every key. In Stretto, the Cache will never store the real key.
The key will be processed by KeyBuilder
. Stretto has two default built-in key builder,
one is TransparentKeyBuilder
, the other is DefaultKeyBuilder
. If your key implements TransparentKey
trait,
you can use TransparentKeyBuilder
which is faster than DefaultKeyBuilder
. Otherwise, you should use DefaultKeyBuilder
You can also write your own key builder for the Cache, by implementing KeyBuilder
trait.TransparentKeyBuilder
as the KeyBuilder
for the Cache
.