Struct stretto::AsyncCacheBuilder

pub struct AsyncCacheBuilder<K: Hash + Eq, V: Send + Sync + 'static, KH = DefaultKeyBuilder, C = DefaultCoster<V>, U = DefaultUpdateValidator<V>, CB = DefaultCacheCallback<V>, S = RandomState> { /* private fields */ }

This is supported on crate feature async only.

The AsyncCacheBuilder struct is used when creating AsyncCache instances if you want to customize the AsyncCache settings.

• num_counters

  num_counters is the number of 4-bit access counters to keep for admission and eviction. Dgraph’s developers have seen good performance in setting this to 10x the number of items you expect to keep in the cache when full.

  For example, if you expect each item to have a cost of 1 and max_cost is 100, set num_counters to 1,000. Or, if you use variable cost values but expect the cache to hold around 10,000 items when full, set num_counters to 100,000. The important thing is the number of unique items in the full cache, not necessarily the max_cost value.

• max_cost

  max_cost is how eviction decisions are made. For example, if max_cost is 100 and a new item with a cost of 1 increases total cache cost to 101, 1 item will be evicted.

  max_cost can also be used to denote the max size in bytes. For example, if max_cost is 1,000,000 (1MB) and the cache is full with 1,000 1KB items, a new item (that’s accepted) would cause 5 1KB items to be evicted.

  max_cost could be anything as long as it matches how you’re using the cost values when calling insert.

• key_builder

  KeyBuilder is the hashing algorithm used for every key. In Stretto, the Cache will never store the real key. The key will be processed by KeyBuilder. Stretto has two default built-in key builder, one is TransparentKeyBuilder, the other is DefaultKeyBuilder. If your key implements TransparentKey trait, you can use TransparentKeyBuilder which is faster than DefaultKeyBuilder. Otherwise, you should use DefaultKeyBuilder You can also write your own key builder for the Cache, by implementing KeyBuilder trait.

  Note that if you want 128bit hashes you should use the full (u64, u64), otherwise just fill the u64 at the 0 position, and it will behave like any 64bit hash.

• buffer_size

  buffer_size is the size of the insert buffers. The Dgraph’s developers find that 32 * 1024 gives a good performance.

  If for some reason you see insert performance decreasing with lots of contention (you shouldn’t), try increasing this value in increments of 32 * 1024. This is a fine-tuning mechanism and you probably won’t have to touch this.

• metrics

  Metrics is true when you want real-time logging of a variety of stats. The reason this is a AsyncCacheBuilder flag is because there’s a 10% throughput performance overhead.

• ignore_internal_cost

  Set to true indicates to the cache that the cost of internally storing the value should be ignored. This is useful when the cost passed to set is not using bytes as units. Keep in mind that setting this to true will increase the memory usage.

• cleanup_duration

  The Cache will cleanup the expired values every 500ms by default.

• update_validator

  By default, the Cache will always update the value if the value already exists in the cache. UpdateValidator is a trait to support customized update policy (check if the value should be updated if the value already exists in the cache).

• callback

  CacheCallback is for customize some extra operations on values when related event happens..

• coster

  Coster is a trait you can pass to the AsyncCacheBuilder in order to evaluate item cost at runtime, and only for the insert calls that aren’t dropped (this is useful if calculating item cost is particularly expensive, and you don’t want to waste time on items that will be dropped anyways).

  To signal to Stretto that you’d like to use this Coster trait:

  1. Set the Coster field to your own Coster implementation.
  2. When calling insert for new items or item updates, use a cost of 0.

• hasher

  The hasher for the AsyncCache, default is SipHasher.

Implementations

impl<K: Hash + Eq, V: Send + Sync + 'static> AsyncCacheBuilder<K, V>

pub fn new(num_counters: usize, max_cost: i64) -> Self

Create a new AsyncCacheBuilder

impl<K: Hash + Eq, V: Send + Sync + 'static, KH: KeyBuilder<K>> AsyncCacheBuilder<K, V, KH>

pub fn new_with_key_builder(num_counters: usize, max_cost: i64, kh: KH) -> Self

Create a new AsyncCacheBuilder

impl<K, V, KH, C, U, CB, S> AsyncCacheBuilder<K, V, KH, C, U, CB, S> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,
    C: Coster<V>,
    U: UpdateValidator<V>,
    CB: CacheCallback<V>,
    S: BuildHasher + Clone + 'static + Send, 

pub fn finalize(self) -> Result<AsyncCache<K, V, KH, C, U, CB, S>, CacheError>

Build Cache and start all threads needed by the Cache.

impl<K, V, KH, C, U, CB, S> AsyncCacheBuilder<K, V, KH, C, U, CB, S> where
    K: Hash + Eq,
    V: Send + Sync + 'static,
    KH: KeyBuilder<K>,
    C: Coster<V>,
    U: UpdateValidator<V>,
    CB: CacheCallback<V>,
    S: BuildHasher + Clone + 'static, 

pub fn set_num_counters(self, num_counters: usize) -> Self

Set the number of counters for the Cache.

num_counters is the number of 4-bit access counters to keep for admission and eviction. Dgraph’s developers have seen good performance in setting this to 10x the number of items you expect to keep in the cache when full.

For example, if you expect each item to have a cost of 1 and max_cost is 100, set num_counters to 1,000. Or, if you use variable cost values but expect the cache to hold around 10,000 items when full, set num_counters to 100,000. The important thing is the number of unique items in the full cache, not necessarily the max_cost value.

pub fn set_max_cost(self, max_cost: i64) -> Self

Set the max_cost for the Cache.

max_cost is how eviction decisions are made. For example, if max_cost is 100 and a new item with a cost of 1 increases total cache cost to 101, 1 item will be evicted.

max_cost can also be used to denote the max size in bytes. For example, if max_cost is 1,000,000 (1MB) and the cache is full with 1,000 1KB items, a new item (that’s accepted) would cause 5 1KB items to be evicted.

max_cost could be anything as long as it matches how you’re using the cost values when calling insert.

pub fn set_buffer_size(self, sz: usize) -> Self

Set the insert buffer size for the Cache.

buffer_size is the size of the insert buffers. The Dgraph’s developers find that 32 * 1024 gives a good performance.

If for some reason you see insert performance decreasing with lots of contention (you shouldn’t), try increasing this value in increments of 32 * 1024. This is a fine-tuning mechanism and you probably won’t have to touch this.

pub fn set_metrics(self, val: bool) -> Self

Set whether record the metrics or not.

Metrics is true when you want real-time logging of a variety of stats. The reason this is a Builder flag is because there’s a 10% throughput performance overhead.

pub fn set_ignore_internal_cost(self, val: bool) -> Self

Set whether ignore the internal cost or not.

By default, when insert a value in the Cache, there will always 56 for internal cost, because the size of stored item in Cache is 56(excluding the size of value). Set it to true to ignore the internal cost.

pub fn set_cleanup_duration(self, d: Duration) -> Self

Set the cleanup ticker for Cache, each tick the Cache will clean the expired entries.

pub fn set_key_builder<NKH: KeyBuilder<K>>(
    self,
    kh: NKH
) -> AsyncCacheBuilder<K, V, NKH, C, U, CB, S>

Set the KeyBuilder for the Cache

KeyBuilder is the hashing algorithm used for every key. In Stretto, the Cache will never store the real key. The key will be processed by KeyBuilder. Stretto has two default built-in key builder, one is TransparentKeyBuilder, the other is DefaultKeyBuilder. If your key implements TransparentKey trait, you can use TransparentKeyBuilder which is faster than DefaultKeyBuilder. Otherwise, you should use DefaultKeyBuilder You can also write your own key builder for the Cache, by implementing KeyBuilder trait.

Note that if you want 128bit hashes you should use the full (u64, u64), otherwise just fill the u64 at the 0 position, and it will behave like any 64bit hash.

pub fn set_coster<NC: Coster<V>>(
    self,
    coster: NC
) -> AsyncCacheBuilder<K, V, KH, NC, U, CB, S>

Set the coster for the Cache.

Coster is a trait you can pass to the Builder in order to evaluate item cost at runtime, and only for the insert calls that aren’t dropped (this is useful if calculating item cost is particularly expensive, and you don’t want to waste time on items that will be dropped anyways).

To signal to Stretto that you’d like to use this Coster trait:

1. Set the Coster field to your own Coster implementation.
2. When calling insert for new items or item updates, use a cost of 0.

pub fn set_update_validator<NU: UpdateValidator<V>>(
    self,
    uv: NU
) -> AsyncCacheBuilder<K, V, KH, C, NU, CB, S>

Set the update validator for the Cache.

By default, the Cache will always update the value if the value already exists in the cache. UpdateValidator is a trait to support customized update policy (check if the value should be updated if the value already exists in the cache).

pub fn set_callback<NCB: CacheCallback<V>>(
    self,
    cb: NCB
) -> AsyncCacheBuilder<K, V, KH, C, U, NCB, S>

Set the callbacks for the Cache.

CacheCallback is for customize some extra operations on values when related event happens.

pub fn set_hasher<NS: BuildHasher + Clone + 'static>(
    self,
    hasher: NS
) -> AsyncCacheBuilder<K, V, KH, C, U, CB, NS>

Set the hasher for the Cache. Default is SipHasher.