Crate kalosm_common

Structs

AttentionMask

Cache

KvCache

A growable kv cache. This cache wraps candles KvCache with exponentially larger allocations as the sequence length increases.

MaskCache

TensorCache

A growable tensor cache. This cache wraps candles [Cache] with exponentially larger allocations as the sequence length increases.

Enums

CacheError

Functions

accelerated_device_if_available

Create a candle device that uses any available accelerator.

copy_tensor_into_vec

Clear a Vec<T> and copy the contents of a tensor into it.

maybe_autoreleasepool

Wrap a closure in a release pool if the metal feature is enabled