Struct PromptCache 

pub struct PromptCache { /* private fields */ }

Thread-safe prompt cache backed by DashMap.

Each entry carries its own TTL (defaulting to default_ttl). Expired entries are lazily evicted on access.

Implementations

impl PromptCache

pub fn new(default_ttl: Duration) -> Self

Create a new cache with the given default TTL for entries.

pub fn get_text(&self, key: &str) -> Option<TextPromptClient>

Retrieve a cached text prompt if it exists and has not expired.

Cache key format: "{name}:{version}" or "{name}:latest".

pub fn put_text(&self, key: &str, prompt: TextPromptClient)

Insert (or replace) a text prompt in the cache using the default TTL.

pub fn get_text_expired(&self, key: &str) -> Option<TextPromptClient>

Retrieve a cached text prompt even if it has expired.

Used for fallback behavior: when the API is unreachable, an expired cached entry is better than no entry at all.

pub fn get_chat(&self, key: &str) -> Option<ChatPromptClient>

Retrieve a cached chat prompt if it exists and has not expired.

pub fn put_chat(&self, key: &str, prompt: ChatPromptClient)

Insert (or replace) a chat prompt in the cache using the default TTL.

pub fn get_chat_expired(&self, key: &str) -> Option<ChatPromptClient>

Retrieve a cached chat prompt even if it has expired.

Used for fallback behavior: when the API is unreachable, an expired cached entry is better than no entry at all.

pub fn clear(&self)

Remove all entries from the cache.

pub fn invalidate_by_prefix(&self, prefix: &str)

Remove all entries whose key starts with the given prefix.

Used to invalidate all cached versions/labels for a prompt name after a create or update operation.