Skip to main content

system_mutation_decision

lean_ctx_proxy::cache_policy

Function system_mutation_decision 

Source 
pub fn system_mutation_decision(
    prefix_before: u64,
    system_before: &Value,
    system_after: &Value,
    reuse: u32,
    cost: &ModelCost,
) -> MutationDecision
Expand description

Priced gate for compressing the system prompt while the client caches a message prefix (#1912). The system block sits in front of every cached message, so rewriting it busts the whole prefix: the bust is priced on prefix_before (system + cached messages, see prefix_tokens) and the saving on the measured shrink of the system block alone.