---
title: Caching
description: Keeping computed values with cache::fetch, keys that change when a record is saved, serving stale values while one request refreshes them, the memory and database stores, testing with a cache per test, and answering conditional GETs with ETags.
category: Going Further
order: 8
---
Some values are slow to work out and change rarely: a list from another
service's API, a summary built from many queries. Glacier's cache keeps
such a value under a key for a while, so later requests use it instead
of working it out again. Values live in the process's memory, or in a
database table every server shares. Separately, every page and JSON
response gets an `ETag`, so a browser or API client that already has the
response is answered `304 Not Modified` without the body.
After reading this guide, you will know:
- How to cache a value with `cache::fetch`, and read, write and delete
one yourself.
- How `cache::key` makes keys from records, so a cached value is never
out of date.
- How `fetch_flexible` serves a stale value while one request refreshes
it.
- What happens when many requests miss the same key at once.
- How to choose between the memory and database stores, and prune the
database one.
- How the cache behaves in tests.
- How ETags and `fresh_when` answer conditional GETs.
- What isn't built yet.
The examples come from `services`, the shop from
[Calling Other Services](calling_other_services.html). Its front page
lists the prices it sells, which it gets from Stripe's API: a call that
takes a few hundred milliseconds, for a list that changes a few times a
year.
## Caching a Value
### `cache::fetch`
`glacier::cache::fetch(key, ttl, compute)` returns the value cached under
`key`. When there isn't one, or it has expired, it calls `compute`, a
closure that returns a future of a `Result`, caches the value for `ttl`,
and returns it. The `services` app's order summary, at
`/api/orders/:id`, names the product, which means asking Stripe; the
answer is cached for a day:
```glacier
# GET /api/orders/:id. The key changes whenever the order is saved, so
# the cached summary is never out of date, and needs no deleting.
def summary(ctx) do
order = Order.find(params[:id])!
stripe = ctx.state::<Stripe>()?
product = glacier::cache::fetch(&glacier::cache::key(&order, "product"), 1.days(), fn -> Catalogue.name(stripe, &order.price_id) end)!
json [id: order.id, product: product, amount: order.price_text(), status: order.status]
end
```
`fn -> … end` is a closure that returns the future, without awaiting it,
so `fetch` awaits it only on a miss. A `ttl` is a duration such as
`10.minutes()` or `1.days()`.
If `compute` returns an error, nothing is cached, and `fetch` returns
the error, so the next call tries again.
### Reading, Writing and Deleting
`fetch` covers most uses. `write(key, value, ttl)`, `read::<T>(key)` and
`delete(key)` do one step each:
```glacier
test "keeps a value until it expires" do
glacier::cache::write("greeting", "hello", 10.minutes())!
assert glacier::cache::read::<String>("greeting")! == Some("hello".to_owned())
# A value read as a type it doesn't fit is a miss.
assert glacier::cache::read::<i64>("greeting")! == None
travel(11.minutes())
assert glacier::cache::read::<String>("greeting")! == None
end
test "forgets a value deleted" do
glacier::cache::write("greeting", "hello", 10.minutes())!
glacier::cache::delete("greeting")!
assert glacier::cache::read::<String>("greeting")! == None
end
```
Values are stored as JSON, so any type that serde can serialise and
deserialise can be cached: a list of the app's own `Price`s, a `String`,
a `serde_json::Value`. `read` returns `None` for a key that has no value,
an expired one, or one that doesn't fit the type asked for.
Each read, write and delete is a `tracing` span named `glacier.cache`,
with `cache.operation` (`get`, `set` or `delete`), `cache.key` and, for a
read, `cache.hit`, inside the request's span, as queries are.
### Reading Once, and Forgetting Groups (new)
`cache::pull(key)!` reads a value and deletes it, for something to use
once. `cache::delete_prefix(prefix)!` forgets every key that starts with
`prefix`, and `cache::clear()!` forgets everything; both return how many
values there were:
```glacier
test "pull reads once; delete_prefix forgets a group" do
glacier::cache::write("invite/ada", "abc123", 1.hours())!
assert glacier::cache::pull::<String>("invite/ada")! == Some("abc123".to_owned())
assert glacier::cache::pull::<String>("invite/ada")! == nil
for id in [1, 2, 3] do
glacier::cache::write(&"report/#{id}", &id, 1.days())!
end
glacier::cache::write("prices", &10, 1.days())!
assert glacier::cache::delete_prefix("report/")! == 3
assert glacier::cache::read::<i32>("prices")! == Some(10)
assert glacier::cache::clear()! == 1
end
```
A prefix is matched exactly, case and all. `glacier cache clear` (`app
cache:clear` on a server) clears the app's configured store from the
command line, after a deploy that changes what's cached, say.
### Once a Request (new)
`cache::memo(key, fn -> … end)!` keeps a value for the rest of the
request, in memory, without going to the cache store again: for a value
several parts of a page ask for. The services app reads its price list
from the cache once a request:
```glacier
# The prices, once a request: a page that needs them in several places
# reads the cache once, and keeps the list it read.
def prices_now(stripe: &Stripe) -> Array<Price> do
glacier::cache::memo("stripe/prices", fn -> Catalogue.prices(stripe) end)!
end
```
The next request starts afresh, and outside a request (in a job, say)
`memo` computes the value every time. An error isn't kept.
### Counters (new)
`cache::increment(key, by, ttl)!` adds to a number in one step and
returns the new number, so requests counting at once each count, and
with the database store so do other servers. A key with no number starts
from 0 and expires after `ttl`; one already counting keeps its expiry.
`cache::decrement` takes away, and `cache::read::<i64>` reads the number:
```glacier
test "counts in place" do
assert glacier::cache::increment("views/7", 1, 1.days())! == 1
assert glacier::cache::increment("views/7", 1, 1.days())! == 2
assert glacier::cache::decrement("views/7", 1, 1.days())! == 1
assert glacier::cache::read::<i64>("views/7")! == Some(1)
# The first increment set the expiry; after it, counting starts again.
travel(25.hours())
assert glacier::cache::increment("views/7", 1, 1.days())! == 1
end
```
[Rate limits](authentication.html#rate-limits-in-code) count this way.
### Keys from Records
A value worked out from a record goes out of date when the record
changes. Rather than deleting the value then, put the record's version
in the key. `cache::key(&record, suffix)` is the record's table, its id,
its `updated_at`, then the suffix:
`orders/1-20261005T021941.186953+0000/product`. Saving the record
changes `updated_at`, and so the key, and the next `fetch` finds nothing
under the new key and works the value out again:
```glacier
test "a record's key changes when it's saved" do
order = insert(Order)
key = glacier::cache::key(&order, "product")
assert key.starts_with(&format!("orders/{}-2026", order.id))
assert key.ends_with("/product")
assert glacier::cache::key(&order, "product") == key
travel(1.seconds())
order.email = "bo@example.com"
order.save()!
assert glacier::cache::key(&order, "product") != key
end
```
The summary [above](#cache-fetch) uses one, so marking an order paid
gives it a new summary at once:
```glacier
test "the summary is cached until the order changes" do
order = insert(Order, price_id: "price_basic")
res = get order_summary_path(order.id)
assert res.body =~ "\"product\":\"Basic\""
assert res.body =~ "\"status\":\"pending\""
# Cached: Stripe's new name for the price isn't seen.
stub_request("GET", "https://api.stripe.com/v1/prices", 200, serde_json::json!({"data": [{"id": "price_basic", "nickname": "Basic plus", "unit_amount": 1200, "currency": "gbp"}]}))
travel(2.hours())
assert get(order_summary_path(order.id)).body =~ "\"product\":\"Basic\""
# Saving the order changes its key, so the summary is worked out again.
order.email = "bo@example.com"
order.save()!
assert get(order_summary_path(order.id)).body =~ "\"product\":\"Basic plus\""
end
```
The values under old keys are never read again, and stay until they
expire. A model without timestamps has no `updated_at`, and its keys are
the table and id alone, which don't change. The key reflects only the
record's own `updated_at`: a value that also depends on other records,
such as an author's books, needs those in its key too.
### Values That Are Someone's
A key names a value for everyone who asks with it. A value that's one
user's or one org's, such as a count of their notes, needs their
[Access](authentication.html#who-a-request-is-for-access) in its key, or
the next user would be shown the first one's. `cache::fetch_for(access,
key, …)` puts it there (`cache::key_for(access, key)` is the key it uses,
`user:12/notes/count`), and keys without one stay shared:
```glacier
test "a value cached for someone is theirs" do
ada = UserAccess { user: insert(User) }
bob = UserAccess { user: insert(User) }
insert_for(ada, Note)
assert glacier::cache::fetch_for(&ada, "notes/count", 5.minutes(), fn -> Note.for(ada).count() end)! == 1
# The same key, for Bob: his own value, not Ada's.
assert glacier::cache::fetch_for(&bob, "notes/count", 5.minutes(), fn -> Note.for(bob).count() end)! == 0
assert glacier::cache::key_for(&ada, "notes/count") == "user:#{ada.user.id}/notes/count"
end
```
`glacier audit` points out a plain `cache::fetch` in a controller that
reads `access`, in case its value is someone's.
### Many Callers at Once
When a value expires on a busy page, many requests can miss it at the
same moment. `fetch` lets one of them compute it; the others wait for
that value rather than computing it too. This holds within one process:
two servers that miss together each compute it once.
## Serving Stale Values
`fetch_flexible(key, fresh, stale, compute)` is `fetch` for values that
can be a little out of date. A value younger than `fresh` is returned as
it is. One older than that, but younger than `fresh` plus `stale`, is
still returned at once, and one request, the first to find it stale,
works out a new value and returns that; the others keep getting the old
one meanwhile, without waiting. Past `fresh` plus `stale`, it's computed
as `fetch` would:
```glacier
test "serves a stale value while one caller refreshes it" do
first = glacier::cache::fetch_flexible("flexible/greeting", 10.minutes(), 1.hours(), fn -> std::future::ready(Ok::<_, Error>("hello".to_owned())) end)!
assert first == "hello"
# Fresh: the cached value, without computing.
travel(5.minutes())
assert glacier::cache::fetch_flexible("flexible/greeting", 10.minutes(), 1.hours(), fn -> std::future::ready(Ok::<_, Error>("unused".to_owned())) end)! == "hello"
# Stale: this caller computes a new value and caches it.
travel(30.minutes())
assert glacier::cache::fetch_flexible("flexible/greeting", 10.minutes(), 1.hours(), fn -> std::future::ready(Ok::<_, Error>("hello again".to_owned())) end)! == "hello again"
assert glacier::cache::read::<String>("flexible/greeting")! == Some("hello again".to_owned())
end
```
The shop's prices are fresh for ten minutes, and served stale for up to
an hour after that:
```glacier
# The prices on sale. A list fetched in the last ten minutes is used as
# it is; for an hour after that, the old list is still served while one
# request fetches a new one.
def prices(stripe: &Stripe) -> Array<Price> do
glacier::cache::fetch_flexible("stripe/prices", 10.minutes(), 1.hours(), fn -> stripe.prices() end)!
end
```
The front page and `/api/prices` both use it, so Stripe is asked at most
once in ten minutes, whatever the traffic:
```glacier
# The shop: the prices on sale, from the cache or from Stripe.
def home(ctx) do
prices = Catalogue.prices(ctx.state::<Stripe>()?)!
render prices, title: "Shop"
end
```
```glacier
test "asks Stripe for the prices once in ten minutes" do
stub_prices("Basic")
get "/"
get "/"
assert get("/api/prices").body =~ "\"nickname\":\"Basic\""
assert requests().len() == 1
# Ten minutes on, it's stale: still served, while a request refreshes it.
# Over an hour after that, it's gone, and the next request waits for it.
stub_prices("Basic, renamed")
travel(2.hours())
assert get("/").body =~ "Basic, renamed"
assert requests().len() == 2
end
```
If the refresh fails, the old value is served and kept, and the next
request tries again. So while Stripe is failing, the shop still shows
its prices, for up to the hour; after that there's no value to serve,
and the error reaches the page:
```glacier
test "serves the old prices while Stripe is failing" do
stub_prices("Basic")
get "/"
stub_request("GET", "https://api.stripe.com/v1/prices", 500, serde_json::json!({}))
travel(30.minutes())
res = get "/"
assert res.status == :ok
assert res.body =~ "<h2>Basic</h2>"
# More than an hour past fresh, there's nothing to serve.
travel(2.hours())
assert get("/").status == :bad_gateway
end
```
## Locks (new)
A lock makes one thing happen at a time for a key, across every process
that shares the cache. The `services` app syncs an order when Stripe says
its checkout changed; Stripe can deliver the same event twice at once,
so the sync holds a lock on the checkout session:
```glacier
# Two deliveries for the same session, at once on two servers, take
# turns: the second waits for the first, then finds the order synced.
lock "checkout:#{id}", wait: 10.seconds() do
session = Stripe.new()?.checkout_session(id)!
order_id = session.client_reference_id.as_deref().and_then(fn id -> id.parse::<i64>().ok() end).ok_or_else(fn -> Error.internal("session #{id} has no order") end)?
order = Order.find(order_id)!
order.stripe_session_id = Some(session.id.clone())
order.amount_total = session.amount_total.unwrap_or_default()
if session.payment_status == "paid" && order.paid_at.is_none() do
order.status = OrderStatus::Paid
order.paid_at = Some(Timestamp::now())
end
order.save()!
if order.status == OrderStatus::Paid do
# Once per event, however often the handler runs.
event.once("receipt", OrderMailer.receipt(&order).deliver_later())!
end
end
```
`lock "key" do … end` runs its body holding the lock and lets it go
after, whether the body succeeds or fails. If another caller holds it,
`wait:` says how long to wait for them to finish; without it, or once
the wait is over, the block fails with `409 Conflict` (`` `checkout:…` is
locked``), as `!` passes on. `ttl:` (a minute unless set) is the longest
the lock is held, so a process that dies holding it doesn't hold it for
ever: give the body less time than that.
The lock is a cache key, added only if it isn't there
(`glacier::cache::add`, which apps can use too), so with
`[cache] store = "database"` it's shared by every server, and with the
memory store by the one process. In Rust, it's
`Lock::new("key").wait(…).run(async { … }).await?`, and
`Lock::held("key")` says whether someone holds it.
### A Few at Once (new)
`funnel "key", limit: 3 do … end` is a lock that lets up to three
callers in at once, as Laravel's `Redis::funnel`: for a service that
takes only so many requests at a time. It takes `wait:` and `ttl:` as a
lock does, and fails with `409 Conflict` when every place is taken
after `wait`:
```glacier
# Stripe's price list, asked for by at most three requests at once
# across every server; a fourth waits its turn, up to five seconds.
def prices_in_turn(stripe: &Stripe) -> Array<Price> do
prices = Array.new()
funnel "stripe:prices", limit: 3, wait: 5.seconds() do
prices = stripe.prices()!
end
prices
end
```
In Rust it's `Lock::new("key").limit(3)`.
## Stores
### `[cache] store`
Where values live is set by `store` in the `[cache]` table of the
config:
| `"memory"` | In the process's memory; the default |
| `"database"` | In the `glacier_cache` table, shared by every server |
| `"null"` | Nothing is kept: every `fetch` computes |
Any other value stops the app when it starts. The memory store is the
fastest, but each server process has its own, and it's emptied whenever
the process restarts. It holds at most 10,000 values: past that, it drops
the expired ones, then the oldest tenth.
The `services` app uses the database store in production, so its
servers share one cache, and a deploy doesn't empty it:
```toml
# Shared by every server, in the glacier_cache table.
[cache]
store = "database"
```
The table comes from a migration that calls
`glacier::cache::create_table`:
```glacier
# The table for `[cache] store = "database"`.
migration do
up do
glacier::cache::create_table(s)
end
down do
drop_table :glacier_cache
end
end
```
With the database store, each value is a row with its key, its value as
JSON, and when it expires. A server also keeps each value it reads or
writes in memory for five seconds, so a busy key isn't read from the
table on every request. The other side of that is that a value another
server writes or deletes can take up to five seconds to be seen. The
store uses the current database connection, so inside a transaction its
writes are part of it; if the transaction rolls back, the row is gone,
though the server's own memory may still serve the value for those five
seconds.
The config's store is the whole process's. A test that wants the
database store has to run outside `#[glacier::test]`, which always uses
a memory store of its own (see [Testing](#testing)), and in a file of
its own, as the `services` app's `tests/cache_store.rs` does:
```rust
let config = Config::from_toml(Env::Test, "[cache]\nstore = \"database\"\n").unwrap();
let _app = App::new().config(config).unwrap();
glacier::cache::write("greeting", "hello", 10.minutes())
.await
.unwrap();
let rows = db::sql("SELECT value FROM glacier_cache WHERE key = 'greeting'")
.fetch_all()
.await
.unwrap();
// Stored as JSON.
assert_eq!(rows[0].get::<String>("value").unwrap(), "\"hello\"");
```
### Pruning the Database Store
Expired rows stay in `glacier_cache` until they're deleted: a `fetch`
ignores them, and overwrites them when it computes the value again, but a
key that's never asked for again stays. `glacier::cache::prune()` deletes
the expired rows and returns how many:
```rust
// Expired values stay in the table until they're pruned.
let past = Timestamp::now() - 1.minutes();
db::sql("UPDATE glacier_cache SET expires_at = ?")
.bind(past)
.execute()
.await
.unwrap();
assert_eq!(glacier::cache::prune().await.unwrap(), 1);
```
The `services` app has the jobs scheduler call it once an hour, with
`prune_with`, as [Calling Other
Services](calling_other_services.html#pruning-old-records) describes:
```glacier
# Webhooks (src/webhooks.glacier) are handled in jobs, and orders
# (`prune` in the model) pruned; these add rows without a model to
# delete once an hour.
jobs do
prune_with "webhook_events", fn -> glacier::webhooks::prune(30.days()) end
prune_with "glacier_cache", glacier::cache::prune
end
```
## Testing
Each `#[glacier::test]`, which every `.glacier` test is, has an empty
memory store of its own, whatever `[cache]` configures. A value one test
caches is never seen by another, even when they run at the same time:
```glacier
test "each test starts with an empty cache" do
# The test above wrote `greeting`; this one doesn't see it.
assert glacier::cache::read::<String>("greeting")! == None
end
```
Expiry follows the test's clock, so `travel` moves a test past a `ttl`
without waiting, as the tests above do. To see whether a value was
cached rather than computed again, count what computing it does: the
shop's tests count the requests made to Stripe with `requests()`.
## The Query Cache
Within one request, a model query run a second time with the same SQL and
values is answered from memory rather than the database, as Rails' query
cache does. A layout, a helper and an action that each load the same
record cost one query between them. There's nothing to turn on: each
request starts with an empty cache, and it's gone when the request ends.
It never answers with stale rows from its own request:
- Any write empties it: saving or deleting a record, `update_all`, or raw
SQL that isn't a `SELECT`.
- So does the end of any transaction, committed or rolled back.
- Only reads through models and queries are kept. Raw `Db.sql(…)` reads,
and reads that lock rows (`FOR UPDATE`), always go to the database.
- A request keeps at most its 100 most recently used queries.
Work outside a request, such as a job, a script or a test's own setup,
isn't cached. For a read that must go to the database inside a request,
for instance one that looks for a row another process writes, wrap it in
`glacier::db::query_cache::uncached(…)`. `glacier::testing::queries`
lists only the statements that reached the database, so a cached read
doesn't appear in it.
## Conditional GETs and ETags
### The `ETag` Middleware
The `ETag` middleware, in `glacier::middleware`, is in both
`Pipeline::browser()` and `Pipeline::api()`. For a successful `GET` or
`HEAD`, a `200` or `201` with a body, it adds:
- `ETag`, a weak tag, `W/"…"`, made from a hash of the body.
- `Cache-Control: max-age=0, private, must-revalidate`, unless the
response has its own `Cache-Control`: a browser may keep the response,
but must ask before using it again, and shared caches mustn't keep it.
A browser sends the tag back in `If-None-Match` when it asks for the page
again. If the new response has the same tag, the middleware answers
`304 Not Modified` with no body, and the browser uses its copy:
```glacier
test "answers 304 Not Modified when the page hasn't changed" do
stub_prices("Basic")
res = get "/"
etag = res.header("etag").unwrap().to_owned()
assert etag.starts_with("W/\"")
assert res.headers["cache-control"] == "max-age=0, private, must-revalidate"
glacier::testing::header("if-none-match", &etag)
again = get "/"
assert again.status == :not_modified
assert again.body == ""
# A price changed: a new page, with a new ETag.
stub_prices("Premium")
travel(2.hours())
changed = get "/"
assert changed.status == :ok
assert changed.header("etag").unwrap() != etag
end
```
The action still runs, and renders the whole page, since the tag is a
hash of what it rendered. What a `304` saves is sending the body. Pair it
with the cache to save the work too, as the front page does.
The middleware leaves a response alone when it already has an `ETag` or
`Last-Modified` header, or its `Cache-Control` says `no-cache` or
`no-store`. It doesn't apply to other methods, other statuses, or empty
bodies. A page that includes something different for each visitor, such
as the shop's form with its CSRF token, has a different tag for each
visitor, so each one's browser can only reuse its own copy. Static files
from `public/` have tags of their own.
### Responses for Someone
Any response to a request that has an
[Access](authentication.html#who-a-request-is-for-access), a signed-in
user's, say, gets `Cache-Control: private` and `Vary: Cookie` unless the
action set its own `Cache-Control`, whatever its status and pipeline. A
shared cache, such as a CDN or a proxy, then never serves one user's page
to another. A page that really is the same for everyone can say so with
its own header.
### `fresh_when`
`fresh_when(&ctx, last_modified)` on a response does the same with a time
as well as a tag: it adds `Last-Modified` and an `ETag`, and answers
`304` when the request's `If-None-Match` matches, or, without one, when
its `If-Modified-Since` isn't older than the time given. The
[Controllers](controllers.html#conditional-gets-with-fresh-when) guide
covers it. A response it has answered keeps its own headers, as the
middleware leaves it alone.
## What Isn't Built Yet
Glacier's caching doesn't yet have:
- **Other stores**, such as Redis or Memcached: the memory and database
stores are the only ones, and a store of the app's own can't be
plugged in.
- **Tags**: forgetting a group of keys works by prefix only.
- **Fragment caching in templates**: cache rendered text with `fetch`
in the action instead.
- **Keys that follow associations**: saving a book doesn't change its
author's key.
- **Caching whole responses**, or setting `Cache-Control` for shared
caches and CDNs, other than with a header of your own.
- **Skipping the action's work for a fresh request**: `fresh_when` and
the `ETag` middleware both decide after the response is rendered.