kache 1.2.0

Compiler cache for Rust, C/C++ and CUDA, with S3, GCS, shared-filesystem and OCI remotes.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

---
title: Bound a remote
description: Let the store expire old objects; Kache never deletes remotely
---

A remote cache grows until the store removes old objects: a lifecycle rule on S3 or GCS, a retention policy on an OCI registry, or a prune job run by the owner of a shared directory. Kache reads and writes remote objects and never deletes them, so a policy can run at any time without pausing builds.

## What the layout needs from a policy

```text
{prefix}/v3/manifests/{crate}/{key}.json      one manifest ...
{prefix}/v3/packs/{crate}/{key}.tar.zst       ... and the pack it describes
{prefix}/v3/predictions/...                   input prediction rows
{prefix}/_manifests/...                       build manifests and shards
```

Two rules keep the store consistent under expiry:

- Expire `v3/manifests/` and `v3/packs/` with the same age. A manifest and its pack are written together and neither is rewritten, so they age together. A manifest that outlives its pack is a miss; a pack that outlives its manifest is unreachable garbage.
- Expire by age since the object was written. Kache never rewrites an entry's manifest or pack, so a hot entry expires too and is uploaded again on the next miss. Build manifests and prediction rows are written again each time they are used, so they expire only when idle. An expired prediction row costs one dep-info pre-pass. Pick an age longer than the interval at which your builds change.

An expired object is a miss. With `cache.remote_key_listing` on, the daemon's remote key index can report an object present for up to `cache.remote_key_cache_refresh_secs` after it expired; the fetch then misses and the crate compiles as usual.

## S3 lifecycle rules

Thirty days is a reasonable start. Every provider below applies the rule on its own schedule, usually once a day.

**AWS S3**

```json title="lifecycle.json"
{
  "Rules": [
    { "ID": "kache-v3", "Status": "Enabled", "Filter": { "Prefix": "artifacts/v3/" }, "Expiration": { "Days": 30 } },
    { "ID": "kache-manifests", "Status": "Enabled", "Filter": { "Prefix": "artifacts/_manifests/" }, "Expiration": { "Days": 30 } }
  ]
}
```

```bash
aws s3api put-bucket-lifecycle-configuration --bucket my-kache --lifecycle-configuration file://lifecycle.json
```

Replace `artifacts/` with your `cache.remote.prefix`. Add an `AbortIncompleteMultipartUpload` rule if your bucket receives multipart uploads from other tools; Kache uploads each object in one request.

On a versioned bucket, also add a `NoncurrentVersionExpiration` rule. There, `Expiration` only hides the current version behind a delete marker, and every rewrite of a build manifest, shard or prediction row keeps the old version, so no space is freed until noncurrent versions expire.

**Cloudflare R2**

R2 supports object lifecycle rules per prefix with the same expiry-by-age semantics. Create one rule per prefix above from the bucket's settings in the dashboard, or with Wrangler.

**MinIO**

```bash
mc ilm rule add --prefix "artifacts/v3/" --expire-days 30 myminio/my-kache
mc ilm rule add --prefix "artifacts/_manifests/" --expire-days 30 myminio/my-kache
```

## Google Cloud Storage

```json title="lifecycle.json"
{
  "lifecycle": {
    "rule": [
      { "action": { "type": "Delete" },
        "condition": { "age": 30, "matchesPrefix": ["artifacts/v3/", "artifacts/_manifests/"] } }
    ]
  }
}
```

```bash
gcloud storage buckets update gs://my-kache --lifecycle-file=lifecycle.json
```

## OCI registry

Kache never deletes tags, so expire them with the registry's retention policy or a scheduled cleanup job. Every object is its own tag, named from an encoding or a hash of its whole key, so tag patterns cannot reliably tell entry manifests, packs and build manifests apart. Give every `kache-v2-` and `kache-v1-` tag the same age.

## Filesystem remote

The owner of the shared directory runs one prune job on one machine. Age by modification time so the result is the same on `noatime` mounts, and skip the staging directory Kache renames from.

```bash
#!/bin/sh
root=/mnt/shared-kache/artifacts
# v3: manifests and packs in one pass, same age, so pairs go together.
find "$root/v3" "$root/_manifests" -type f -mtime +30 -delete
find "$root" -mindepth 1 -type d -empty -not -path '*/.kache-tmp*' -delete
```

Run it from cron or a systemd timer once a day. The job needs no coordination with writers: a writer that loses a race with the prune uploads again on its next miss, and a reader that loses one records a miss.

GC bounds the local store only. There is no kache command that deletes remote objects.