---
title: S3 setup
description: Configure an S3-compatible remote cache and connect kache to it.
---
# S3 setup
kache supports AWS S3 and common S3-compatible storage such as Cloudflare R2,
Ceph, and MinIO. The configuration is the same across providers; adjust the
endpoint and credentials.
For a shared mounted directory instead, see [Filesystem setup](/docs/remote-cache/filesystem-setup).
The kache binary currently compiles only these `s3` and `filesystem` remote
types.
## Minimal configuration
```toml title="~/.config/kache/config.toml"
[cache.remote]
type = "s3"
bucket = "my-build-cache"
```
With just a bucket name and no endpoint, kache uses AWS S3 with the default credential chain (environment variables, `~/.aws/credentials`, IAM role).
<Callout type="warn">
If you omit `region`, kache defaults to `us-east-1` rather than your bucket's actual region. Set `region` to your bucket's region for AWS S3, and to whatever value your provider expects (or `auto`) for S3-compatible endpoints.
</Callout>
## Provider examples
<Tabs items={["AWS S3", "Cloudflare R2", "Ceph / MinIO"]}>
<Tab value="AWS S3">
```toml
[cache.remote]
type = "s3"
bucket = "my-build-cache"
region = "eu-west-1"
profile = "my-aws-profile" # omit to use default profile
```
For CI, prefer IAM roles or environment variables over a stored profile.
</Tab>
<Tab value="Cloudflare R2">
```toml
[cache.remote]
type = "s3"
bucket = "my-build-cache"
endpoint = "https://<account-id>.r2.cloudflarestorage.com"
region = "auto"
```
Set credentials via `KACHE_S3_ACCESS_KEY` and `KACHE_S3_SECRET_KEY` or an AWS profile pointing to R2 API tokens.
</Tab>
<Tab value="Ceph / MinIO">
```toml
[cache.remote]
type = "s3"
bucket = "build-cache"
endpoint = "https://s3.internal.example.com"
profile = "ceph"
```
The `profile` field refers to a named profile in `~/.aws/credentials` or `~/.aws/config`. Region is optional for Ceph but you can set it to any non-empty string if your setup requires it.
kache always uses path-style addressing (`https://endpoint/bucket/key`) for every provider, so self-hosted S3 works without virtual-hosted/bucket-subdomain DNS. This also makes dotted bucket names safe on custom endpoints.
</Tab>
</Tabs>
### macOS and self-hosted LAN endpoints
macOS 15 and newer protects connections to servers reached directly through a
local Wi-Fi or Ethernet interface. When kache runs as an installed LaunchAgent,
allow its **Local Network** prompt; the setting is available later under
**System Settings > Privacy & Security > Local Network**.
If only the installed daemon fails with `No route to host (os error 65)` while
the same endpoint works from `kache daemon run` in a terminal, Local Network
privacy is the likely cause rather than DNS or S3 credentials. Reinstall the
service with the current kache binary so its responsible-code metadata is up to
date. If permission still cannot be granted, use:
```sh
kache daemon uninstall
```
The next terminal build starts the daemon on demand and inherits the terminal's
Local Network permission. Kache logs a specific hint for this failure shape.
## Credential resolution order
When kache needs S3 credentials, it checks these sources in order:
1. `KACHE_S3_ACCESS_KEY` + `KACHE_S3_SECRET_KEY` (explicit env var override)
2. Standard `AWS_ACCESS_KEY_ID` / `AWS_SECRET_ACCESS_KEY` environment credentials
3. The selected shared profile (`KACHE_S3_PROFILE`, `profile`, `AWS_PROFILE`, then `default`)
4. Legacy inline SSO for that profile
5. `credential_process` for that profile
6. Environment-based web identity (`AWS_ROLE_ARN` + `AWS_WEB_IDENTITY_TOKEN_FILE`)
7. ECS/task credentials, then EC2 instance credentials
For CI, explicit environment credentials or IAM roles are the most common. For
local setups with multiple AWS accounts, a named profile keeps credentials
organized.
<Callout type="warn">
Both halves of the explicit pair are required. If only `KACHE_S3_ACCESS_KEY` or only `KACHE_S3_SECRET_KEY` is set, kache logs a warning and falls back to the AWS chain rather than erroring — so a missing half surfaces as confusing "wrong credentials" behavior, not a clear failure.
</Callout>
### Environment overrides
Every remote config field has a matching `KACHE_S3_*` env var, which takes precedence over the config file. This is handy in CI, where setting env vars is easier than shipping a config file:
| Env var | Overrides config field |
| --- | --- |
| `KACHE_S3_BUCKET` | `bucket` |
| `KACHE_S3_ENDPOINT` | `endpoint` |
| `KACHE_S3_REGION` | `region` |
| `KACHE_S3_PREFIX` | `prefix` |
| `KACHE_S3_PROFILE` | `profile` |
| `KACHE_S3_USER_AGENT` | `user_agent` |
| `KACHE_S3_ACCESS_KEY` / `KACHE_S3_SECRET_KEY` | (explicit credentials) |
Existing Kache S3 config keys and defaults remain accepted. A legacy
`[cache.remote]` table without a `type` field and an environment-only setup
rooted in `KACHE_S3_BUCKET` still select S3; the other `KACHE_S3_*` overrides
remain available. `type = "s3"` is the recommended explicit form for new
config files. The AWS SDK-specific migration boundaries below still apply.
<Callout type="warning">
**A background daemon does not inherit these.** All remote I/O happens
daemon-side, and a daemon started automatically by a build outlives that
build and serves every later one on the machine. If it inherited the
starting build's environment, its remote would depend on which build won the
startup race — the failure this prevents logged 2,330 `no remote configured`
errors in six hours on a monorepo whose `KACHE_S3_*` lived in per-checkout
`.cargo/config.toml`, then silently began working after an unrelated restart.
So an auto-started daemon resolves its remote from its watched config file
only, and kache prints a warning when a remote is configured *only* in the
environment. Env overrides still apply to the wrapper and CLI as documented
above, and to a daemon you start yourself:
- **Persistent setup:** put the remote in `[cache.remote]` in the config
file. The daemon watches that file and restarts when it changes.
- **Env-driven setup (CI, service managers):** start the daemon explicitly
from the intended environment with `kache daemon run`, or put the values in
the service definition. That placement is deliberate rather than a race, so
it is honoured.
`kache stats` and `kache doctor` report the remote the daemon actually
resolved, and warn when it differs from yours.
</Callout>
## OpenDAL migration boundaries
The config-file and `KACHE_S3_*` interfaces are retained. Static shared
profiles, legacy inline SSO profiles, `credential_process`, environment-based
web identity, ECS/task credentials, and EC2 instance credentials remain
available. Web identity uses `cache.remote.region` when choosing its STS
endpoint.
<Callout type="warn">
OpenDAL currently signs with SigV4 only. SigV4a / Multi-Region Access Points,
assume-role profiles that rely on `role_arn` with `source_profile` or
`credential_source`, and modern profiles that refer to a separate
`[sso-session ...]` section are not yet compatible. Profile-based web
identity (`role_arn` + `web_identity_token_file` in the profile) is also not
loaded; use its standard environment variables instead. Resolve unsupported
profile credentials before launching kache (standard AWS credential env vars
work), or use `credential_process`. AWS China, ISO, FIPS, and dual-stack
endpoints should also set `endpoint` explicitly.
</Callout>
OpenDAL normalizes object paths, so kache stores objects under a canonical
prefix. A non-canonical `prefix` is normalized rather than rejected: a leading or
trailing slash, surrounding whitespace and `//` are collapsed, and kache logs a
warning when that happens. An empty prefix is valid and stores objects at the
bucket root.
Normalization changes where objects live. `prefix = "team/"` previously wrote
`team//v3/...` and now writes `team/v3/...`, so objects written by an older
kache are no longer found and the remote cache repopulates once. Copy those
objects first if you want to keep them.
Backslashes and `.` / `..` segments are still rejected: there is no safe
normalization for them. A rejected prefix does not fail your build — kache logs
the reason, continues without a remote cache, and reports it in `kache status`.
`AWS_ENDPOINT_URL_S3` remains supported when `endpoint` / `KACHE_S3_ENDPOINT`
is unset. An `endpoint_url` stored only inside an AWS shared profile is not
loaded by OpenDAL; copy it to `cache.remote.endpoint` or
`KACHE_S3_ENDPOINT` during migration.
## S3 bucket layout
Each cached entry is two objects — a packed tarball plus a small JSON manifest used for existence checks and listing:
```
{prefix}/v3/packs/{crate_name}/{cache_key}.tar.zst # the packed artifacts (zstd-compressed tar)
{prefix}/v3/manifests/{crate_name}/{cache_key}.json # small manifest for existence/listing
```
The default prefix is `artifacts`. Organizing by crate name makes filtered listing efficient — `kache sync --pull` issues one `ListObjectsV2` per crate against `{prefix}/v3/manifests/{crate_name}/`, so it only enumerates manifests for crates in your `Cargo.lock` rather than scanning the whole `v3/manifests/` tree. Use `kache sync --pull --all` to list everything.
`kache save-manifest` also writes build manifests under a separate `{prefix}/_manifests/` namespace (and, when a namespace and `Cargo.lock` are present, content-addressed shards under `{prefix}/_manifests/v3/{namespace}/shards/{hash}.json`). Bucket policies that scope by prefix must grant access to `_manifests/` as well as `v3/`.
## Bucket policies
kache needs `s3:GetObject`, `s3:PutObject`, and `s3:ListBucket` on the bucket. For read-only CI runners that pull but don't push, `s3:GetObject` and `s3:ListBucket` are sufficient. If you restrict the policy by prefix, cover both `{prefix}/v3/*` and `{prefix}/_manifests/*` (see [S3 bucket layout](#s3-bucket-layout)).
## Compression
zstd compression applies to the remote `.tar.zst` packs only; the local blob store is kept uncompressed. The default level is `3` — fast to compress and decompress, with reasonable size reduction. Lower levels (1–2) cut compression overhead when network bandwidth matters less than CPU time; higher levels (up to 22) are available but rarely worth it for build artifacts. Values are clamped to the `1`–`22` range.
`KACHE_COMPRESSION_LEVEL` is read when the process that does the compressing starts, so set it on the uploader rather than on a plain `cargo build`:
```sh
KACHE_COMPRESSION_LEVEL=1 kache sync --push # fastest, larger packs
```
<Callout>
The daemon loads its config once at startup, so changing `KACHE_COMPRESSION_LEVEL` for a single `cargo build` does not reconfigure an already-running daemon. Stop and start a manually managed daemon from the intended environment. For an installed service, put the setting in the watched config or service definition; `kache daemon restart` retains the service environment.
</Callout>