git-cache-proxy

A small, read-only caching proxy for Git repositories. It sits between many clients (CI build/test machines) and an origin git server, serving clones and fetches from a local bare mirror and pulling only the delta from upstream on each request.

It exists for the case where an elastic fleet of ephemeral machines (e.g. autoscaled CI runners in another region/cloud) repeatedly clones large repos over a slow or expensive link to an on-prem origin. Point the fleet at a proxy running next to them and the bulk history is served locally; only new commits cross the WAN, deduplicated across the whole fleet.

Why not just a mirror or an HTTP cache?

• A generic HTTP cache doesn't work: git fetch is a negotiated pack transfer (POST /git-upload-pack responses are computed per request), so there's nothing static to cache.
• A scheduled full mirror (Gitea pull-mirror, etc.) means enumerating and replicating every repo, and it can lag the exact commit CI just pushed.

This proxy is lazy and pull-only: it caches on first request, and on every request it does an incremental git fetch from upstream before serving - so a client always gets the ref it asked for, and nothing is ever proactively pushed or replicated to where the proxy runs. That last property tends to be the difference between "sure" and "absolutely not" when the origin is sensitive.

How it compares

1. Caches the negotiated git-upload-pack response, not just static objects - a plain HTTP cache can't, because every fetch is computed per request.
2. Caches on first request; no repo enumeration or replication schedule to run.
3. Serves the exact ref asked for. The proxy fetches upstream before serving; a scheduled mirror can lag its refresh interval (lag); --reference is only as fresh as its local seed (seed).
4. Only new objects cross the WAN per request; the bulk history is served from the local mirror.
5. Never writes to or proactively replicates from the origin - it only ever pulls what a client requested.
6. Works against any unmodified Git smart-HTTP origin. GitLab Geo needs GitLab on both ends.

How it works

All git work is delegated to the system git binary, so protocol correctness - protocol v2, shallow and partial (filtered) clones - comes for free.

GET  <repo>/info/refs?service=git-upload-pack
  1. resolve <repo> -> upstream URL + local bare mirror dir
  2. clone --mirror (first time) or fetch --prune (coalesced per repo)
  3. serve `git upload-pack --advertise-refs` from the mirror
POST <repo>/git-upload-pack
  -> stream `git upload-pack --stateless-rpc` from the mirror (packfile)
anything git-receive-pack  -> 403 (read-only)

Concurrent clients for the same repo are serialized so a burst triggers a single upstream fetch; a short TTL coalesces repeated requests.

git-LFS

LFS objects use a different HTTP API from the git protocol, so they are cached separately:

POST <repo>/info/lfs/objects/batch
  -> forward to upstream, then rewrite each object's download URL back to this
     proxy so the object fetch is cached here
GET  <repo>/info/lfs/objects/<oid>
  -> serve from the on-disk cache, or on a miss fetch it from upstream once
     (verify sha256 == oid, store), then serve
anything with operation=upload  -> 403 (read-only)

Objects are content-addressed and immutable, so a cached object is never stale and is shared across every repo that references the same oid; the first fetch across the fleet pays the WAN cost, the rest are served locally. The upstream LFS transfer is in-process over HTTPS (reqwest + rustls, no OpenSSL), so the binary stays statically linked. Cached objects share the --cache-max-mb budget and LRU eviction with the git mirrors. No configuration is needed: LFS is served on the same endpoints the git client already routes through the proxy.

Benchmark

For a fleet of ephemeral clients fetching the same content, only the first fetch pays the WAN cost; the rest are served locally. Over an emulated 20 Mbit/s, 60 ms-RTT link - cloning a 64 MB repo, and fetching a 32 MB git-LFS object:

The first runner sees no penalty and every subsequent runner is served in-region while nothing crosses the WAN; the saving scales with fleet size and link cost. Reproduce or retune (TOTAL_MB, LFS_MB, RATE_MBIT, RTT_MS) with bench/run.sh - see bench/README.md for the method and its caveats.

Install

cargo install git-cache-proxy

cargo install builds from source and needs a git binary on PATH at runtime (all wire-protocol work is delegated to it). Or pull the container image, which bundles git:

docker pull ghcr.io/rolandjitsu/git-cache-proxy

Quick start

git-cache-proxy --upstream https://git.example.com --cache-root /var/cache/git-cache-proxy
# then, on a client:
git -c url."http://proxy:8080/".insteadOf="https://git.example.com/" clone https://git.example.com/group/repo.git

The insteadOf rewrite is how a client transparently routes through the proxy. Pair it with pushInsteadOf back to the origin so pushes bypass the (read-only) proxy:

git config url."http://proxy:8080/".insteadOf            "https://git.example.com/"
git config url."https://git.example.com/".pushInsteadOf  "https://git.example.com/"

Try it

The image on GHCR runs as-is against any public origin, no config file. Point it at GitHub and clone through it:

docker run --rm -p 8080:8080 ghcr.io/rolandjitsu/git-cache-proxy \
  --upstream https://github.com

# in another terminal - the proxy mirrors the repo on the first request and
# serves it from the local mirror thereafter:
git -c url."http://localhost:8080/".insteadOf="https://github.com/" \
  clone https://github.com/rolandjitsu/git-cache-proxy

Configuration

Every flag has an environment-variable equivalent.

Endpoints: /healthz, /readyz, /metrics (Prometheus - per-repo request and upstream counters, cache-size gauges, LFS object hit/miss counters, and *_duration_seconds fetch/serve latency histograms).

Auth model

• Upstream: the proxy holds a single read-only credential (--upstream-auth-header) used only to fetch/clone. It never writes upstream. The value is the full header line and is passed verbatim to git as http.extraHeader. If it is unset/blank the proxy warns at startup and contacts upstream anonymously.
• Clients: anonymous by default (intended for a network-restricted deployment); set --serve-token to require a bearer token.

GitLab with a personal access token

GitLab's git smart-HTTP endpoint authenticates a PAT via HTTP Basic auth (the PAT is the password; any non-empty username works) — not Authorization: Bearer (OAuth2 access tokens only) and not the PRIVATE-TOKEN header (REST API only). Build the header as Basic auth:

export GITCACHEPROXY_UPSTREAM_AUTH_HEADER="Authorization: Basic $(printf 'oauth2:%s' "$GITLAB_TOKEN" | base64 | tr -d '\n')"

Verify it against the git endpoint before wiring up the proxy — a 200 means the header works, a 401 means the scheme is wrong:

curl -sS -o /dev/null -w '%{http_code}\n' \
  -H "$GITCACHEPROXY_UPSTREAM_AUTH_HEADER" \
  "https://gitlab.example.com/group/repo.git/info/refs?service=git-upload-pack"

When passing it into a container, prefer -e GITCACHEPROXY_UPSTREAM_AUTH_HEADER (no =) so an unset variable is dropped with a warning rather than silently forwarded as an empty string.

Security model

The proxy is a shared, credentialed reader, so a few properties are worth making explicit before you expose it:

• Reachability is the trust boundary. The proxy fetches from upstream with its own single credential and serves the result to whoever asked. There is no per-repo authorization: with --serve-token unset it serves anonymously, and when set the token is one shared secret that grants access to everything the upstream credential can read - including repos a given client could not read directly. Restrict who can reach the proxy (private network, security group, mTLS at the ingress) and treat "can reach the port" as "can read every mirrored repo".
• Plain HTTP. The proxy speaks HTTP, so a --serve-token bearer travels in cleartext. Terminate TLS in front of it (reverse proxy / ingress) on any network you do not fully trust. Upstream fetches use whatever scheme the --upstream URL specifies - use https://.
• Defaults are open. It binds 0.0.0.0:8080 and serves anonymously unless --serve-token is set. That is deliberate for a locked-down CI network; do not place it on an untrusted one without a token and TLS.
• DoS knobs. --max-concurrent-requests caps concurrent upstream clone/fetch work and --max-decoded-body-mb bounds request-body memory (defusing a decompression bomb). --cache-max-mb bounds on-disk growth by evicting least-recently-used idle mirrors; it defaults to 0 (unlimited), so set it - or isolate and monitor the cache volume - on an untrusted network.
• Read-only. Only git-upload-pack (clone/fetch) is served; git-receive-pack (push) is refused and upstream is only ever pulled from, never written.

Deploy

The Dockerfile builds a statically linked (musl) binary and drops it onto a minimal Alpine base. Because the git wire protocol is delegated to the system git binary, the runtime image must contain git - so it is Alpine-with-git rather than a fully distroless/FROM scratch image. (The LFS transfer is in-process, so it adds no runtime tool - only CA certs.) Removing the git dependency and enabling a git-free image means moving the git plumbing in-process to a Rust library - see the roadmap below.

On Kubernetes, a Helm chart lives in chart/ (single-writer Deployment, /healthz+/readyz probes, cache PVC, optional Ingress and Prometheus ServiceMonitor):

helm install git-cache-proxy oci://ghcr.io/rolandjitsu/charts/git-cache-proxy \
  --set upstream=https://your-git-host.example.com

See the chart README for the full values reference.

Status / scope

Working and end-to-end tested against both Git wire protocol versions — the modern v2 (git-protocol header, the default since Git 2.26) and the legacy v0/v1 advertisement — covering full clone, incremental delta fetch, and push rejection. git-LFS is cached too: the batch API is proxied and objects are stored content-addressed on disk, covered by the LFS integration tests (miss/fetch/ verify, cache hit, corrupted-object rejection, upload refusal).

This is early, single-maintainer software: no independent review or wide deployment yet. Pin a version and try it against your own setup before you rely on it.

Not yet implemented, in rough priority order:

• A background/scheduled refresh option (today every info/refs triggers an on-demand, TTL-coalesced fetch).
• No external git binary: move the plumbing in-process to a Rust library (gitoxide/git2). More robust (no subprocess/argv surface, structured errors) and unlocks a fully distroless, git-free image - a larger change, tracked as a possible v2.

Contributions welcome.

License

Apache-2.0. Self-contained - no dependencies outside crates.io.