tiny-proxy

A single-binary Go service that proxies and caches OCI/Docker images, Maven artifacts, GitHub and Gitea release downloads, and Debian/APT and Pacman package mirrors through named virtual repositories, each with its own upstream, upstream credentials, egress proxy (HTTP/SOCKS5), client auth mode, and cache TTLs. Cached objects live on the local filesystem or in S3 (or any S3-compatible store, e.g. MinIO).

Run

go build -o tiny-proxy .
cp config.example.yaml config.yaml   # edit to taste
./tiny-proxy -config config.yaml

Usage

Docker / OCI

The first path segment after /v2/ names the virtual repo:

docker pull localhost:8080/dockerhub/library/nginx:1.27

Note: Docker treats plain HTTP registries as insecure — add the proxy host to insecure-registries in daemon.json, or put the proxy behind TLS.

• Tags are cached for cache.manifest_ttl; digest-addressed manifests and blobs are immutable and cached forever.
• Multiple upstreams may be configured as an ordered fallback list: a 404, 401, 403, 429, 502, 503, 504 or connection error falls through to the next one. Typical use is another tiny-proxy first and the registry itself second — see Chaining vrepos.
• An upstream URL whose path starts with /v2/ names a repository root, not a registry: the image path is appended to it as-is. https://proxy.example/v2/dockerhub fetches https://proxy.example/v2/dockerhub/nginx/manifests/1.27. Any other URL gets /v2/ added, as before.
• Single-component names (nginx) become library/nginx only when the upstream host is Docker Hub (docker.io / docker.com); a chained proxy does that on its own side.

Maven

GET https://localhost:8080/maven/central/com/google/guava/guava/33.0.0-jre/guava-33.0.0-jre.jar

In settings.xml:

<mirror>
  <id>tiny-proxy</id>
  <mirrorOf>central</mirrorOf>
  <url>http://localhost:8080/maven/central</url>
</mirror>

GitHub releases

Paths are /github/<vrepo>/<owner>/<repo>/<selector> where selector is a tag or an alias — latest (latest stable release) or snapshot (most recent prerelease):

request	response
/github/gh/golang/go/latest	302 → the resolved tag (or content in place when redirect: false)
/github/gh/golang/go/v1.2.3.json	release metadata: the GitHub API release object with each asset's browser_download_url rewritten to this proxy
/github/gh/golang/go/latest.json	same, resolved via the alias
/github/gh/golang/go/v1.2.3/app.zip	asset download, cached forever
/github/gh/golang/go/latest/app.zip	302 → the asset under the resolved tag

curl -s http://localhost:8080/github/gh/golang/go/latest.json | jq -r '.tag_name'
curl -LO http://localhost:8080/github/gh/golang/go/latest/go1.23.4.linux-amd64.tar.gz

• Metadata and alias resolutions are cached for cache.metadata_ttl; assets are immutable and cached forever. Aliases share the tag's cache entries.
• allow: ["myorg/*", "foo/bar"] restricts which owner/repo pairs may be proxied (path.Match globs). Omit it to allow everything.
• redirect: false serves alias URLs in place instead of 302-redirecting to the resolved tag (cache keys are shared either way).
• Set upstream.username/password to a GitHub user + PAT for private repos and to avoid the unauthenticated API rate limit (60 req/h/IP). For GitHub Enterprise point upstream.url at your instance (API <url>/api/v3).
• Tags containing / work in the .json and asset forms; latest and snapshot are reserved names.
• Assets whose names end in .json work too: the proxy tries the tag metadata first and serves the asset form when no such tag exists.
• browser_download_url values in .json metadata are root-relative paths on this proxy (e.g. /github/gh/o/r/v1.2.3/app.zip), not absolute URLs.
• Assets are cached forever, matching the immutable-artifact assumption of the other modules; if an upstream asset is re-uploaded under the same name, delete the key under data/github/… to refresh.

Gitea releases

Identical to the GitHub type — same URL grammar under /gitea/<vrepo>/... (aliases, .json metadata with rewritten asset URLs, immutable asset cache). Point upstream.url at your Gitea instance; its API is served at <url>/api/v1. There is no default upstream. Set upstream.username/password to a user + access token for private repos.

curl -s http://localhost:8080/gitea/gt/myorg/mytool/latest.json | jq -r '.tag_name'
curl -LO http://localhost:8080/gitea/gt/myorg/mytool/latest/mytool_linux-amd64

Debian / APT

A pass-through cache for APT repositories (Debian, Ubuntu, or any mirror):

GET http://localhost:8080/debian/deb/dists/bookworm/Release
GET http://localhost:8080/debian/deb/pool/main/h/hello/hello_1.0-1_amd64.deb

In sources.list:

deb http://localhost:8080/debian/deb bookworm main contrib non-free-firmware

The UI page for a debian vrepo (/ui/debian/<vrepo>/) generates these lines for you: it probes the upstream mirror's dists/ index and each suite's Release file, then lists one ready-to-paste deb line per distro version with that suite's real component list, plus the architectures each version carries. Works for any APT archive — Debian's bookworm/trixie/sid, Ubuntu's noble/jammy, *-updates, *-backports and friends. The probe result is memoized per vrepo for cache.metadata_ttl (failures for a minute), so browsing the page doesn't re-hit the mirror; Release files are fetched in a bounded worker pool. When the mirror is unreachable the page falls back to listing the dists already in the cache.

• Metadata under dists/** (Release, InRelease, Packages., Sources., Contents-, Translation-) is cached for cache.metadata_ttl; anything under pool/ or by-hash/ is content-addressed and cached forever. This keeps apt update correct for by-hash-capable clients (apt 1.2+).
• Signatures need no special handling: InRelease and Release.gpg are passed through byte-identical, so apt verifies them against the distro's regular keyring. The proxy neither strips nor re-signs anything, and no key needs to be installed on clients.
• All architectures a suite publishes are served (binary-amd64, binary-arm64, …); pin one with deb [arch=amd64] <uri> <suite> …. deb-src lines work the same way.
• Multiple upstreams may be configured as mirrors: a 404, 401, 403, 429, 502, 503, 504 or connection error falls through to the next one.
• If-Modified-Since revalidation is not forwarded; metadata_ttl is the freshness knob. Keep it short (minutes) so InRelease and Packages.* don't drift stale-inconsistent.

Pacman

A pass-through cache for Arch Linux package mirrors:

GET http://localhost:8080/pacman/arch/core/os/x86_64/core.db
GET http://localhost:8080/pacman/arch/core/os/x86_64/linux-6.10.2-1-x86_64.pkg.tar.zst

In pacman.conf:

[core]
Server = http://localhost:8080/pacman/arch/$repo/os/$arch

• Repo databases (<repo>.db, <repo>.files, in any compression form, plus their .sig and the lastupdate stamp) are cached for cache.metadata_ttl; packages (*.pkg.tar.* and signatures) are immutable and cached forever.
• Multiple upstreams may be configured as mirrors with 404/401/403/429/502/503/504/error fall-through.
• pacman has no native HTTP-auth support; run pacman vrepos in public mode on a trusted network, or behind an auth-injecting local mirror.

Go modules

A Go module proxy (the GOPROXY protocol) whose upstreams are ordered routing rules by module path. A rule either relays to another module proxy, or builds the module itself from a git repository — through the Gitea or GitLab REST API, or over the git smart-HTTP protocol for any other host. No git binary is involved.

export GOPROXY=http://localhost:8080/go/go
export GONOSUMDB=go.example.com     # modules built from private repos
go get go.example.com/team/lib@latest

- name: go
  type: go
  vanity_hosts: [go.example.com]
  upstreams:
    - kind: gitea                       # goproxy (default) | gitea | gitlab | git
      modules: ["go.example.com/team/*"]
      url: https://gitea.example.com
      repo: "Team/{1}"                  # {n} = n-th glob element captured
      password: <read:repository token>
    - url: https://proxy.golang.org     # no modules: every module

• modules uses GOPRIVATE syntax: each element is a glob matched against the leading elements of the module path. The matched prefix is the repository, repo names it under url, and the rest of the module path is the module's directory in the repo. go.example.com/team/lib/sub/v2 above is the /v2 module in sub/ (or sub/v2/) of Team/lib.
• Git-backed rules follow the go command's own rules: release tags are <dir>/vX.Y.Z, /vN modules live in a vN/ major subdirectory or the directory itself, v2+ tags without a go.mod are +incompatible, and zips leave out nested modules and vendor/. A branch or commit query (@main, @abc1234) gets a vX.0.0-<time>-<hash> pseudo-version, or the release tagged at that commit. go.mod is served as committed, so a fork that keeps its upstream module path works through a replace directive.
• Auth per rule: gitea takes username + password (Basic) or a bare password token; gitlab sends password as a Bearer token; git sends Basic, with username defaulting to git.
• Rules are tried in order. A git-backed rule falls through when the repo or version does not exist; a goproxy rule also on 401/403/410/429 and gateway errors. Versioned .info/.mod/.zip are cached forever; @v/list, @latest and branch queries for cache.metadata_ttl.
• vanity_hosts: ?go-get=1 lookups for a matching module path on one of these hosts get a go-import meta tag pointing the go command at this proxy (go_get: mod, default) or at the repository (go_get: git), so GOPRIVATE users without GOPROXY set still resolve the import path.
• Pseudo-versions always use a vX.0.0 base: the forge APIs and depth-1 fetches do not give the commit graph needed to find the nearest tag. Via a proxy the go command accepts this. Only HTTPS repositories are supported (no SSH).

Configuration

See config.example.yaml. Per virtual repo:

field	meaning
type	docker, maven, github, gitea, debian, pacman or go
upstream.url	registry / repository base URL (github: defaults to https://github.com; gitea: required, API at <url>/api/v1)
upstream.username / password	optional upstream credentials. For Bearer-challenge registries (Docker Hub, GHCR) the proxy performs the token exchange automatically.
upstream.proxy	optional egress proxy: http://, https:// or socks5://[user:pass@]host:port
upstreams	docker, maven, debian, pacman, go (not github/gitea): ordered fallback list, replacing upstream. Each entry takes the same url/username/password/proxy fields plus (maven only) filters below. A 404, 401, 403, 429, 502, 503, 504 or connection error falls through to the next upstream; any other status (a 500 included) is returned as-is, and so is whatever the last upstream answers. For docker, a url whose path starts with /v2/ is a repository root (another tiny-proxy's vrepo) and gets no second /v2/.
upstreams[].groups	maven only: wildcard group allowlist, e.g. com.google.* (* crosses dots; bare com.google also covers the subtree). Empty = all groups.
upstreams[].artifacts	maven only: wildcard artifact-name allowlist, e.g. my-lib-*. Empty = all artifacts.
upstreams[].snapshots_only	maven only: only -SNAPSHOT paths are requested from this upstream.
upstream.cache / upstreams[].cache	true (default) stores what this upstream serves; false proxies it through without writing to storage. Use it when the upstream already caches for you — typically another vrepo of this proxy. Reads are unaffected: anything already cached under the vrepo is still served from storage.
upstream.retry.attempts / upstreams[].retry.attempts	retries for this upstream, before the request falls through to the next one. Covers a 429 Too Many Requests, a connection that fails before any response arrives, and a transfer that dies mid-body (resumed with a byte range). Default 2; 0 disables retrying.
upstream.retry.max_wait	cap on a single wait between those retries, a Retry-After sent by the upstream included. Default 5s.
auth.mode	public (no auth), password (any login + this password), users (map of login: password)
cache.manifest_ttl / metadata_ttl	how long tags / maven-metadata.xml / SNAPSHOTs / release metadata / APT and pacman indexes count as fresh. Blobs and released artifacts are immutable and cached forever. 0 = always revalidate.
cache.not_found_ttl	maven only: how long a path that every upstream answered 404 for is answered 404 from memory, without asking them again. Only definite misses count — not when an upstream was rate limited, down or unreachable along the way — and for maven-metadata.xml/SNAPSHOT paths it never outlasts metadata_ttl. HEAD misses are fetched with GET and stored (the client gets the headers at once; the body lands in the background, at most 16 at a time — past that a HEAD is answered by a HEAD), so a HEAD does not repeat the walk either — except through a cache: false upstream, which gets the HEAD. A 404 answered from memory carries X-Tinyproxy-Miss-Ttl (seconds left), and a tiny-proxy chained in front remembers it no longer than that. Answers from memory are counted in tinyproxy_cache_remembered_misses_total. Default 10m; negative disables.
upstreams[].kind	go only: goproxy (default), gitea, gitlab or git — see Go modules
upstreams[].modules	go only: GOPRIVATE-style module path patterns this rule serves. Empty = every module (goproxy rules only).
upstreams[].repo	go only, git-backed kinds: repository path under url, {n} = n-th captured glob element. Defaults to all captures joined by /.
upstreams[].go_get	go only: vanity answer, mod (default, point at this proxy) or git (point at the repo)
vanity_hosts	go only: hosts whose ?go-get=1 lookups this vrepo answers
allow	github, gitea only: glob allowlist of owner/repo pairs; empty = allow all
redirect	github, gitea only: true (default) 302s aliases to the resolved tag; false serves content under the alias URL

Retries, rate limits and interrupted transfers

Upstreams fail in three ways worth retrying, and retry.attempts / retry.max_wait govern all three.

Rate limits (429). Upstreams throttle busy clients — Received status code 429 from server: Too Many Requests in a Maven or Gradle build. The proxy absorbs it in two steps:

1. The upstream client retries the request retry.attempts times, waiting the upstream's Retry-After (or an exponential backoff with jitter when it sends none), capped at retry.max_wait per wait.
2. If it is still throttled, the request falls through to the next upstreams entry, exactly like a 404.
3. (maven) The throttled upstream is then benched: for its Retry-After (clamped to 10s–5m; 30s when it sent none) it is skipped outright for every path, in every vrepo that lists the same URL through the same egress and account, instead of being asked — and throttling harder — about each one. While benched it still counts as rate limited, so a path nobody else serves is answered 429, not 404. Skips are counted in tinyproxy_upstream_cooldown_skips_total.

A chained tiny-proxy (another instance, or a vrepo of this one) marks what it relays with X-Tinyproxy-Answer: relayed on a 429 that is about its own upstreams — so the tier in front does not bench it, and keeps getting what it already has cached — and uncertain on a 404 given while one of its upstreams could not answer, so the tier in front does not remember it as a miss.

With several mirrors of the same host (Maven Central and a mirror of it) retry.attempts: 0 on the throttled one is usually the better choice: the mirror answers at once, and every retry spends more of the quota. The same goes for a chained upstream (another tiny-proxy, or localhost), which already retries and falls through on its own — retrying it reruns its whole walk.

A 429 is only returned to the client when no upstream could serve the path. That answer wins over a 404 from another mirror — a throttled upstream may well hold the artifact, so reporting it missing would make the build fail for the wrong reason — and carries the upstream's Retry-After when it sent one. Nothing about a 429 is cached.

Long throttling windows are not waited out: an upstream asking for minutes is capped to max_wait, the attempts run out, and the next mirror is tried instead of pinning the build's connection open. Retries are logged at warn level (upstream request failed, retrying, with reason=rate_limited).

Connection failures. A request that never produces a response — connection refused or reset, a TLS or DNS failure — is retried the same way (reason=transport_error), then falls through to the next upstream. Only bodyless GET/HEAD requests are retried, so a retry never has to replay a request body. A cancelled request is not retried: the client has already gone.

Gateway errors (502, 503, 504). These mean the upstream could not answer, not that the object is bad: an ingress in front of a chained tiny-proxy that is down, or a tiny-proxy whose own upstream failed (it answers 502 upstream error). They are not retried, but fall through to the next upstream like a 404. A plain 500 is the upstream's own answer and is returned as-is, as is a gateway error from the last upstream in the list.

Interrupted transfers. A connection that dies after the response headers — Error reading from server. Remote end closed connection in apt — is past the point any request-level retry can help, so the proxy resumes it instead: it re-requests the remainder from the same upstream with Range: bytes=<n>- and If-Range, and carries on streaming to the client where it left off. The resumed response is accepted only if it is a 206 whose Content-Range starts exactly where the transfer stopped; a 200 (the upstream ignoring the range, or the file having changed under us) is refused rather than spliced into the middle of an object.

Nothing broken is ever cached. A transfer that ends short of the full object — a read error, or a clean close after fewer bytes than the Content-Length promised — is discarded rather than committed, so a transient network hiccup cannot leave a truncated artifact in the cache. Because released artifacts never expire, a truncated one used to stay truncated forever. As a backstop, any stored object whose size disagrees with its recorded metadata is discarded and refetched (corrupt cache entry, discarding), which heals entries written before this check existed — no manual cleanup needed.

Chaining vrepos

A vrepo may list another vrepo of the same proxy as an upstream — an aggregator in front of a shared cache. Without cache: false both layers store the same bytes; with it the aggregator only routes:

- name: central                    # caches Maven Central
  type: maven
  upstream: { url: https://repo1.maven.org/maven2 }

- name: deps                       # aggregator: routes, does not store
  type: maven
  upstreams:
    - url: https://mvn.example.com
    - url: http://localhost:8080/maven/proxy
      cache: false
    - url: http://localhost:8080/maven/central
      cache: false

Point the chained upstream at this proxy's own base URL; it is an ordinary HTTP request, so a vrepo with auth.mode other than public needs username/password on the upstream entry. Keep the chain shallow — each hop is a real request — and acyclic: a vrepo that reaches itself will recurse until the client times out.

The same works across instances: a home proxy can put a remote tiny-proxy first and the real upstreams second, so it serves from the remote cache when that is reachable and goes direct when it is not. The fall-through on 502/503/504 and connection errors is what makes the remote optional. For docker, point the upstream at the remote vrepo's /v2/<vrepo> root:

- name: dockerhub
  type: docker
  upstreams:
    - url: https://proxy.example/v2/dockerhub   # remote tiny-proxy vrepo
      retry: { attempts: 0 }                    # down? go direct at once
    - url: https://registry-1.docker.io

retry: { attempts: 0 } stops a dead remote from costing each request the connection-failure retries before the direct upstream is tried. Leave cache on when the remote is across a WAN link and the local copy is the point; set cache: false when both sit on the same network.

The UI's Upstream column shows each upstream's host and path, marking the ones this vrepo does not cache with (uncached).

Metrics

GET /metrics serves Prometheus text exposition. Everything a request can be blamed on carries a label: vrepo for what the client asked for, vrepo plus upstream (the URL as configured) for what the proxy had to fetch.

series	meaning
tinyproxy_requests_total{type,vrepo}	client requests per proxy type and repo. An empty vrepo is a request that reached no repo: unknown name, bad path, or auth refused
tinyproxy_request_duration_seconds{type,vrepo}	histogram of time spent serving a request in a vrepo — a cache hit and an upstream fetch both land here
tinyproxy_cache_hits_total{type,vrepo} / _misses_total / _errors_total	cache lookups served from storage, fetched from upstream, and failed
tinyproxy_transfer_failures_total{type,vrepo}	transfers that ended short of the full object and were discarded instead of cached
tinyproxy_upstream_requests_total{vrepo,upstream}	upstream attempts, counted per wire attempt — a retried 429, a token exchange and a resumed range request each count
tinyproxy_upstream_responses_total{vrepo,upstream,status}	upstream responses by status code
tinyproxy_upstream_errors_total{vrepo,upstream}	attempts that never produced a response (refused, reset, TLS or DNS failure)
tinyproxy_upstream_received_bytes_total{vrepo,upstream} / _sent_bytes_total	body bytes read from and written to upstreams
tinyproxy_upstream_wire_received_bytes_total{vrepo,upstream} / _wire_sent_bytes_total	socket bytes: headers, chunked framing, TLS records and handshakes included
tinyproxy_upstream_connect_seconds{vrepo,upstream}	histogram of dial time, DNS included
tinyproxy_upstream_tls_handshake_seconds{vrepo,upstream}	histogram of TLS handshake time
tinyproxy_upstream_ttfb_seconds{vrepo,upstream}	histogram of the wait between sending a request and the first response byte — the upstream's own think time
tinyproxy_upstream_body_seconds{vrepo,upstream}	histogram of body streaming time, from first byte to EOF or close
tinyproxy_upstream_retries_total{reason}	retried attempts, by rate_limited or transport_error
tinyproxy_upstream_resumes_total{result}	transfers that died mid-body and were re-requested with a byte range, by ok or failed
tinyproxy_client_requests_total	client requests across all endpoints, /ui/ and /metrics included
tinyproxy_client_received_bytes_total / _sent_bytes_total	body bytes read from and written to clients
tinyproxy_client_wire_received_bytes_total / _wire_sent_bytes_total	socket bytes exchanged with clients, headers and framing included
tinyproxy_cache_corrupt_entries_total	stored objects found to disagree with their metadata; discarded and refetched
tinyproxy_storage_items / tinyproxy_storage_bytes	per-vrepo gauges, collected on scrape via one storage List per vrepo

Plus Go runtime stats (tinyproxy_uptime_seconds, goroutines, memstats).

Body vs wire bytes. The body counters are payload — what the artifact weighs. The wire counters are taken at the socket and are the real traffic: request lines, headers, chunked framing and, upstream, the TLS records and handshake. Upstream sockets belong to exactly one vrepo/upstream pair, so those totals are labelled; client sockets are not, because a keep-alive connection carries requests for several vrepos — so the client wire counters are process-wide rather than split on a guess.

Reading the timings. The connection histograms only record when a connection is actually opened, so a quiet connect_seconds beside a busy upstream_requests_total means the pool is doing its job. ttfb_seconds is the upstream thinking; body_seconds is the link. A rise in the first is the upstream's problem, a rise in the second is usually yours.

Two ratios earn their keep. tinyproxy_upstream_requests_total over tinyproxy_client_requests_total rises when an upstream starts throttling, flapping or truncating transfers. And in a fallback list (maven, debian, pacman) the status="404" share of tinyproxy_upstream_responses_total shows how much of the walk each mirror is absorbing — a mirror that answers nothing but 404 is costing a round trip on every miss and belongs later in the list.

metrics.token protects the endpoint — empty (the default) means open; when set, clients must send Authorization: Bearer <token> or a ?token=<token> query param. Use a 15-30s+ scrape interval: each scrape lists every vrepo's storage prefix, and S3 List paginates the whole prefix.

How it works

• Read-only pull-through: GET/HEAD only; pushes get 405.
• Cache miss → stream from upstream while tee-ing into storage (write-through, single-flight per key). Hit → served from storage.
• storage.type: none runs the proxy without any cache: every request goes upstream, nothing is stored, and the UI and storage gauges report empty. upstream.cache: false is the same idea scoped to one upstream.
• Docker image names are normalized for Docker Hub (nginx → library/nginx).
• Docker clients authenticate with plain Basic auth (supported natively by docker login); the Bearer token dance happens only between the proxy and the upstream.
• /healthz answers liveness probes; /metrics exposes Prometheus metrics.
• Every request gets a trace id, carried on every log line it causes and passed to the upstreams it fans out to. See Logging.

Logging

Each inbound request is given a trace id — adopted from the caller's X-Request-Id or W3C traceparent when it sent one, minted otherwise — that is echoed back in X-Request-Id, attached to every log record the request produces, and forwarded to each upstream it calls. Grepping one id shows the whole life of a request:

INFO upstream request trace=9f2a1c8e40b7d5e3 vrepo=debian-security url=https://deb.debian.org/debian-security/... status=200 dur_ms=41
WARN upstream transfer interrupted, resuming trace=9f2a1c8e40b7d5e3 key=debian/debian-security/pool/... offset=131072 attempts_left=1
INFO upstream request trace=9f2a1c8e40b7d5e3 vrepo=debian-security url=https://deb.debian.org/debian-security/... status=206 dur_ms=38
INFO request done trace=9f2a1c8e40b7d5e3 method=GET path=/debian/debian-security/pool/.../libblkid-dev_2.41.5-0+deb13u1_arm64.deb status=200 bytes=289456 dur_ms=402

Because chained tiny-proxy hops adopt each other's X-Request-Id, one id follows a request across all of them.

field	meaning
log.level	debug, info, warn or error. Default info, which already gives one line per finished client request and one per upstream attempt. debug adds a line when each request arrives, which only matters for requests that never finish.
log.format	text (default) or json
log.requests	inbound access log. Default true; false keeps the trace ids and metrics but drops the lines.
log.upstream	per-upstream request log. Default true.

Out of scope

Pushes/deploys, HTTPS termination (use a reverse proxy), cache eviction/GC, admin API.