tiny-proxy

A single-binary Go service that proxies and caches OCI/Docker images, Maven artifacts, GitHub and Gitea release downloads, and Debian/APT and Pacman package mirrors through named virtual repositories, each with its own upstream, upstream credentials, egress proxy (HTTP/SOCKS5), client auth mode, and cache TTLs. Cached objects live on the local filesystem or in S3 (or any S3-compatible store, e.g. MinIO).

Run

go build -o tiny-proxy .
cp config.example.yaml config.yaml   # edit to taste
./tiny-proxy -config config.yaml

Usage

Docker / OCI

The first path segment after /v2/ names the virtual repo:

docker pull localhost:8080/dockerhub/library/nginx:1.27

Note: Docker treats plain HTTP registries as insecure — add the proxy host to insecure-registries in daemon.json, or put the proxy behind TLS.

Maven

GET https://localhost:8080/maven/central/com/google/guava/guava/33.0.0-jre/guava-33.0.0-jre.jar

In settings.xml:

<mirror>
  <id>tiny-proxy</id>
  <mirrorOf>central</mirrorOf>
  <url>http://localhost:8080/maven/central</url>
</mirror>

GitHub releases

Paths are /github/<vrepo>/<owner>/<repo>/<selector> where selector is a tag or an alias — latest (latest stable release) or snapshot (most recent prerelease):

request	response
/github/gh/golang/go/latest	302 → the resolved tag (or content in place when redirect: false)
/github/gh/golang/go/v1.2.3.json	release metadata: the GitHub API release object with each asset's browser_download_url rewritten to this proxy
/github/gh/golang/go/latest.json	same, resolved via the alias
/github/gh/golang/go/v1.2.3/app.zip	asset download, cached forever
/github/gh/golang/go/latest/app.zip	302 → the asset under the resolved tag

curl -s http://localhost:8080/github/gh/golang/go/latest.json | jq -r '.tag_name'
curl -LO http://localhost:8080/github/gh/golang/go/latest/go1.23.4.linux-amd64.tar.gz

• Metadata and alias resolutions are cached for cache.metadata_ttl; assets are immutable and cached forever. Aliases share the tag's cache entries.
• allow: ["myorg/*", "foo/bar"] restricts which owner/repo pairs may be proxied (path.Match globs). Omit it to allow everything.
• redirect: false serves alias URLs in place instead of 302-redirecting to the resolved tag (cache keys are shared either way).
• Set upstream.username/password to a GitHub user + PAT for private repos and to avoid the unauthenticated API rate limit (60 req/h/IP). For GitHub Enterprise point upstream.url at your instance (API <url>/api/v3).
• Tags containing / work in the .json and asset forms; latest and snapshot are reserved names.
• Assets whose names end in .json work too: the proxy tries the tag metadata first and serves the asset form when no such tag exists.
• browser_download_url values in .json metadata are root-relative paths on this proxy (e.g. /github/gh/o/r/v1.2.3/app.zip), not absolute URLs.
• Assets are cached forever, matching the immutable-artifact assumption of the other modules; if an upstream asset is re-uploaded under the same name, delete the key under data/github/… to refresh.

Gitea releases

Identical to the GitHub type — same URL grammar under /gitea/<vrepo>/... (aliases, .json metadata with rewritten asset URLs, immutable asset cache). Point upstream.url at your Gitea instance; its API is served at <url>/api/v1. There is no default upstream. Set upstream.username/password to a user + access token for private repos.

curl -s http://localhost:8080/gitea/gt/myorg/mytool/latest.json | jq -r '.tag_name'
curl -LO http://localhost:8080/gitea/gt/myorg/mytool/latest/mytool_linux-amd64

Debian / APT

A pass-through cache for APT repositories (Debian, Ubuntu, or any mirror):

GET http://localhost:8080/debian/deb/dists/bookworm/Release
GET http://localhost:8080/debian/deb/pool/main/h/hello/hello_1.0-1_amd64.deb

In sources.list:

deb http://localhost:8080/debian/deb bookworm main contrib non-free-firmware

The UI page for a debian vrepo (/ui/debian/<vrepo>/) generates these lines for you: it probes the upstream mirror's dists/ index and each suite's Release file, then lists one ready-to-paste deb line per distro version with that suite's real component list, plus the architectures each version carries. Works for any APT archive — Debian's bookworm/trixie/sid, Ubuntu's noble/jammy, *-updates, *-backports and friends. The probe result is memoized per vrepo for cache.metadata_ttl (failures for a minute), so browsing the page doesn't re-hit the mirror; Release files are fetched in a bounded worker pool. When the mirror is unreachable the page falls back to listing the dists already in the cache.

• Metadata under dists/** (Release, InRelease, Packages., Sources., Contents-, Translation-) is cached for cache.metadata_ttl; anything under pool/ or by-hash/ is content-addressed and cached forever. This keeps apt update correct for by-hash-capable clients (apt 1.2+).
• Signatures need no special handling: InRelease and Release.gpg are passed through byte-identical, so apt verifies them against the distro's regular keyring. The proxy neither strips nor re-signs anything, and no key needs to be installed on clients.
• All architectures a suite publishes are served (binary-amd64, binary-arm64, …); pin one with deb [arch=amd64] <uri> <suite> …. deb-src lines work the same way.
• Multiple upstreams may be configured as mirrors: a 404, 401, 403, 429 or connection error falls through to the next one.
• If-Modified-Since revalidation is not forwarded; metadata_ttl is the freshness knob. Keep it short (minutes) so InRelease and Packages.* don't drift stale-inconsistent.

Pacman

A pass-through cache for Arch Linux package mirrors:

GET http://localhost:8080/pacman/arch/core/os/x86_64/core.db
GET http://localhost:8080/pacman/arch/core/os/x86_64/linux-6.10.2-1-x86_64.pkg.tar.zst

In pacman.conf:

[core]
Server = http://localhost:8080/pacman/arch/$repo/os/$arch

• Repo databases (<repo>.db, <repo>.files, in any compression form, plus their .sig and the lastupdate stamp) are cached for cache.metadata_ttl; packages (*.pkg.tar.* and signatures) are immutable and cached forever.
• Multiple upstreams may be configured as mirrors with 404/401/403/429/error fall-through.
• pacman has no native HTTP-auth support; run pacman vrepos in public mode on a trusted network, or behind an auth-injecting local mirror.

Configuration

See config.example.yaml. Per virtual repo:

field	meaning
type	docker, maven, github, gitea, debian or pacman
upstream.url	registry / repository base URL (github: defaults to https://github.com; gitea: required, API at <url>/api/v1)
upstream.username / password	optional upstream credentials. For Bearer-challenge registries (Docker Hub, GHCR) the proxy performs the token exchange automatically.
upstream.proxy	optional egress proxy: http://, https:// or socks5://[user:pass@]host:port
upstreams	maven, debian, pacman only: ordered fallback list, replacing upstream. Each entry takes the same url/username/password/proxy fields plus (maven only) filters below. A 404, 401, 403, 429 or connection error falls through to the next upstream; any other status is returned as-is.
upstreams[].groups	maven only: wildcard group allowlist, e.g. com.google.* (* crosses dots; bare com.google also covers the subtree). Empty = all groups.
upstreams[].artifacts	maven only: wildcard artifact-name allowlist, e.g. my-lib-*. Empty = all artifacts.
upstreams[].snapshots_only	maven only: only -SNAPSHOT paths are requested from this upstream.
upstream.cache / upstreams[].cache	true (default) stores what this upstream serves; false proxies it through without writing to storage. Use it when the upstream already caches for you — typically another vrepo of this proxy. Reads are unaffected: anything already cached under the vrepo is still served from storage.
upstream.retry.attempts / upstreams[].retry.attempts	retries for this upstream, before the request falls through to the next one. Covers a 429 Too Many Requests, a connection that fails before any response arrives, and a transfer that dies mid-body (resumed with a byte range). Default 2; 0 disables retrying.
upstream.retry.max_wait	cap on a single wait between those retries, a Retry-After sent by the upstream included. Default 5s.
auth.mode	public (no auth), password (any login + this password), users (map of login: password)
cache.manifest_ttl / metadata_ttl	how long tags / maven-metadata.xml / SNAPSHOTs / release metadata / APT and pacman indexes count as fresh. Blobs and released artifacts are immutable and cached forever. 0 = always revalidate.
allow	github, gitea only: glob allowlist of owner/repo pairs; empty = allow all
redirect	github, gitea only: true (default) 302s aliases to the resolved tag; false serves content under the alias URL

Retries, rate limits and interrupted transfers

Upstreams fail in three ways worth retrying, and retry.attempts / retry.max_wait govern all three.

Rate limits (429). Upstreams throttle busy clients — Received status code 429 from server: Too Many Requests in a Maven or Gradle build. The proxy absorbs it in two steps:

1. The upstream client retries the request retry.attempts times, waiting the upstream's Retry-After (or an exponential backoff with jitter when it sends none), capped at retry.max_wait per wait.
2. If it is still throttled, the request falls through to the next upstreams entry, exactly like a 404.

A 429 is only returned to the client when no upstream could serve the path. That answer wins over a 404 from another mirror — a throttled upstream may well hold the artifact, so reporting it missing would make the build fail for the wrong reason — and carries the upstream's Retry-After when it sent one. Nothing about a 429 is cached.

Long throttling windows are not waited out: an upstream asking for minutes is capped to max_wait, the attempts run out, and the next mirror is tried instead of pinning the build's connection open. Retries are logged at warn level (upstream request failed, retrying, with reason=rate_limited).

Connection failures. A request that never produces a response — connection refused or reset, a TLS or DNS failure — is retried the same way (reason=transport_error), then falls through to the next upstream. Only bodyless GET/HEAD requests are retried, so a retry never has to replay a request body. A cancelled request is not retried: the client has already gone.

Interrupted transfers. A connection that dies after the response headers — Error reading from server. Remote end closed connection in apt — is past the point any request-level retry can help, so the proxy resumes it instead: it re-requests the remainder from the same upstream with Range: bytes=<n>- and If-Range, and carries on streaming to the client where it left off. The resumed response is accepted only if it is a 206 whose Content-Range starts exactly where the transfer stopped; a 200 (the upstream ignoring the range, or the file having changed under us) is refused rather than spliced into the middle of an object.

Nothing broken is ever cached. A transfer that ends short of the full object — a read error, or a clean close after fewer bytes than the Content-Length promised — is discarded rather than committed, so a transient network hiccup cannot leave a truncated artifact in the cache. Because released artifacts never expire, a truncated one used to stay truncated forever. As a backstop, any stored object whose size disagrees with its recorded metadata is discarded and refetched (corrupt cache entry, discarding), which heals entries written before this check existed — no manual cleanup needed.

Chaining vrepos

A vrepo may list another vrepo of the same proxy as an upstream — an aggregator in front of a shared cache. Without cache: false both layers store the same bytes; with it the aggregator only routes:

- name: central                    # caches Maven Central
  type: maven
  upstream: { url: https://repo1.maven.org/maven2 }

- name: deps                       # aggregator: routes, does not store
  type: maven
  upstreams:
    - url: https://mvn.example.com
    - url: http://localhost:8080/maven/proxy
      cache: false
    - url: http://localhost:8080/maven/central
      cache: false

Point the chained upstream at this proxy's own base URL; it is an ordinary HTTP request, so a vrepo with auth.mode other than public needs username/password on the upstream entry. Keep the chain shallow — each hop is a real request — and acyclic: a vrepo that reaches itself will recurse until the client times out.

The UI's Upstream column shows each upstream's host and path, marking the ones this vrepo does not cache with (uncached).

Metrics

GET /metrics serves Prometheus text exposition. Everything a request can be blamed on carries a label: vrepo for what the client asked for, vrepo plus upstream (the URL as configured) for what the proxy had to fetch.

series	meaning
tinyproxy_requests_total{type,vrepo}	client requests per proxy type and repo. An empty vrepo is a request that reached no repo: unknown name, bad path, or auth refused
tinyproxy_request_duration_seconds{type,vrepo}	histogram of time spent serving a request in a vrepo — a cache hit and an upstream fetch both land here
tinyproxy_cache_hits_total{type,vrepo} / _misses_total / _errors_total	cache lookups served from storage, fetched from upstream, and failed
tinyproxy_transfer_failures_total{type,vrepo}	transfers that ended short of the full object and were discarded instead of cached
tinyproxy_upstream_requests_total{vrepo,upstream}	upstream attempts, counted per wire attempt — a retried 429, a token exchange and a resumed range request each count
tinyproxy_upstream_responses_total{vrepo,upstream,status}	upstream responses by status code
tinyproxy_upstream_errors_total{vrepo,upstream}	attempts that never produced a response (refused, reset, TLS or DNS failure)
tinyproxy_upstream_received_bytes_total{vrepo,upstream} / _sent_bytes_total	body bytes read from and written to upstreams
tinyproxy_upstream_wire_received_bytes_total{vrepo,upstream} / _wire_sent_bytes_total	socket bytes: headers, chunked framing, TLS records and handshakes included
tinyproxy_upstream_connect_seconds{vrepo,upstream}	histogram of dial time, DNS included
tinyproxy_upstream_tls_handshake_seconds{vrepo,upstream}	histogram of TLS handshake time
tinyproxy_upstream_ttfb_seconds{vrepo,upstream}	histogram of the wait between sending a request and the first response byte — the upstream's own think time
tinyproxy_upstream_body_seconds{vrepo,upstream}	histogram of body streaming time, from first byte to EOF or close
tinyproxy_upstream_retries_total{reason}	retried attempts, by rate_limited or transport_error
tinyproxy_upstream_resumes_total{result}	transfers that died mid-body and were re-requested with a byte range, by ok or failed
tinyproxy_client_requests_total	client requests across all endpoints, /ui/ and /metrics included
tinyproxy_client_received_bytes_total / _sent_bytes_total	body bytes read from and written to clients
tinyproxy_client_wire_received_bytes_total / _wire_sent_bytes_total	socket bytes exchanged with clients, headers and framing included
tinyproxy_cache_corrupt_entries_total	stored objects found to disagree with their metadata; discarded and refetched
tinyproxy_storage_items / tinyproxy_storage_bytes	per-vrepo gauges, collected on scrape via one storage List per vrepo

Plus Go runtime stats (tinyproxy_uptime_seconds, goroutines, memstats).

Body vs wire bytes. The body counters are payload — what the artifact weighs. The wire counters are taken at the socket and are the real traffic: request lines, headers, chunked framing and, upstream, the TLS records and handshake. Upstream sockets belong to exactly one vrepo/upstream pair, so those totals are labelled; client sockets are not, because a keep-alive connection carries requests for several vrepos — so the client wire counters are process-wide rather than split on a guess.

Reading the timings. The connection histograms only record when a connection is actually opened, so a quiet connect_seconds beside a busy upstream_requests_total means the pool is doing its job. ttfb_seconds is the upstream thinking; body_seconds is the link. A rise in the first is the upstream's problem, a rise in the second is usually yours.

Two ratios earn their keep. tinyproxy_upstream_requests_total over tinyproxy_client_requests_total rises when an upstream starts throttling, flapping or truncating transfers. And in a fallback list (maven, debian, pacman) the status="404" share of tinyproxy_upstream_responses_total shows how much of the walk each mirror is absorbing — a mirror that answers nothing but 404 is costing a round trip on every miss and belongs later in the list.

metrics.token protects the endpoint — empty (the default) means open; when set, clients must send Authorization: Bearer <token> or a ?token=<token> query param. Use a 15-30s+ scrape interval: each scrape lists every vrepo's storage prefix, and S3 List paginates the whole prefix.

How it works

• Read-only pull-through: GET/HEAD only; pushes get 405.
• Cache miss → stream from upstream while tee-ing into storage (write-through, single-flight per key). Hit → served from storage.
• storage.type: none runs the proxy without any cache: every request goes upstream, nothing is stored, and the UI and storage gauges report empty. upstream.cache: false is the same idea scoped to one upstream.
• Docker image names are normalized for Docker Hub (nginx → library/nginx).
• Docker clients authenticate with plain Basic auth (supported natively by docker login); the Bearer token dance happens only between the proxy and the upstream.
• /healthz answers liveness probes; /metrics exposes Prometheus metrics.
• Every request gets a trace id, carried on every log line it causes and passed to the upstreams it fans out to. See Logging.

Logging

Each inbound request is given a trace id — adopted from the caller's X-Request-Id or W3C traceparent when it sent one, minted otherwise — that is echoed back in X-Request-Id, attached to every log record the request produces, and forwarded to each upstream it calls. Grepping one id shows the whole life of a request:

INFO upstream request trace=9f2a1c8e40b7d5e3 vrepo=debian-security url=https://deb.debian.org/debian-security/... status=200 dur_ms=41
WARN upstream transfer interrupted, resuming trace=9f2a1c8e40b7d5e3 key=debian/debian-security/pool/... offset=131072 attempts_left=1
INFO upstream request trace=9f2a1c8e40b7d5e3 vrepo=debian-security url=https://deb.debian.org/debian-security/... status=206 dur_ms=38
INFO request done trace=9f2a1c8e40b7d5e3 method=GET path=/debian/debian-security/pool/.../libblkid-dev_2.41.5-0+deb13u1_arm64.deb status=200 bytes=289456 dur_ms=402

Because chained tiny-proxy hops adopt each other's X-Request-Id, one id follows a request across all of them.

field	meaning
log.level	debug, info, warn or error. Default info, which already gives one line per finished client request and one per upstream attempt. debug adds a line when each request arrives, which only matters for requests that never finish.
log.format	text (default) or json
log.requests	inbound access log. Default true; false keeps the trace ids and metrics but drops the lines.
log.upstream	per-upstream request log. Default true.

Out of scope

Pushes/deploys, HTTPS termination (use a reverse proxy), cache eviction/GC, admin API.