since the Docker Hub registry has a rate limit for Pulls since November 2020 in place, this limit could be triggered if you are experimenting with multiple images or in larger docker environments.
Rate Limits - free tier
- Anonymous pulls: 100/6h
- Authenticated pulls 200/6h
Official Blog Post
Reference
Solution
a solution could be deploying a Docker Registry pull through cache. Every Docker Host after deploying the Registry and configuring the Hosts will pull the Images from a selfhosted Registry Server. If the Registry has the requested Image in cache it serves it from here, otherwise it pulls the image puts in it cache and serves it to the requesting host.
Setup Registry
i have currently 2 swarm Deployments in use.
- Deployment 1: 3 node raspi Cluster with traefik and wildcard cert
- Deployment 2: 3 node mixed Arch Cluster (x86-64, aarch64) with a dedicated 4th single-node as reverse Proxy and wildcard cert
Both deployments are already configured to fetch and use wildcard certificates for internal domains. The Setup of this configuration is not covered in this Post.
Deployment 1 - Registry with Traefik
for Deployment 1 i will leverage Traefik as a Reverse Proxy for the registry itself.
Requirements - Deployment 1
1
2
3
4
|
├── registry
│ ├── cache
│ ├── config
│ └── docker-compose.yml
|
config.yml inside registry/config/
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
|
version: 0.1
log:
level: warn
fields:
service: registry
environment: development
storage:
cache:
blobdescriptor: inmemory
filesystem:
rootdirectory: /var/lib/registry-cache
maintenance:
uploadpurging:
enabled: true
age: 168h
interval: 24h
dryrun: false
http:
addr: :5000
secret: asecretforlocaldevelopment
debug:
addr: localhost:5001
headers:
X-Content-Type-Options: [nosniff]
http2:
disabled: false
redis:
addr: authentik-redis:6379
pool:
maxidle: 16
maxactive: 64
idletimeout: 300s
dialtimeout: 10ms
readtimeout: 10ms
writetimeout: 10ms
proxy:
remoteurl: https://registry-1.docker.io
username: {{ docker-hub-user }} #optional
password: {{Â docker-hub-password }} #optional
health:
storagedriver:
enabled: true
interval: 10s
threshold: 3
|
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
---
version: "3"
services:
registry:
container_name: registry
image: registry:latest
volumes:
- {{ storage-path }}registry/config/config.yml:/etc/docker/registry/config.yml
- {{ storage-path }}registry/cache:/var/lib/registry-cache
restart: unless-stopped
labels:
traefik.enable: true
traefik.http.routers.registry.entryPoints: https
traefik.http.services.registry.loadbalancer.server.port: 5000
networks:
- proxy
networks:
proxy: # rename this to your custom docker network.
driver: bridge
external: true
|
Deployment 2 - Registry with Nginx Proxy Manager
for Deployment 2 i will leverage Nginx Proxy Manager as a Reverse Proxy for the registry itself.
Requirements Deployment 2
1
2
3
4
5
6
7
8
9
|
---
version: "3"
services:
registry:
container_name: registry
image: registry:latest
volumes:
- /home/rocky/docker/registry/config.yml:/etc/docker/registry/config.yml
- /home/rocky/docker/registry/cache:/var/lib/registry-cache
|
Setup Docker Hosts
to let the hosts use the Registry they need to be configured.
on the docker host open the file /etc/docker/daemon.json with root privilege and insert the following.
1
2
3
|
{
"registry-mirrors": ["https://{{ internal-registry }}"]
}
|
if you already put any config in this file before, beware of the json formatting.
example:
1
2
3
4
5
6
7
8
9
10
11
|
{
"ipv6": true,
"fixed-cidr-v6": "fd00:0:3:ee07:8000::/96",
"experimental": true,
"ip6tables": true,
"default-address-pools": [
{ "base": "192.0.2.0/16", "size": 24 },
{ "base": "fd00:0:3:ee07:8000::/80", "size": 96 }
],
"registry-mirrors": ["https://{{ internal-registry }}"]
}
|
restart the Docker Server with sudo systemctl restart docker.service and enter into the shell docker info which should show the option Registry Mirrors:
to verify the operation of the registry open 2 shells
in the 1st shell enter docker logs -f registry
in the 2nd shell enter docker image pull nginx:latest
the 1st shell shoud show something like this:
1
2
3
4
5
6
7
8
9
10
11
12
|
192.0.2.21 - - [25/Aug/2023:11:53:01 +0000] "GET /v2/ HTTP/1.1" 200 2 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:01 +0000] "HEAD /v2/library/nginx/manifests/latest HTTP/1.1" 200 1862 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:02 +0000] "GET /v2/library/nginx/manifests/sha256:104c7c5c54f2685f0f46f3be607ce60da7085da3eaa5ad22d3d9f01594295e9c HTTP/1.1" 200 1862 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:02 +0000] "GET /v2/library/nginx/manifests/sha256:d204087971390839f077afcaa4f5a771c1694610f0f7cb13a2d2a3aa520b053f HTTP/1.1" 200 1778 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:03 +0000] "GET /v2/library/nginx/blobs/sha256:ab73c7fd672341e41ec600081253d0b99ea31d0c1acdfb46a1485004472da7ac HTTP/1.1" 200 8164 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:03 +0000] "GET /v2/library/nginx/blobs/sha256:76d048093f368f97120bd8d501c5c4b96d11c35cb5ad3845c951f2ca56f7d751 HTTP/1.1" 200 627 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:04 +0000] "GET /v2/library/nginx/blobs/sha256:658197f4b59221c1c269f31e4d0a4106fb22e8518f3153bd804e1ebe7491dd28 HTTP/1.1" 200 958 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:05 +0000] "GET /v2/library/nginx/blobs/sha256:a2543a59b279bfb9ec06516b396a34a38fd76210a8cbf955e5d6ec792599eb9c HTTP/1.1" 200 369 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:05 +0000] "GET /v2/library/nginx/blobs/sha256:3972a57e55756de565ed72021b453e7defd697a02da70c81f89da230ea9ae656 HTTP/1.1" 200 1214 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:06 +0000] "GET /v2/library/nginx/blobs/sha256:82359da50743ca88b17d4cb3bf971e2b0a5e4e0a78cabfcae1dee23dcbad550b HTTP/1.1" 200 1404 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:03 +0000] "GET /v2/library/nginx/blobs/sha256:4ee097f9a36616fddb52e45aba72142c4bc6f2e594f0a746e406acfde4f02f51 HTTP/1.1" 200 29157256 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
192.0.2.21 - - [25/Aug/2023:11:53:03 +0000] "GET /v2/library/nginx/blobs/sha256:6710b2157bb528b7cb408cf8144a8dc9466d9fee44cda1277adb272c69f37973 HTTP/1.1" 200 38016713 "" "docker/24.0.5 go/go1.20.6 git-commit/a61e2b4 kernel/6.4.11-200.fc38.aarch64 os/linux arch/arm64 UpstreamClient(Docker-Client/24.0.5 \\(linux\\))"
|
inside the registry folder on your docker host you can enter du -sh -- * .* and you should see the size of the cache folder increase.
