feat(docker): enhance base image mirroring process and update documentation
quality / commitlint (push) Skipped
quality / changes (push) Successful in 8s
quality / docker-check (push) Skipped
quality / openapi (push) Successful in 40s
quality / web (push) Successful in 1m27s
quality / go (push) Successful in 1m4s
quality / bird2 (push) Successful in 18s
CD / quality (push) Successful in 3m45s
CD / publish (push) Canceled after 10m39s

- Updated `mirror-base-images.sh` to skip existing tags in Gitea and retry on 429 errors, improving efficiency and reducing unnecessary requests.
- Modified CI workflow to utilize a temporary environment file for mirrored base images, ensuring better handling of existing images.
- Enhanced README documentation to reflect changes in the mirroring process and clarify the behavior when tags already exist.
This commit is contained in:
Denozordec
2026-08-18 18:22:26 +07:00
parent e0ecabb22f
commit 0148d4ca37
4 changed files with 76 additions and 26 deletions
+1 -1
View File
@@ -24,7 +24,7 @@ Runner: `ubuntu-latest`, **bird2** из apt, Docker для **docker-check** (PR)
1. `pnpm exec semantic-release` — тег `vX.Y.Z` на **текущий commit** (без дополнительного commit в main). 1. `pnpm exec semantic-release` — тег `vX.Y.Z` на **текущий commit** (без дополнительного commit в main).
2. Gitea Release + `CHANGELOG.md` как attachment (не в git). 2. Gitea Release + `CHANGELOG.md` как attachment (не в git).
3. Зеркало base-образов в `evobgp-buildcache:base-*` (`deploy/docker/mirror-base-images.sh`). 3. Зеркало base-образов в `evobgp-buildcache:base-*` (`deploy/docker/mirror-base-images.sh`; skip существующих тегов, `linux/amd64`, retry при 429).
4. `docker buildx bake default --push` с `VERSION=X.Y.Z`, `pull=false`, named builder `evobgp` (`cleanup: false`). 4. `docker buildx bake default --push` с `VERSION=X.Y.Z`, `pull=false`, named builder `evobgp` (`cleanup: false`).
Если releasable-коммитов нет — semantic-release no-op, образы не публикуются. Если releasable-коммитов нет — semantic-release no-op, образы не публикуются.
+7 -10
View File
@@ -108,10 +108,9 @@ jobs:
password: ${{ secrets.ACTIONS_PAT }} password: ${{ secrets.ACTIONS_PAT }}
- name: Mirror base images into buildcache - name: Mirror base images into buildcache
if: steps.rel.outputs.released == 'true' if: steps.rel.outputs.released == 'true'
id: mirror
continue-on-error: true
env: env:
REGISTRY: git.shx.one/${{ steps.meta.outputs.owner_lc }} REGISTRY: git.shx.one/${{ steps.meta.outputs.owner_lc }}
MIRROR_ENV_FILE: ${{ runner.temp }}/mirror-base.env
run: sh deploy/docker/mirror-base-images.sh run: sh deploy/docker/mirror-base-images.sh
- name: Build and push images (bake) - name: Build and push images (bake)
if: steps.rel.outputs.released == 'true' if: steps.rel.outputs.released == 'true'
@@ -126,16 +125,14 @@ jobs:
CACHE_REF_WEB: git.shx.one/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:web-buildcache CACHE_REF_WEB: git.shx.one/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:web-buildcache
CACHE_REF_BIRDC: git.shx.one/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:birdc-buildcache CACHE_REF_BIRDC: git.shx.one/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:birdc-buildcache
BUILDX_BAKE_ENTITLEMENTS_FS: "0" BUILDX_BAKE_ENTITLEMENTS_FS: "0"
MIRROR_ENV_FILE: ${{ runner.temp }}/mirror-base.env
run: | run: |
set -euxo pipefail set -euxo pipefail
cache="git.shx.one/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache" if [ -f "${MIRROR_ENV_FILE}" ]; then
if [ "${{ steps.mirror.outcome }}" = "success" ]; then set -a
export BASE_GOLANG="${cache}:base-golang-1.24-alpine" # shellcheck disable=SC1090
export BASE_DEBIAN="${cache}:base-debian-bookworm-slim" . "${MIRROR_ENV_FILE}"
export BASE_NODE="${cache}:base-node-22-alpine" set +a
export BASE_NGINX="${cache}:base-nginx-1.27-alpine"
export BASE_UBUNTU="${cache}:base-ubuntu-noble"
export BASE_DISTROLESS="${cache}:base-distroless-static-debian12-nonroot"
fi fi
docker buildx bake --allow=fs.read="${{ github.workspace }}" \ docker buildx bake --allow=fs.read="${{ github.workspace }}" \
-f deploy/docker/docker-bake.hcl default --push -f deploy/docker/docker-bake.hcl default --push
+1 -1
View File
@@ -27,7 +27,7 @@ BuildKit кэширует `/go/pkg/mod`, `~/.cache/go-build` и pnpm store че
- `git.shx.one/<owner>/evobgp-buildcache:go-buildcache`**запись** только из `go-build-all` - `git.shx.one/<owner>/evobgp-buildcache:go-buildcache`**запись** только из `go-build-all`
- `git.shx.one/<owner>/evobgp-buildcache:web-buildcache`**запись** только из `web-build` - `git.shx.one/<owner>/evobgp-buildcache:web-buildcache`**запись** только из `web-build`
- `git.shx.one/<owner>/evobgp-buildcache:birdc-buildcache`**запись** только из `go-birdc` - `git.shx.one/<owner>/evobgp-buildcache:birdc-buildcache`**запись** только из `go-birdc`
- `git.shx.one/<owner>/evobgp-buildcache:base-*` — зеркало FROM-образов (`mirror-base-images.sh`), без rebuild - `git.shx.one/<owner>/evobgp-buildcache:base-*` — зеркало FROM (`mirror-base-images.sh`): только `linux/amd64`, skip если тег уже в Gitea (без повторного pull с ECR / 429). Неуспешный copy не валит CD — bake берёт public FROM для этой базы.
`pull = false` в bake: не перекачивать FROM, если слой уже в builder. Не запускайте `docker system prune -a` на runner. `pull = false` в bake: не перекачивать FROM, если слой уже в builder. Не запускайте `docker system prune -a` на runner.
+67 -14
View File
@@ -2,27 +2,80 @@
# Copy-by-digest зеркало базовых образов в Gitea Container Registry (без rebuild). # Copy-by-digest зеркало базовых образов в Gitea Container Registry (без rebuild).
# Требует: docker login в REGISTRY host, docker buildx. # Требует: docker login в REGISTRY host, docker buildx.
# REGISTRY=git.shx.one/<owner> (без суффикса /evobgp-buildcache) # REGISTRY=git.shx.one/<owner> (без суффикса /evobgp-buildcache)
#
# Не ходим в public.ecr.aws, если тег уже есть в Gitea (иначе 429 Too Many Requests).
# Копируем только linux/amd64 — bake platforms совпадает, multi-arch индекс ECR не нужен.
# Шаг CD не должен падать: неуспешный copy оставляет bake на public FROM для этой базы.
set -eu set -eu
REGISTRY="${REGISTRY:?REGISTRY required (git.shx.one/<owner>)}" REGISTRY="${REGISTRY:?REGISTRY required (git.shx.one/<owner>)}"
CACHE_REPO="${REGISTRY}/evobgp-buildcache" CACHE_REPO="${REGISTRY}/evobgp-buildcache"
ENV_FILE="${MIRROR_ENV_FILE:-}"
PLATFORM="${MIRROR_PLATFORM:-linux/amd64}"
RETRIES="${MIRROR_RETRIES:-4}"
tmp="$(mktemp)"
cleanup() { rm -f "$tmp"; }
trap cleanup EXIT
dest_exists() {
docker buildx imagetools inspect "$1" >/dev/null 2>&1
}
copy_retry() {
src="$1"
dest="$2"
n=0
while [ "$n" -lt "$RETRIES" ]; do
n=$((n + 1))
if docker buildx imagetools create --platform "$PLATFORM" --tag "$dest" "$src"; then
return 0
fi
delay=$((n * 25))
echo "mirror retry ${n}/${RETRIES}, sleep ${delay}s: ${dest}"
sleep "$delay"
done
return 1
}
mirror() { mirror() {
src="$1" src="$1"
tag="$2" tag="$2"
echo "mirror ${src} -> ${CACHE_REPO}:${tag}" var="$3"
docker buildx imagetools create --tag "${CACHE_REPO}:${tag}" "${src}" dest="${CACHE_REPO}:${tag}"
if dest_exists "$dest"; then
echo "skip (already in registry): ${dest}"
printf '%s=%s\n' "$var" "$dest" >>"$tmp"
return 0
fi
echo "mirror ${src} -> ${dest} (${PLATFORM})"
if copy_retry "$src" "$dest"; then
printf '%s=%s\n' "$var" "$dest" >>"$tmp"
return 0
fi
echo "warn: ${dest} not mirrored — bake uses public FROM for ${var}"
return 0
} }
mirror "public.ecr.aws/docker/library/golang:1.24-alpine" "base-golang-1.24-alpine" mirror "public.ecr.aws/docker/library/golang:1.24-alpine" "base-golang-1.24-alpine" BASE_GOLANG
mirror "public.ecr.aws/docker/library/debian:bookworm-slim" "base-debian-bookworm-slim" sleep 8
mirror "public.ecr.aws/docker/library/node:22-alpine" "base-node-22-alpine" mirror "public.ecr.aws/docker/library/debian:bookworm-slim" "base-debian-bookworm-slim" BASE_DEBIAN
mirror "public.ecr.aws/docker/library/nginx:1.27-alpine" "base-nginx-1.27-alpine" sleep 8
mirror "public.ecr.aws/docker/library/ubuntu:noble" "base-ubuntu-noble" mirror "public.ecr.aws/docker/library/node:22-alpine" "base-node-22-alpine" BASE_NODE
mirror "gcr.io/distroless/static-debian12:nonroot" "base-distroless-static-debian12-nonroot" sleep 8
mirror "public.ecr.aws/docker/library/nginx:1.27-alpine" "base-nginx-1.27-alpine" BASE_NGINX
sleep 8
mirror "public.ecr.aws/docker/library/ubuntu:noble" "base-ubuntu-noble" BASE_UBUNTU
sleep 8
mirror "gcr.io/distroless/static-debian12:nonroot" "base-distroless-static-debian12-nonroot" BASE_DISTROLESS
echo "BASE_GOLANG=${CACHE_REPO}:base-golang-1.24-alpine" echo "----- mirrored BASE_* -----"
echo "BASE_DEBIAN=${CACHE_REPO}:base-debian-bookworm-slim" cat "$tmp"
echo "BASE_NODE=${CACHE_REPO}:base-node-22-alpine" if [ -n "$ENV_FILE" ]; then
echo "BASE_NGINX=${CACHE_REPO}:base-nginx-1.27-alpine" env_dir="$(dirname "$ENV_FILE")"
echo "BASE_UBUNTU=${CACHE_REPO}:base-ubuntu-noble" if [ -d "$env_dir" ]; then
echo "BASE_DISTROLESS=${CACHE_REPO}:base-distroless-static-debian12-nonroot" cp "$tmp" "$ENV_FILE"
echo "wrote ${ENV_FILE}"
else
echo "warn: MIRROR_ENV_FILE dir missing (${env_dir}) — skip write"
fi
fi