Skip to content

ci(release): build multi-arch docker images in the tag pipeline #2139

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
erubboli merged 2 commits into from
Oct 12, 2026
Merged

ci(release): build multi-arch docker images in the tag pipeline #2139

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
62f518a
ci(release): build multi-arch docker images in the tag pipeline
nullPointerEnjoyer Oct 12, 2026
9ab54c4
ci(release): fail loudly when the docker assembly is skipped
nullPointerEnjoyer Oct 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
78 changes: 75 additions & 3 deletions .github/workflows/release.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,13 +27,53 @@ jobs:
binary_list: api-blockchain-scanner-daemon,api-web-server,dns-server,node-daemon,wallet-address-generator,wallet-cli,wallet-rpc-daemon
secrets: inherit

build-docker:
# Docker images still build only after the OS binaries built (a red OS
# build aborts before any image is pushed). The two per-arch builds run on
# native runners and serialize behind the release_docker.yml
# 'docker-release' concurrency group. Within one release run at most one
# job is ever pending in that group, so the legs of a single run cannot
# drop each other; across runs the one-pending-slot rule still applies
# (a newer pending run cancels an older pending one — see the build job
# in release_docker.yml), so an overlapping release or dispatch can
# cancel this run's queued arm64 leg, which then skips the assembly and
# leaves the release amd64-only (create-release does not wait for it);
# the docker-release-status job below then fails the run so the gap is
# visible instead of silent.
# The assembly then merges the two per-arch image
# sets into multi-arch manifest lists under the unsuffixed tags.
# create-release deliberately does NOT depend on the arm64 build or the
# assembly: if either fails, the OS binaries and the amd64 images are
# still released, and the arm64 half can be repaired afterwards by
# re-running the failed jobs of this run or by dispatching
# release_docker.yml (build with tag_suffix arm64, then assembly) from
# the tag ref.
build-docker-amd64:
needs: [build-macos, build-linux, build-windows]
uses: ./.github/workflows/release_docker.yml
secrets: inherit

build-docker-arm64:
needs: [build-macos, build-linux, build-windows]
uses: ./.github/workflows/release_docker.yml
with:
platform: arm64
tag_suffix: arm64
runner: ubuntu-24.04-arm
secrets: inherit

assemble-docker:
needs: [build-docker-amd64, build-docker-arm64]
uses: ./.github/workflows/release_docker.yml
with:
# release.yml only triggers on semver tag pushes, so the ref name is
# the tag (e.g. 'v1.4.2' or '1.4.2'); release_docker.yml strips a
# leading 'v' before validating it.
version: ${{ github.ref_name }}
assemble_manifests: 'true'
secrets: inherit

create-release:
needs: [build-macos, build-linux, build-windows, build-docker]
needs: [build-macos, build-linux, build-windows, build-docker-amd64]
runs-on: ubuntu-latest
if: startsWith(github.ref, 'refs/tags/')
steps:
Expand Down Expand Up @@ -98,4 +138,36 @@ jobs:
artifactErrorsFailBuild: true
name: "Release ${{ steps.get_version.outputs.VERSION }}"
body: ${{ steps.generate_body.outputs.BODY }}
token: ${{ secrets.GITHUB_TOKEN }}
token: ${{ secrets.GITHUB_TOKEN }}

# Runs after everything else and fails the run when a published release
# lacks its multi-arch assembly (arm64 build failed/canceled — including a
# concurrency-dropped pending leg, where no job fails and the run would
# otherwise conclude 'cancelled' — or the assembly itself failed). The
# status job therefore runs always(); the script only alerts when the
# release actually shipped, so a deliberately canceled run still
# terminates quietly and a red run here means "repair the arm64 half",
# not "the release is missing".
docker-release-status:
needs: [create-release, build-docker-amd64, build-docker-arm64, assemble-docker]
if: ${{ always() }}
runs-on: ubuntu-latest
permissions: {}
timeout-minutes: 10
steps:
- name: Check multi-arch completeness
env:
RELEASE: ${{ needs.create-release.result }}
AMD64: ${{ needs.build-docker-amd64.result }}
ARM64: ${{ needs.build-docker-arm64.result }}
ASSEMBLY: ${{ needs.assemble-docker.result }}
run: |
if [[ "$ASSEMBLY" != 'success' ]]; then
echo "multi-arch manifest assembly incomplete (release=$RELEASE, amd64=$AMD64, arm64=$ARM64, assembly=$ASSEMBLY)" >&2
if [[ "$RELEASE" == 'success' ]]; then
echo "the release shipped amd64-only" >&2
echo "repair by re-running the failed jobs of this run, or by dispatching release_docker.yml (platform=arm64, tag_suffix=arm64, runner=ubuntu-24.04-arm, then assemble_manifests=true) from the tag ref" >&2
exit 1
fi
echo "the release itself did not ship; fix the earlier failed or skipped job first" >&2
fi
61 changes: 48 additions & 13 deletions .github/workflows/release_docker.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,36 @@ name: Build Docker

on:
workflow_call:
# Inputs for callers such as release.yml. They must be declared here:
# a workflow_call trigger without an inputs block drops every value a
# caller passes, which would silently turn an arm64 build into an amd64
# build on the wrong runner.
inputs:
version:
description: 'Version for image tags, e.g. 1.4.1 (defaults to the pushed tag name, or the short commit SHA for ad-hoc dispatches)'
required: false
type: string
default: ''
platform:
description: 'Target CPU architecture (amd64 or arm64)'
required: false
type: string
default: 'amd64'
tag_suffix:
description: 'Suffix appended to every image tag, e.g. arm64 (leave empty for the default arch)'
required: false
type: string
default: ''
runner:
description: 'Runner label; only ubuntu-24.04 / ubuntu-24.04-arm are honored, anything else falls back to ubuntu-latest'
required: false
type: string
default: 'ubuntu-latest'
assemble_manifests:
description: "Skip image builds and only assemble multi-arch manifests from already-pushed '<tag>-amd64'/'<tag>-arm64' tags"
required: false
type: string
default: 'false'
workflow_dispatch:
inputs:
version:
Expand Down Expand Up @@ -32,11 +62,17 @@ permissions:

jobs:
build-docker-images:
# workflow_dispatch builds may push shared mutable tags ('<version>',
# 'v<version>', 'latest') with the repo's Docker Hub credentials, so they
# are restricted to protected refs; feature-branch dispatches cannot push.
# Tag refs are allowed at the same trust level as the release pipeline.
if: github.event_name == 'workflow_call' || (inputs.assemble_manifests != 'true' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-') || startsWith(github.ref, 'refs/tags/v')))
# Builds are skipped for assembly-only calls; all remaining builds push
# shared mutable tags ('<version>', 'v<version>', 'latest') with the
# repo's Docker Hub credentials, so they are restricted to protected
# refs. A called workflow inherits the caller's github.ref, so the tag
# pipeline's calls pass the same ref checks as a manual dispatch from a
# tag: any tag ref qualifies (the trigger glob of release.yml allows
# bare 'X.Y.Z' as well as 'vX.Y.Z', and 'Determine version' validates
# the grammar and strips a leading 'v' before anything is pushed);
Comment thread
nullPointerEnjoyer marked this conversation as resolved.
# feature-branch dispatches (or hypothetical feature-branch callers)
# cannot push.
if: inputs.assemble_manifests != 'true' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-') || startsWith(github.ref, 'refs/tags/'))
permissions:
contents: read
# Explicit bound below the 360-minute GitHub default, with headroom for
Expand Down Expand Up @@ -153,15 +189,14 @@ jobs:
env:
DOCKER_BUILDKIT: 1

# Registry-side assembly of multi-arch manifest lists from images that were
# pushed earlier by separate per-arch dispatches of the build job above
# ("--platform arm64 --tag_suffix arm64"). Nothing is rebuilt: "docker
# buildx imagetools create" only references already-pushed manifests. Runs
# from the default branch via "gh workflow run ... --ref <branch>": GitHub
# validates the workflow registration against the default branch but reads
# the file content from the dispatched ref.
# Registry-side assembly of multi-arch manifest lists from images that
# were pushed earlier by the separate per-arch builds of the build job
# above ("--platform arm64 --tag_suffix arm64"). Called automatically
# after both builds by release.yml (workflow_call), or manually via
# "gh workflow run ... --ref <protected ref>". Nothing is rebuilt: "docker
# buildx imagetools create" only references already-pushed manifests.
assemble-manifests:
if: github.event_name == 'workflow_dispatch' && inputs.assemble_manifests == 'true' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-') || startsWith(github.ref, 'refs/tags/v'))
if: inputs.assemble_manifests == 'true' && (github.ref == 'refs/heads/master' || startsWith(github.ref, 'refs/heads/release-') || startsWith(github.ref, 'refs/tags/'))
permissions:
contents: read
timeout-minutes: 60
Expand Down
8 changes: 6 additions & 2 deletions build-tools/docker/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,9 +64,13 @@ This command mounts the same `~/.mintlayer` directory as a volume in the `wallet

## Multi-arch (amd64 / arm64) images

By default the build produces `linux/amd64` images, exactly as it always has. Two extra knobs exist for building other architectures:
The release pipeline builds `linux/amd64` and `linux/arm64` natively on separate
GitHub runners and assembles the results into multi-arch manifest lists under
the unsuffixed tags, so `docker pull` auto-selects the architecture. A local
build defaults to `linux/amd64`, exactly as it always has. Two extra knobs
exist for building other architectures:

- `build.py --platform arm64` builds `linux/arm64` images (must run on an arm64 machine, e.g. the `ubuntu-24.04-arm` GitHub runner, since the compilation happens inside the container).
- `build.py --tag_suffix arm64` appends the suffix to every built and pushed tag (e.g. `node-daemon:1.4.1-arm64`), so a non-default architecture never overwrites the default-arch tags.

To assemble the suffixed images into a proper multi-arch manifest list under the unsuffixed tags (`docker pull mintlayer/node-daemon` then auto-selects the architecture), dispatch the **"Assemble multi-arch Docker manifests"** workflow (`release_docker_manifest.yml`) with the release version after both architectures have been pushed.
To assemble the suffixed images into a proper multi-arch manifest list under the unsuffixed tags (`docker pull mintlayer/node-daemon` then auto-selects the architecture), run the **"Build Docker"** workflow (`release_docker.yml`) with `assemble_manifests: true` and the release version after both architectures have been pushed. The release pipeline (`release.yml`) runs the same assembly automatically for semver tags.
Loading