Skip to content

[CI] Make GitHub Build workflow fit the hosted runner (test PR) - #49

Open
pggPL wants to merge 2 commits into
mainfrom
build_ci_fix
Open

[CI] Make GitHub Build workflow fit the hosted runner (test PR)#49
pggPL wants to merge 2 commits into
mainfrom
build_ci_fix

Conversation

@pggPL

@pggPL pggPL commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Test PR on the fork to exercise the Build GitHub workflow.

Why upstream Build has been red since Dec 2025:

  • Core: 111 CUDA TUs x 6 archs with MAX_JOBS=1; first TU alone takes 32 min, job hits the 6h limit (shows as cancelled).
  • PyTorch / All: same build inside docker; runner runs out of resources ("hosted runner lost communication").
  • JAX: sccache 0.17 + nvcc 13.3 in the jax image dies immediately (fatbinary: Could not open input file ...cubin, Compiler killed by signal 1).
  • env: on the docker-exec steps never reached the container, so MAX_JOBS/NVTE_FRAMEWORK were not applied there.

Changes mirror the working minimum_versions.yml: NVTE_CUDA_ARCHS=90, MAX_JOBS=2, env passed via docker exec -e, libcuda stub for the sanity import, no sccache in the CUDA 13 JAX job.

Build a single CUDA arch (sm90) with MAX_JOBS=2, pass build env into the
docker-based jobs, add libcuda stub for the sanity import and drop sccache
in the CUDA 13 JAX job where it crashes nvcc.

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
… to seed the cache

Signed-off-by: Pawel Gadzinski <pgadzinski@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant