Repository navigation
Give the Linux wheel verification an MPI to load - #193
Merged
Merged
Conversation
The Linux leg's import check has never been able to pass for an MPI-linked wheel: ImportError: libmpi.so.40: cannot open shared object file By design the Linux wheel does not vendor MPI -- CIBW_REPAIR_WHEEL_COMMAND_LINUX excludes libmpi/libopen-rte/libopen-pal so it links against the runtime system's OpenMPI, because vendoring is a dead end (auditwheel bundles shared libraries but never the bin/orted that OpenMPI's singleton startup needs). The other half of that contract is that something must supply MPI wherever the wheel is imported, and nothing did: CIBW_BEFORE_ALL_LINUX builds OpenMPI into /usr/local/openmpi inside the manylinux *container*, while "Verify Wheel Installs" runs on the runner host. Installing libopenmpi-dev/openmpi-bin there makes the host resemble the end user's machine the wheel already documents as a requirement. Same packages setup_nwx_dev_env uses, so the ABI matches what CIBW_BEFORE_ALL_LINUX compiled against. macOS is unaffected -- delocate vendors MPI into the wheel there -- which is why the ParallelZone release that surfaced this passed on macos-14 and failed only on Linux. This was masked rather than new: import_check.py arrived in 65681cd, the same commit that introduced the SKBUILD_CMAKE_DEFINE quoting defect fixed in #192. Every build died before reaching verification, so the check had never actually run on Linux. Found by NWChemEx/ParallelZone#161. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
|
🚀 [bumpr] Bumped! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The Linux
build_pypi_distleg fails verification for any MPI-linked wheel:The wheel itself builds and repairs correctly — this is the "Verify Wheel Installs" step.
Why
By design the Linux wheel does not vendor MPI.
CIBW_REPAIR_WHEEL_COMMAND_LINUXexcludeslibmpi/libopen-rte/libopen-palso it links against the runtime system's OpenMPI, because vendoring is a dead end — auditwheel bundles shared libraries but never the accompanyingbin/ortedthat OpenMPI's singleton startup needs. The action documents this at lines 244-250.The other half of that contract is that something has to supply MPI wherever such a wheel is imported. Nothing did:
CIBW_BEFORE_ALL_LINUXbuilds OpenMPI into/usr/local/openmpiinside the manylinux container, while verification runs on the runner host, which has no MPI at all.Installing
libopenmpi-dev/openmpi-binthere makes the host resemble the end user's machine the wheel already documents as a requirement. Same packagessetup_nwx_dev_envuses, so the ABI matches whatCIBW_BEFORE_ALL_LINUXcompiled against (Ubuntu 24.04 ships the OpenMPI 4.1.x providinglibmpi.so.40).macOS is unaffected — delocate vendors MPI into the wheel there. That's exactly the split seen in the run that surfaced this:
macos-14passed,ubuntu-latestfailed.Not a regression
import_check.pyarrived in65681cd— the same commit that introduced theSKBUILD_CMAKE_DEFINEquoting defect fixed in #192. Every build died before reaching verification, so this check had never once run on Linux. #192 fixed the first bug and uncovered the second.How it was found
NWChemEx/ParallelZone#161, a no-op rebuild to republish wheels against NWXCMake 0.2.2.
deploy_to_pypihasneeds: build_pypi_dist, so the failing Linux leg skipped the publish entirely and nothing was released — which is the blocker for rebuilding the ecosystem.🤖 Generated with Claude Code