Skip to content

Don't fetch a dependency another package already provided - #89

Merged
ryanmrichard merged 1 commit into
masterfrom
dedup_namespaced_dep_targets
Sep 15, 2026
Merged

ryanmrichard merged 1 commit into
masterfrom
dedup_namespaced_dep_targets

Conversation

@ryanmrichard

Copy link
Copy Markdown
Member

The bug

get_dependencies() decided whether a dependency was already resolved by testing
TARGET ${NWX_DEP_TARGET_<name>}, which defaults to the bare dep name. For
third-party packages that name is usually not a target that exists:

dep targets it actually defines TARGET <bare name>
pybind11 pybind11::pybind11, pybind11::headers, pybind11_headers ✗ never
catch2 Catch2, Catch2::Catch2 (names are case-sensitive) ✗ never

So the short-circuit never fired for either — which is why every configure logs
-- Fetching dependency: pybind11 four or five times.

That was noise, not breakage, while FetchContent_MakeAvailable() was the only way
those packages entered a build: it dedups against its own earlier population.

#87/#88 removed that guarantee. Now that the generated <name>Config.cmake
files declare their real dependencies, pluginplayConfig.cmake calls
nwx_config_find_dependency(pybind11) — so resolving pluginplay from its installed
wheel find_package()es pybind11 and creates pybind11::pybind11 et al. as
IMPORTED targets. The subsequent get_dependencies(pybind11) missed them, fell
through to FetchContent_MakeAvailable(), and add_subdirectory()'d pybind11's own
CMakeLists over the top:

CMake Error at build/_deps/pybind11-src/CMakeLists.txt:280 (add_library):
  add_library cannot create ALIAS target "pybind11::pybind11_headers" because
  another target with the same name already exists.

...plus eight more from pybind11Common.cmake. This is the failure on every
test_cmake_build leg of NWChemEx/SCF#75, whose integration-testing step pip
installs the published ecosystem before configuring.

The fix

Two parts:

  1. dependencies/pybind11.cmake and dependencies/catch2.cmake declare the target
    they actually provide via _gd_target_<name>.
  2. get_dependencies() re-checks after the include() — the earliest point at
    which that name is known, since the existing check runs before it and, on a first
    call, has no cache entry to consult anyway — and skips the fetch when the target
    already exists.

Part 2 is load-bearing on its own: with only part 1, the first
get_dependencies(pybind11) in a configure still fetched, because the pre-include()
check has nothing to test yet. I verified that intermediate state failed before
adding it.

Catch2 is fixed alongside as the same latent defect. Nothing find_package()es
Catch2 ahead of get_dependencies() today — it is test-only and so never appears in
an installed target's INTERFACE_LINK_LIBRARIES — so it relies on
FetchContent_MakeAvailable()'s dedup exactly the way pybind11 used to.

Verification

Against the real published wheels (nwchemex-pluginplay 1.0.65, pybind11 3.1.0)
with get_dependencies(pluginplay pybind11), which is the SCF path:

  • Control, unpatched → fails with exactly the error above (CMakeLists.txt:280,
    pybind11Common.cmake:68/75/82/112/121/146), byte-identical to the SCF#75 CI log.
  • Patched → -- pybind11: already provided (pybind11::pybind11), configures clean.

Also confirmed:

  • no-wheel path still fetches pybind11 and catch2 normally, and both targets exist afterward;
  • a repeated get_dependencies(pybind11) short-circuits via the existing cache check;
  • ecosystem deps are unaffected — nwx_ecosystem_dependency only sets
    _gd_target_<name> in its installed-wheel branch, where it already sets
    _gd_uses_fc FALSE.

🤖 Generated with Claude Code

get_dependencies() decided whether a dependency was already resolved by
testing TARGET ${NWX_DEP_TARGET_<name>}, which defaults to the bare dep
name. For third-party packages that name is usually not a target that
exists -- pybind11 defines pybind11::pybind11, Catch2 defines
Catch2::Catch2 (and CMake target names are case-sensitive, so even
"Catch2" would not match "catch2"). The check therefore never fired for
either, which is why every configure logs "Fetching dependency: pybind11"
four or five times.

That was noise, not breakage, while FetchContent_MakeAvailable() was the
only way those packages entered a build: it dedups against its own
earlier population. Declaring real dependencies in the generated
<name>Config.cmake files removed that guarantee. pluginplayConfig.cmake
now calls nwx_config_find_dependency(pybind11), so resolving pluginplay
from its installed wheel find_package()es pybind11 and creates
pybind11::pybind11 et al. as IMPORTED targets. The subsequent
get_dependencies(pybind11) missed them, fell through to
FetchContent_MakeAvailable(), and add_subdirectory()'d pybind11's own
CMakeLists over the top -- which dies re-creating targets that already
exist:

  add_library cannot create ALIAS target "pybind11::pybind11_headers"
  because another target with the same name already exists

This is the failure on every test_cmake_build leg of NWChemEx/SCF#75,
where the integration-testing step pip installs the published ecosystem
before configuring.

Fixed in two parts. The dep files now name the target they actually
provide via _gd_target_<name>. get_dependencies() re-checks after the
include() -- the earliest point at which that name is known, since the
existing check runs before it and, on a first call, has no cache entry
to consult anyway -- and skips the fetch when the target is already
there.

Verified against the real published wheels (nwchemex-pluginplay 1.0.65,
pybind11 3.1.0) with get_dependencies(pluginplay pybind11): unpatched
fails with exactly the error above, patched logs "pybind11: already
provided (pybind11::pybind11)" and configures clean. Also confirmed the
no-wheel path still fetches both packages normally and that a repeated
get_dependencies(pybind11) short-circuits.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@ryanmrichard
ryanmrichard merged commit a88c64d into master Sep 15, 2026
7 checks passed
@ryanmrichard
ryanmrichard deleted the dedup_namespaced_dep_targets branch September 15, 2026 19:15
@github-actions

Copy link
Copy Markdown

🚀 [bumpr] Bumped!
New version:v0.2.4
Changes:v0.2.3...v0.2.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant