Repository navigation
fix(ci): rewrite every workspace path dependency in the publish tree - #43
Merged
Merged
Conversation
Publish crates for tag 0.31.2 (run 43930447468) died in refresh_compatible_registry_lock with exit 101: error: failed to select a version for the requirement `pd-host-schema = "^0.1.0"` candidate versions found which didn't match: 0.31.2 location searched: .../crates/pd-host-schema required by package `pd-vm v0.31.2` rewrite_manifests bumps [workspace.package] version to the release version, which moves crates/pd-host-schema to 0.31.2, but it only rewrote path dependencies named pd-host-function and pd-vm. pd-vm's build-dependency (and dev-dependency) kept `pd-host-schema = "^0.1.0"`, so cargo could not resolve the publish tree at all and aborted the `cargo update -p pd-host-function@0.22.2 --precise 0.22.7` that runs before anything is published. Changes: - package_versions is the single source of truth for both the publish plan and the dependency rewrite, so a crate added to PACKAGE_ORDER cannot keep a stale requirement behind. - Path dependencies are rewritten per section: normal and build tables get the publish version, [dev-dependencies] (including target-specific tables) lose the version so cargo strips them from the packaged manifest. Checked with `cargo package --offline`: a versioned path dev-dependency that is not on crates.io fails packaging with `no matching package named ...`, which would block pd-host-function (published before the pd-vm it dev-depends on). - pd-host-schema now leads PACKAGE_ORDER: it is a normal dependency of pd-host-function and a build dependency of pd-vm, and had never been published. The workflow PACKAGE_ORDER is kept in sync and asserted. - verify_publish_order rejects an order that omits a workspace crate with a normal/build edge, which would have caught this before upload. - stale_path_dependencies re-checks the rewritten tree, and refresh_compatible_registry_lock prints cargo's output instead of failing with a bare CalledProcessError. Verification: python3 -m unittest scripts.test_publish_crates scripts.test_verify_publish_order (18 tests), py_compile, YAML parse, verify_publish_order.py (new order passes, the 0.31.2 order now fails), and a --prepare-only run over a copy of this tree for 0.31.2/0.22.7 that finishes the lock refresh and prints the full plan. Nothing was published; no tag or version was changed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Publish crates, tag0.31.2, run 43930447468 failed before uploading anything:and then
scripts/publish_crates.pyexited 101 atcargo update -p pd-host-function@0.22.2 --precise 0.22.7.Root cause
rewrite_manifests()rewrites[workspace.package] versionto the release version, which movescrates/pd-host-schema(it usesversion.workspace = true) to0.31.2. But it only rewrote pathdependencies whose package name was
pd-host-functionorpd-vm.pd-vmdeclarespd-host-schemaas a build dependency (and a dev dependency) withversion = "0.1.0", so thepublish tree required
^0.1.0from a path crate that now is0.31.2.That mismatch makes the whole workspace unresolvable, so the
cargo updatethatrefresh_compatible_registry_lock()runs before publishing aborted with 101 — the pathdependency, not the registry entry, was the blocker. Reproduced locally with
--prepare-only --version 0.31.2on an untouched copy of the tree: identical error, identicalexit status.
Two secondary problems in the same area:
pd-host-schemais a normal dependency ofpd-host-functionand a build dependency ofpd-vmand has never been published (crates.io returns no versions), yet it was missing from
PACKAGE_ORDER, sopd-vm 0.31.2could never be published even with correct manifests.verify_publish_order.pysilently ignored workspace crates absent from the order, so the gatecould not catch that.
Fix
package_versions()is now the single source of truth for the publish plan and the pathdependency rewrite. Adding a crate to
PACKAGE_ORDERcan no longer leave a stale requirementbehind.
the publish version;
[dev-dependencies]lose the version, so cargo strips them from thepackaged manifest instead of demanding an unpublished version. Verified with
cargo package --no-verify --offline: a versioned path dev-dependency that is not on crates.iofails packaging with
no matching package named ..., which would blockpd-host-function(published before the
pd-vmit dev-depends on).pd-host-schemaleadsPACKAGE_ORDER; the workflowPACKAGE_ORDERis synced and asserted by atest.
verify_publish_order.pynow rejects an order that omits a workspace crate with a normal/buildedge.
stale_path_dependencies()re-checks the rewritten tree and fails loudly;refresh_compatible_registry_lock()prints cargo's output instead of raising a bareCalledProcessError.Verification
python3 -m unittest scripts.test_publish_crates scripts.test_verify_publish_order— 18 tests,OKpython3 -m py_compileon all four scripts;yaml.safe_loadon both workflowspython3 scripts/verify_publish_order.py pd-host-schema pd-host-function pd-vm pd-vm-nostd rustscript— valid; the old 0.31.2 order now fails with both missing edges named--prepare-only --version 0.31.2 --host-version 0.22.7on a copy of this tree: the lock refresh finishes (updatespd-host-function 0.22.2 -> 0.22.7plus the workspace crates) and prints[["pd-host-schema","0.31.2"],["pd-host-function","0.22.7"],["pd-vm","0.31.2"],["pd-vm-nostd","0.31.2"],["rustscript","0.31.2"]]cargo metadatain the rewritten tree:pd-vmbuild deppd-host-schema ^0.31.2, dev dep*cargo package -p pd-host-schema --no-verify --offlinein the rewritten tree succeedsNo crates were published, no tag or version was changed, and nothing was merged here. Full
end-to-end proof still requires the real
Publish cratesrun, which must publishpd-host-schemafirst.