Repository navigation
fix(ci): rewrite publish git/path dependencies for crates.io - #4
Merged
Merged
Conversation
`cargo publish -p <member>` resolves the whole workspace, so rewriting path
dependencies into registry requirements inside the checkout deadlocked the
0.2.0 tag: publishing pd-edge-abi failed 18 times with
failed to select a version for the requirement `pd-edge-abi = "^0.2.0"`
candidate versions found which didn't match: 0.1.1
required by package `pd-edge v0.2.0`
because the workspace member pd-edge already demanded an abi version that only
exists after abi publishes.
- .github/scripts/prepare_publish.py: copy each released crate into its own
standalone tree, drop path/git/rev/branch/tag selectors, apply the release
version to workspace crates and the registry version to pd-vm /
pd-host-function, remove dev-dependencies, materialise [workspace.package]
inheritance, and derive the publish order from the normal/build graph.
- .github/scripts/publish_crates.py: publish by --manifest-path in that order,
confirm each version in the sparse index (not only the crates.io API) before
building on it, skip already-published versions, and stop retrying once a
dependency is indexed.
- publish-crates.yml now calls the scripts, drops the rustscript checkout that
staging made unnecessary and the deprecated `cargo publish --token`.
- tests run against verbatim 0.2.0 manifest fixtures plus the live manifests,
covering the abi-before-pd-edge order, selector removal, dev-dep removal and
the CLI dry-run plan.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Failure and version check
Follow-up: root cause of the remaining 0.2.0 failure (run 43928737529)
The rewrite above was correct but structurally unable to work, because
cargo publish -p <member>resolves the whole workspace before it packages anything. Rewritingpd-edge's path dependency into a registry requirement in place made the workspace unresolvable untilpd-edge-abi 0.2.0existed — and that only happens by publishing it from that same workspace:pd-edge-host-function 0.2.0 already published; skipping(it had gone out in the earlier run), so the job died on abi after ~6 minutes of pointless retries.--no-verifydoes not help: it skips the build, not resolution.Fix (commit b1dd164):
.github/scripts/prepare_publish.pycopies each released crate into its own standalone tree and rewrites that copy:path/git/rev/branch/tagare dropped,[workspace.package]inheritance (version,edition, …) is materialised, workspace crates get the release version, external crates get explicit registry versions (pd-vm/pd-host-function= 0.22.2), anddev-dependenciesare removed (a crates.io consumer never builds them;reqwestmust not gate a release). Any remaining path/git dependency without a known registry version fails the job loudly instead of shipping an unresolvable manifest.pd-edge-host-function→pd-edge-abi→pd-edgeregardless of the declared order..github/scripts/publish_crates.pypublishes withcargo publish --manifest-path <staged>/Cargo.toml, checks versions against the sparse index (index.crates.io) rather than only the crates.io API — the API answers before Cargo can see a version, which is the other way this exact error appears — and confirms a dependency is indexed before publishing its dependent. Already-published versions are skipped, and the deprecated--tokenflag is replaced byCARGO_REGISTRY_TOKEN.publish-crates.ymlnow only derives the version and calls the two scripts; the rustscript checkout is gone because no published manifest references a path outside the crate.Verification
python3 -m unittest discover -s .github/workflows -p test_publish_crates.py -v→ 11 tests, OK. Fixtures under.github/scripts/fixtures/pd-edge-0.2.0/are verbatim copies of the 0.2.0 manifests and reproduce the failure shape: they assertpd-edge-abidepends on no crate released after it, thatpd-edgeresolvesedge_abito{ package = "pd-edge-abi", version = "0.2.0", default-features = false }with nopath, that[dev-dependencies]is gone, and that a reversed order is re-sorted to host-function → abi → pd-edge.python3 -m py_compileon both scripts and the test file: OK.yaml.safe_loadonpublish-crates.yml: OK.bash -non bothrun:blocks: OK.git diff --check: OK.cargo publish --dry-run --no-verify --manifest-pathon the staged trees for 0.2.0:pd-edge-abi:Packaged 22 files, 121.8KiB→aborting upload due to dry run(exit 0). This is the exact step that failed 18× in CI.pd-edge-host-function: exit 0 (crate already on crates.io).pd-edge: still reportsfailed to select a version for the requirement pd-edge-abi = "^0.2.0"— expected before abi is published, and precisely why the publisher now completes abi and waits for its index entry before touching pd-edge.pd-edge* 0.2.0for abi/pd-edge; the next tag run is the real end-to-end test.