Skip to content

fix: give every tool babylon's manifests as committed - #213

Merged
lukekarrys merged 2 commits into
mainfrom
lk/workspace-protocol-parity
Nov 18, 2026
Merged

lukekarrys merged 2 commits into
mainfrom
lk/workspace-protocol-parity

Conversation

@lukekarrys

@lukekarrys lukekarrys commented Nov 18, 2026 •

Copy link
Copy Markdown
Member

Fixes #210 (babylon: only vlt gets workspace:* manifests).

What changed

  • Deleted suite/scripts/add-workspace-protocol.js and emptied BENCH_SETUP_VLT. Every tool now installs babylon exactly as committed.
  • clean_workspace_protocol stays, with a new comment: it now undoes vlt's own manifest rewrite between tools.
  • METHODOLOGY and FIXTURES/BABYLON.md say how each tool links babylon's siblings.

Answers to the issue's questions

  1. Does vlt still need the rewrite? No. vlt 1.8.0 migrates plain sibling ranges to workspace:* itself on install: 80 package.json files on babylon, 257 of the 323 sibling specs. A second install on the migrated manifests changes nothing. Correction (see the review): vlt 1.8.0 leaves the 66 exact and peer specs on the published @babylonjs/* names as they are, and in about half of cold installs resolves four of them to registry copies instead of the workspace (install: workspace migration skips exact/peer specs on published workspace names, and they resolve to registry copies in ~half of cold installs vltpkg#2154). METHODOLOGY now says so.

  2. Rewrite for every tool? Not needed once no tool gets it.

  3. Why does pnpm look up siblings? pnpm 12 does read linkWorkspacePackages: true and links the siblings (295 link: entries in the lockfile). It still asks the registry about each plain-range sibling first. Cold, behind a logging proxy:

    manifests requests sibling lookups
    plain ranges (as committed) 1,760 240 (223 × 404, 17 × 200)
    workspace:* 1,520 0

    pnpm also installs @babylonjs/shared-ui-components from the registry, because it is a peer dependency of smartFiltersEditorControl. The audit's probe-ws failure was a probe whose pnpm-workspace.yaml lacked linkWorkspacePackages: true. With one that sets linkWorkspacePackages: true, as babylon has, pnpm installs it.

Every tool, vlt included now, downloads babylonjs-gltf2interface@6 from the registry. packages/tools/devHost asks for ^6.42.0 and the workspace has 8.49.6, so the registry is the correct source.

Expected change in published babylon numbers

Only vlt's numbers should move:

  • Where manifests start out unmigrated (clean; registry-* uses npm as the client), vlt's timed run now includes writing the 80 package.json files.
  • vlt now downloads one more package, babylonjs-gltf2interface@6.

The other tools already got the committed manifests, so their inputs don't change. pnpm's ~240 sibling lookups remain: they are its default behaviour.

Testing

🤖 Generated with Claude Code

add-workspace-protocol.js rewrote babylon's plain sibling ranges to
`workspace:*`, in vlt's setup only. vlt 1.8 does that rewrite itself on
install (--migrate-workspaces is on by default), so the script now only
handed vlt an adjusted project. Drop it; clean_workspace_protocol still
restores the manifests vlt migrates.

pnpm's registry lookups for siblings are its own default: with plain
ranges it queries the registry for each one (240 lookups cold, 223 of them
404) before linking it, and with `workspace:*` it makes none.
linkWorkspacePackages is read by pnpm 12. METHODOLOGY now says how each
tool handles babylon's siblings.

Fixes #210

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
benchmarks Ready Ready Preview Nov 18, 2026 3:27am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 18 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 8a4336b2-b5a0-46a5-8255-472e33281fb5
Commits

Reviewing files that changed from the base of the PR and between e15714c and ae6b983.

Files selected for processing: 5

  • docs/FIXTURES/BABYLON.md
  • docs/METHODOLOGY.md
  • suite/scripts/add-workspace-protocol.js
  • suite/scripts/clean-helpers.sh
  • suite/scripts/variations/common.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@lukekarrys

Copy link
Copy Markdown
Member Author

Review: the code is right; fix the docs, and decide on one vlt behaviour first

The change itself is correct and minimal. Removing add-workspace-protocol.js and clearing BENCH_SETUP_VLT gives every tool the committed manifests. On a local babylon run, every tool installed with exit 0. vlt's rewrite is fully undone between runs (0 modified files after each clean_all, and vlt never touches vlt.json), and the add/remove snapshots restore byte-identically. PR CI is green, and all 140 benchmarks.json exit codes are 0. Babylon doesn't run on PR pushes, so the babylon checks were local.

Decide before merging: vlt's sibling resolution on babylon flips between two states

With the manifests as committed, vlt 1.8.0's migration rewrites 257 of the 323 sibling specs. The 66 it leaves alone are exact and peer specs on the published @babylonjs/* names (e.g. "@babylonjs/gui": "8.49.6" as a devDependency in accessibility, gui-editor and inspector, and ^8.0.0 peers). In 5 of 10 cold installs, through a logging proxy against registry.npmjs.org, vlt resolved @babylonjs/gui, loaders, materials and serializers to registry copies for those packages and downloaded 4 extra tarballs. In the other 5 it linked the workspace copies. The start state was identical every time.

  • Downloads per run across repeated runs: 1, 5, 1, 5, 1, 5, 1. The real clean.sh run gave 5, with a package count of 1,605 against 1,601.
  • The lockfiles from the two kinds of run differ: in one, ~npm~@babylonjs+gui@8.49.6~peer.… nodes exist and the edges from accessibility, etc. point at them; in the other, the same edges point at workspace~packages+public+@babylonjs+gui.
  • Before this PR (workspace:* manifests), vlt downloaded 0 siblings in 3 of 3 runs.

This looks like a vlt bug (incomplete migration plus nondeterministic resolution), not a problem with this PR. But with this PR, vlt's published babylon clean results would alternate between two states: ±4 downloads, package count 1,601 or 1,605, and different timings. Options: merge as-is under the "default experience" principle and document it (plus a vlt issue), or wait for a vlt fix. Note this was measured against npmjs through a proxy, not registry.vlt.io (401 without CI's token), but it's a resolution issue, so it likely reproduces there.

Fix: docs and PR text

  • docs/METHODOLOGY.md ("vlt rewrites those ranges to workspace:*") and the PR body ("links every sibling") are inaccurate, given the above: 66 specs aren't rewritten, and four of them sometimes resolve to the registry. vlt's rewrite also changes unrelated fields: in 36 of the 80 files, "bugs": {"url": …} becomes "bugs": [{"type": "link", "url": …}]. That's harmless here (restored every run), but it's part of what vlt writes during the timed run, and worth reporting to vlt.
  • PR body, "Where manifests start out unmigrated (clean, registry-clean)": the registry-* variations run npm as the client (BENCH_COMMAND_VLT_REG uses $BENCH_NPM_INSTALL, suite/scripts/registry/common.sh:122), so vlt's migration isn't involved there. Only clean starts each timed run with unmigrated manifests. Elsewhere, the first (untimed) warmup migrates them.
  • PR body, "The audit's probe-ws failure was a probe without a pnpm-workspace.yaml": the probe had one (packages: ["packages/*"]), but without linkWorkspacePackages: true. The conclusion still holds: reproduced ERR_PNPM_FETCH_404 without the key, and link:../b with it.
  • docs/FIXTURES/BABYLON.md:19 still says babylon catches tools handling "workspace: protocol specifiers" badly, while two lines later no manifest uses workspace:. Suggest "cross-workspace ranges".
  • docs/METHODOLOGY.md:49, "with the same manifests for every tool": berry and zpm still set packageManager and write .yarnrc.yml, so "the same dependency specs" is more accurate.

Nits

  • clean_workspace_protocol now undoes vlt's migration, so its name is a little off. Optional rename.
  • Pre-existing, local only: after clean.sh, the process-count pass can leave vlt's migrated manifests behind. A local clean then ci sequence would give npm and yarn 1 workspace:* manifests. CI runs one variation per job, so it's not affected.

Verified

  • node --test suite/scripts/ (24 pass); no leftover references to add-workspace-protocol.
  • A local babylon clean for all 10 tools, and add-overlap for vlt and pnpm, in a CI-like sandbox through a logging proxy, with the audit's tool versions (vlt 1.8.0, pnpm 12.21.0, and so on).
  • pnpm's ~241 sibling lookups (224 of them 404s) remain. That's pnpm's default behaviour, as the PR says.

🤖 Reviewed with Claude Code

vlt 1.8.0 rewrites 257 of babylon's 323 sibling specs; the 66 exact/peer
specs on @babylonjs/* stay as-is, and about half of cold installs resolve
four of them to registry copies. Only `clean` times the migration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lukekarrys
lukekarrys merged commit 1a730a6 into main Nov 18, 2026
12 checks passed
@lukekarrys
lukekarrys deleted the lk/workspace-protocol-parity branch November 18, 2026 03:27
lukekarrys added a commit that referenced this pull request Nov 18, 2026
…225)

Since #213 every tool installs babylon's manifests as committed: plain
semver sibling ranges, which vlt 1.8.0 rewrites to `workspace:*` on its
first install. A vlt user's repo stays migrated after that install, so in
every variation but `clean`, vlt's --prepare now runs
`migrate_workspace_protocol` (clean-helpers.sh) before its first install
of a set: `vlt install --lockfile-only --view=silent`, which rewrites the
manifests with no node_modules and no tarballs; the lockfile it writes is
discarded, or a pre-existing one is restored byte for byte. Later prepares
find `workspace:` present and do nothing; clean_all undoes it between
tools. `clean` sets BENCH_SETUP_VLT empty, so the migration stays part of
the timed first install there. If --lockfile-only fails, the helper warns
and vlt's first install migrates as before.

METHODOLOGY documents where the migration happens and that it does not
make vlt's first warm babylon run like the later ones: vltpkg/vltpkg#2155
(optional `canvas` written to the lockfile but not installed on a first
install) means vlt installs 1,601 packages instead of 1,618 in the
no-lockfile variations, which slightly flatters vlt, and the first
install from a lockfile downloads those 17. No workaround in the suite.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

babylon: only vlt gets workspace:* manifests; other tools look up or download workspace siblings from the registry

1 participant