Skip to content

Migrate babylon's workspace ranges for vlt in an untimed setup step - #225

Merged
lukekarrys merged 1 commit into
mainfrom
lk/vlt-migrate-in-setup
Nov 18, 2026
Merged

lukekarrys merged 1 commit into
mainfrom
lk/vlt-migrate-in-setup

Conversation

@lukekarrys

Copy link
Copy Markdown
Member

What

Since #213 every tool gets babylon's manifests as committed: 86 workspace packages with plain semver sibling ranges. vlt 1.8.0 rewrites 257 of the 323 sibling ranges to workspace:* on its first install. A vlt user's repo stays migrated after that install (the rewritten manifests get committed), so in the warm variations vlt should start from already-migrated manifests, and in clean the migration should stay part of the timed first install.

  • suite/scripts/clean-helpers.sh: new migrate_workspace_protocol (the inverse of clean_workspace_protocol), plus its case entry and help line. No-op without vlt, without packages/, or once any manifest already has "workspace: (a grep, ~8 ms). Otherwise it runs vlt install --lockfile-only --view=silent, which rewrites the manifests exactly as a full install does (80 package.json files, plus 36 unrelated "bugs" rewrites, a vlt quirk) with no node_modules and no tarballs (~17 s cold). The lockfile it writes is discarded, or a pre-existing vlt-lock.json is put back byte for byte. If --lockfile-only fails (e.g. registry/token trouble in CI) it warns and returns 0, and vlt's first install migrates as part of its timed run, as today.
  • variations/common.sh: BENCH_SETUP_VLT="bash $BENCH_SCRIPTS/clean-helpers.sh migrate_workspace_protocol". That is appended to vlt's --prepare only (append_setup, set_per_pm_prepares, ci_prepare, ci_setup_prepare, prepend_setup in build/run, and add-remove-state.sh's baseline), never to hyperfine's shared --setup.
  • variations/clean.sh: BENCH_SETUP_VLT="" right after sourcing common.sh, before collect_process_count reads it, so the process-count pass matches the timed runs.
  • registry-* use npm as the client and don't read BENCH_SETUP_VLT.
  • docs/METHODOLOGY.md: the babylon paragraph now says where the migration happens, and what it doesn't fix (below).

What this does not fix

The migration does not make vlt's first warm run on babylon identical to later ones. That comes from vltpkg/vltpkg#2155: on a first install without a lockfile, vlt writes the optional dependency canvas (17 packages with its tree) to the lockfile but doesn't install it; the next install, from that lockfile, does. So:

  • in the no-lockfile variations (clean, cache, node_modules and their combinations without a lockfile) vlt installs 1,601 packages where its own lockfile and the other tools have 1,618. That slightly flatters vlt until #2155 is fixed;
  • in the lockfile variations the first install from the lockfile downloads those 17.

No workaround in the suite: forcing canvas in would make vlt behave unlike vlt. The existing #2154 note (sibling resolution flips in installs without a lockfile) stays; it is independent of the migration.

Per-variation effect

Variation Before vlt's first install of a set Later prepares
clean nothing: migration is timed in every run (unchanged) –
cache, node_modules, cache+node_modules migration; lockfile discarded (these start without one) grep no-op
lockfile, cache+lockfile, lockfile+node_modules, cache+lockfile+node_modules migration before the warmup that creates the lockfile; no lockfile exists yet, so nothing to restore grep no-op
ci, ci+setup migration after BENCH_CLEAN_PRE, before the untimed lockfile-generating install grep no-op (the untimed install runs every prepare regardless)
add-*, remove-* migration once, in the baseline build before the snapshot snapshot restore
build, run, build-cache fixtures have no packages/: no-op –
registry-* npm client: not applicable –

clean_all (--setup/--cleanup, and between tools) restores the manifests with clean_workspace_protocol, so the next tool starts from the project as committed.

Verified

  • node --test suite/scripts/: 46 pass, including four new migrate_workspace_protocol tests with a fake vlt on PATH (manifest rewritten and lockfile gone; pre-existing lockfile kept byte-identical; no-op without packages/; no-op when "workspace: is present, including workspace: specs inside a workspace's node_modules, which don't count).
  • Real runs on babylon, vlt only (BENCH_INCLUDE=vlt BENCH_WARMUP=1 BENCH_RUNS=2), through scripts/benchmark.sh with the fixture's vlt.json pointed at registry.npmjs.org (not committed), in a sandbox with a fresh tmpfs HOME. A logging vlt wrapper recorded every vlt call with the fixture state (ws_specs = workspace:* count):
    • cache+lockfile: install --lockfile-only runs once right after --setup (ws_specs 0 -> 257, no lockfile left), then the warmup install creates the lockfile, then both timed runs start from it with the prepare a no-op. The process-count pass repeats that once for its own set. Timed runs 6.5 s / 6.3 s; the warmup from migrated manifests 10 s.
    • clean: zero --lockfile-only calls; every install starts at ws_specs=0, 21 s each.
    • After each variation git status shows the manifests restored, and no vlt-lock.json, node_modules or .bench-vlt-lock.json left behind. All exit codes 0 (hyperfine, both variations, and both scripts).
    • The package counts show #2155 as documented: clean 1,601; cache+lockfile min 1,601 (first install, no lockfile) / max 1,618 (from the lockfile).

🤖 Generated with Claude Code

Since #213 every tool installs babylon's manifests as committed: plain
semver sibling ranges, which vlt 1.8.0 rewrites to `workspace:*` on its
first install. A vlt user's repo stays migrated after that install, so in
every variation but `clean`, vlt's --prepare now runs
`migrate_workspace_protocol` (clean-helpers.sh) before its first install
of a set: `vlt install --lockfile-only --view=silent`, which rewrites the
manifests with no node_modules and no tarballs; the lockfile it writes is
discarded, or a pre-existing one is restored byte for byte. Later prepares
find `workspace:` present and do nothing; clean_all undoes it between
tools. `clean` sets BENCH_SETUP_VLT empty, so the migration stays part of
the timed first install there. If --lockfile-only fails, the helper warns
and vlt's first install migrates as before.

METHODOLOGY documents where the migration happens and that it does not
make vlt's first warm babylon run like the later ones: vltpkg/vltpkg#2155
(optional `canvas` written to the lockfile but not installed on a first
install) means vlt installs 1,601 packages instead of 1,618 in the
no-lockfile variations, which slightly flatters vlt, and the first
install from a lockfile downloads those 17. No workaround in the suite.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Nov 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
benchmarks Ready Ready Preview Nov 18, 2026 9:00am UTC

Request Review

@coderabbitai

coderabbitai Bot commented Nov 18, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 44 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: c98ba872-9994-4ce6-950b-33318e405b69
Commits

Reviewing files that changed from the base of the PR and between 0b3a2dc and 6a49475.

Files selected for processing: 5

  • docs/METHODOLOGY.md
  • suite/scripts/clean-helpers.sh
  • suite/scripts/clean-helpers.test.js
  • suite/scripts/variations/clean.sh
  • suite/scripts/variations/common.sh
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@lukekarrys
lukekarrys merged commit 6c6c64a into main Nov 18, 2026
78 checks passed
@lukekarrys
lukekarrys deleted the lk/vlt-migrate-in-setup branch November 18, 2026 09:20

This branch was previously deployed

1 inactive deployment
Benchmark — 6a494750 Deployed Nov 18, 2026 by lukekarrys via Run Benchmarks (build-cache, next) #1467
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant