Skip to content

Take Vitest 4.1.12 for the last dev-tool alerts in #37 - #41

Merged
threeqube merged 1 commit into
mainfrom
vitest-4
Nov 2, 2026
Merged

threeqube merged 1 commit into
mainfrom
vitest-4

Conversation

@threeqube

@threeqube threeqube commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Part of #37 (PR 3 of 3). It leaves #37 open: the @stablelib/ed25519 alert waits on digitalcredentials/verifier-core#84.

What changes

The test runner, Vitest, goes from 3.2.7 to 4.1.12: the version the vitest and @vitest/mocker alerts name as fixed. 4.1.12 is the newest 4.x and has been out since 15 October. Vitest 5 is out, but two major versions in one step is a bigger change to the runner the consistency suite depends on, and 5.1.0 was published on the day of this change.

package.json: vitest ^3.2.4 → ^4.1.12. In package-lock.json:

Packages
Updated vitest and its own @vitest/* packages 3.2.7 → 4.1.12; chai 5.3.3 → 6.3.0, es-module-lexer 1.7.0 → 2.3.2, std-env 3.10.0 → 4.3.0, tinyexec 0.3.2 → 1.3.1, tinyrainbow 2.0.0 → 3.2.0
Added obug 2.2.1, @standard-schema/spec 1.1.0 (both for Vitest 4); @vitest/mocker 4.1.12 moves to the top level
Removed tinypool (Vitest 4 no longer uses it); both vite 7 copies and vite-node (Vitest 4 runs on our own vite 8); esbuild and rollup with their 52 per-platform binaries, which only vite 7 needed; eight small Vitest 3 helpers

That's 3 added, 67 removed, 12 updated, and nothing else: no package outside Vitest's tree moved. One entry is held on purpose: rollup pinned @types/estree to exactly 1.0.9, and with rollup gone npm picked 1.0.10, published four days earlier. It's type definitions only, unrelated to this change, so it stays at 1.0.9 (npm install leaves it there).

Vitest's own packages are published by its GitHub Actions, with npm provenance; the other new versions by their usual publishers, all at least four weeks old. Every changed entry's checksum matches the npm registry.

This should close the last 5 dev-tool alerts: tinypool #5, #6 (critical), vitest #3, @vitest/mocker #2, #20. Only @stablelib/ed25519 #1 stays open.

Nothing ships (verified)

dist/ built on this branch is byte-identical to dist/ built on main: all 12 files. Vitest only runs the tests.

What was exposed

Developer machines and CI only. Read against our setup (reasoned from the advisories and our code; not tested):

  • tinypool (critical): needs something else in the test process to have polluted Object.prototype first, through a separate bug fed attacker-controlled input. Our tests run on fixtures in this repo.
  • @vitest/mocker: needs a dev server using its mockerPlugin or interceptorPlugin that an attacker can reach. Nothing here uses either, or Vitest's browser mode; the browser tests use Playwright.

The tests still mean the same thing

The risk in a test-runner upgrade is that tests quietly stop catching mistakes. Checked against Vitest 3 on main:

  • Same tests: all 105,720 pass, and every test name, result and per-file count is identical.
  • Same failures: four deliberate mistakes in src/outcomes.ts (reassurance in a "couldn't check", naming an unconfirmed issuer, addressing a nameless issuer by identifier, a green "Expired") fail the same tests in the same numbers on both: 34,004, 1, 24 and 1,823.
  • Same speed: each consistency file about 7.5 s on both; the slowest single test about 0.4 s, well inside the 5 s limit, which is unchanged.

The 60-second limit

The consistency suite pauses after each test because Vitest 3 fails a run, every test passing, when a long file's messages wait 60 seconds ("Timeout calling onTaskUpdate"). Vitest 4.1.12 sets no time limit on those messages (its worker connection has timeout: -1; Vitest 3's had 60 s). Slowing one part to 83–88 s, twice: Vitest 3 without the pause fails with exactly that error; Vitest 4.1.12 passes with and without it.

The pause stays in case a later Vitest brings the limit back. It adds about half a second to the suite (7.6 s against 7.1 s). Its comment and the README row now say which version it was for. These are the only changes outside the dependency files.

Checks

  • Lint, typecheck, 105,720 unit tests on Vitest 4.1.12. 60 browser tests (1 skipped: it runs only against a published copy, which CI's preview job does). Run locally on Node 26; CI runs Node 24.
  • Reviewed independently, in fresh copies (git archive, npm ci):
    • Lockfile: every entry's tarball, checksum, dependencies and engines rechecked against the registry. npm ci installs 468 packages and leaves it untouched, as does a following npm install. Re-running the upgrade from main's lockfile gives the same file except the held @types/estree.
    • Alerts: no version in any open alert's range is left.
    • Builds: byte-identical by sha256.
    • Tests: the same 105,720 tests; three deliberate mistakes of the reviewer's own fail the same tests on both versions; the 5 s and 30 s limits still apply.
    • The 60-second limit: reproduced as above, and traced to timeout: -1.
    • Findings: the comment's first wording contradicted itself and said the pause cost nothing; both fixed.
    • Not this PR's: the test files themselves are never type-checked (tsconfig.json covers src only). The same 18 errors appear when they are, on Vitest 3 and 4 alike, so this upgrade added none, but CI wouldn't have caught it if it had.
  • Smoke test, comparing the live demo (main) with this PR's preview. Expected, written before running: no case differs, since the built plugin is byte-identical.
    Result: 0 of 21 differ (12 demo buttons, 9 vc-test-fixtures links), after a noise check of the live demo against itself (also 0 of 21). The preview is f745639 (main ae26bf8 + 961a0a6), and once its own address and the file names that follow from it are swapped back, its JavaScript is byte-identical to the live demo's. By hand in Safari, the same on every case compared. CI passed three times out of three (105,720 unit tests, 60 browser tests, 58 against the preview; Node 24).

Reaching the wallet

Nothing. The built plugin doesn't change, so there's nothing for a pin bump to pick up.

🤖 Generated with Claude Code

Vitest 3 to 4.1.12, the version the vitest and @vitest/mocker alerts name.
Vitest 4 drops tinypool and runs on our own vite 8, so tinypool and the two
vite 7 copies leave the lockfile. @types/estree stays at 1.0.9: rollup's exact
pin went with it, and the newer one isn't this change's business.

The consistency suite's pause after each test was for Vitest 3's 60-second
message limit. On Vitest 4 a slowed part passes without it; the comment and
README say so, and the pause stays.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 2, 2026

Copy link
Copy Markdown
Contributor

Try it: https://digitalcredentials.github.io/verifier-plugin/pr-41/

Updates on every push to this PR, and goes away when it closes. The browser tests run against this copy too.

@threeqube

Copy link
Copy Markdown
Collaborator Author

Smoke test

The live demo (main at ae26bf8) against this PR's preview (pr-41 at f745639, which is main plus 961a0a6).

Expected, written before running: no case differs, since the built plugin is byte-identical.

The code served: the preview's JavaScript is byte-identical to the live demo's once its own address (/pr-41/) and the file names that follow from it are swapped back.

Automated, with a local compare script at phone width, comparing every line the verifier-plugin box shows:

  • Noise check first, the live demo against itself: 0 of 21 differ.
  • Live demo against pr-41: 0 of 21 differ (12 demo buttons, 9 vc-test-fixtures links).

CI, three runs: all passed, each with 105,720 unit tests and 60 browser tests (1 skipped), on Node 24. 58 browser tests against the published preview passed (3 skipped). No "Timeout calling onTaskUpdate".

By hand, in Safari (a browser the automated runs don't use): live against pr-41 side by side. The Details tab on 11 of the 12 demo buttons (Verified seen in each page's log, code=verified checks=5), Developer view on 7, JSON on 3. The same on both sides every time: headline, explanation, advice, all five rows, and the log's code and checks.

@threeqube
threeqube merged commit 94675b0 into main Nov 2, 2026
10 checks passed
@threeqube
threeqube deleted the vitest-4 branch November 2, 2026 23:03
github-actions Bot added a commit that referenced this pull request Nov 2, 2026
@threeqube

Copy link
Copy Markdown
Collaborator Author

Post-merge check (2 November 2026): the live demo now, against the last build before #39. All expectations, written down before running, were met: no change in behaviour, speed or size.

  • Before: main at 713b8b4 (published 29 October).
  • After: main at 94675b0, live now (this PR's merge). This PR left the built plugin byte-identical, so for this PR the check mainly confirms that what's published is what was tested, and that the vulnerable test tools are gone from the lockfile.
Check Expected Result
Live is what was tested Live serves the published build of 94675b0 version.txt, index.html and both JavaScript files match gh-pages 2a3c0b1 byte for byte
Behaviour, Chromium 0 of 12 demo buttons and 0 of 9 vc-test-fixtures links differ 0 of 21; button text identical across all 15 rounds
Behaviour, WebKit Same 0 of 21; identical across all 15 rounds
Page open to first verdict Within noise (about 10%) Chromium 146 → 146 ms; WebKit 169 → 172 ms (medians, 15 rounds)
All 12 buttons, click to verdict Within noise Chromium 52.2 → 52.9 ms; WebKit 69.0 → 69.0 ms
The 9 links (live GitHub, registries, status lists) Within noise 7.35 → 7.42 s summed medians, 6 rounds each (1%, network-bound)
verify() in Node, 150 credentials × 15, network replayed, clock frozen Within noise +0.9% and −0.4% in two runs; per-credential median ratio 1.001 and 0.999
Download size, index JS Within about 1% Brotli 182,255 → 182,153 bytes; gzip 277,916 → 277,964
Patched versions on main As in #37 vite 8.3.5, undici 6.29.1 and 7.30.1, shell-quote 1.11.0, source-map-js 1.2.2, fast-uri 3.1.9, Vitest 4.1.12, tinypool gone; only Dependabot alert #1 (@stablelib/ed25519, waiting on verifier-core#84) is open

Method: both builds were served at the real address from gh-pages history (the built plugin refuses local addresses), each round opened both fresh and alternated which went first, and timings were taken inside the page. The timings come from one Mac; Firefox wasn't run; the wallet isn't covered until its verifier-plugin pin is bumped.

@threeqube

Copy link
Copy Markdown
Collaborator Author

In the wallet: lcw-sandbox.org (9 November 2026)

All clear. This PR has now reached the wallet. James deployed lcw-sandbox.org with lcw-front-end 0034b08, which pins release at a2a2181, the build of main 7c57591. That build includes #31, #33, #38, #39, #41, #42 and #43. I wrote the expectations down before checking.

  • Right build: a2a2181's dist/ is byte-identical to a fresh build of 7c57591, and its wording is in the live wallet's JavaScript.
  • Same behaviour as main: I gave the deployed <verifier-credential> on lcw-sandbox.org and the live demo (also 7c57591) the same 107 credentials and the same registries: the 12 demo situations, 4 built for No status behind a seal that didn't hold #33 and Say when a seal can never be checked, instead of "try again" #42, and the 91 in vc-test-fixtures. All 107 gave the same verdict, severity and card text.
  • Signed in to the wallet (the deployed smoke test's demo account, read only): the LCW Experience Badge reads "Verified", with the issuer "a known issuer" and Tampering "none detected", at desktop and phone widths. The card logs no console errors of its own. The only errors are the known ones: the OpenID Federation registries block browser requests.
What this PR changed In the wallet
Dev tools that build and test the plugin; nothing earners should notice ✅ the deployed build matches a fresh build of 7c57591 and the demo on all 107 credentials

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant