Repository navigation
Make check ids required, and rename SuiteSummary.verified to hasFailures - #69
Merged
ottonomy merged 2 commits intoOct 20, 2026
Conversation
`runSuites` now computes `<phase>.<suite>.<check>` for every row it writes — normal, not-run, and both kinds of synthetic `.applies` row — so `CheckResult.id` is required and the verifier's after-the-fact `populateCheckIds` pass is gone. The parse-failure row gets its `cryptographic.parsing.envelope` id where it is built, and the summary code's fallback for a missing id goes with it. `computeId` moves to its own module so the suite runner can use it without importing the summarizer. `SuiteSummary.verified` becomes `hasFailures`, true when any check in the suite failed. "Verified" now exists only at the top level, where it means "no fatal check failed" — a non-fatal failure no longer reads as `verified: false` inside a `verified: true` result. `CheckResult.suite` is no longer deprecated: it is how rows are grouped, it matches `SuiteSummary.suite`, and it cannot be recovered from `id`, because suite ids contain dots. `check` stays deprecated. Docs: record that check ids and the built-in suites' phase tags are fixed for 2.x, and complete the API guide's list of ids with the status list-issuer check, the parsing suite and the Open Badges suites' synthetic `.applies` rows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Oct 14, 2026
`hasFailures` was always `status === 'failure' || status === 'mixed'` and nothing read it, so it goes before the 2.0 shape freeze. Adding a field later is not a breaking change; removing one is. `fatalFailureAt` is now documented as the per-suite fatal signal. A fatal failure halts the subject, so at most one summary carries it, and a result is unverified exactly when one of its summaries does. `status` stays a plain count-based fact; the docs now tell UIs to colour the suite with `fatalFailureAt` as an error and other failing suites as warnings, so a non-fatal failure no longer reads as red inside a verified credential. Docs fixes in the same pass: - `verified` on both result types said "no check failed"; it means no fatal check failed. - The API guide's rendering example and prompt appendix showed a credential as unverified because of a registry failure, which is non-fatal. The examples now show a verified credential with warnings and an unverified one with a fatal signature failure. - `'mixed'` was documented as covering failure plus skipped; the code returns `'failure'` for that case. Tests cover a non-fatal custom failure (suite fails, no `fatalFailureAt`, still verified) and the one-fatal-summary invariant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #46
Stacked on #68 — review and merge that first (or merge the stack together).
Departures
None. Everything in "What to build" landed: ids stamped in the suite runner and
required on the type, the after-the-fact population removed, the summary's id
fallback removed,
suiteun-deprecated (checkstill deprecated),SuiteSummary.verifiedrenamed tohasFailures, the stability statement aboutcheck ids and built-in phase tags written down, and the documentation sweep
(type comments, API guide, timing guide, README, CHANGELOG).
Two small extras inside the ticket's intent, both noted below: the API guide's
example gained the
status.list-issuerrow it was missing, and one line ofdocs/architecture.mdthat spells out theCheckResultshape now listsid.Decisions not in the issue
computeIdmoved tosrc/compute-id.ts. The suite runner now needs it,and
fold-results.tsalready importsNOT_RUN_REASON_PREFIXfromrun-suites.ts, so importing the other way would have closed a cycle. The newmodule is where the "ids are API, fixed for 2.x" note lives.
fold-results.tsre-exportscomputeId, so the barrel export and everyexisting import path are unchanged.
status, registry, recognition, parsing, then the three Open Badges bundles),
not suite-execution order — smaller diff, same content.
hasFailuresiscounts.failed > 0, i.e. any failure, fatal or not. Theissue's wording ("true when any check in the suite failed") and the old
field's inverse both read that way.
cryptographic.status.list-issuerrow, because adding the missing id to the reference list while the worked
example still showed the status suite with one check would have left the two
disagreeing. Its summary row now reads "2 of 2 checks skipped".
docs/architecture.mdline 313 (CheckResult: { suite, check, outcome, … }) now includesid. The file is not in the ticket's documentation list,but that line states the shape of the type this change makes required; leaving
it would have been a known-wrong line.
foldCheckResultsneed ids now. Rather than thread thephase through every call site,
test/fold-results.spec.tshas a smallFIXTURE_PHASESmap from fixture suite id to phase, somkResultstamps thesame id the runner would — including
custom, which is deliberately absentfrom the map so it produces
unknown.custom.x.test/run-suites.spec.tscovering the acceptanceitem that the runner stamps every row: normal and not-run rows, an untagged
suite (
unknown.prefix), the synthetic.appliesskip, and the.appliesfailure from a throwing predicate. The parse-failure id was already covered by
an existing test in
test/verify-credential.spec.ts.Agent-Run: d76a817e-79b3-4e1c-bc5f-101e5d4c866e