Repository navigation
test: run the analyzer sweeps' rows in parallel, and leave the pragma pass to each rule's focused tests - #250
Merged
CaffeinatedCoder merged 1 commit intoOct 31, 2026
Conversation
… pass to each rule's focused tests DroppedTaskVariantTests is one class of 1,838 rows, which xunit's default mode ran one after another. On CI it kept the test step going for two to seven minutes after every other project had finished (#245). The analyzer test project now runs with parallelMode "all" (xunit.runner.json), so every row of every theory is scheduled on its own. No test there shares state: each builds its own compilation. The four sweeps (DroppedTaskVariantTests, TypeArgumentConstraintKindTests, QueryDelegateConditionTests and TypeArgumentSiteRun, which TypeArgumentConstraintVariantTests runs) set VariantSweep.Behaviors, which skips the verifier's second run of each reporting case under '#pragma warning disable'. Whether a pragma silences a rule is the same for every shape, and the focused tests of each rule (IgnoredOutcomeAnalyzerTests, TypeArgumentConstraintAnalyzerTests, DefaultProducingCallAnalyzerTests) keep it. The verifier's generated-code pass already never ran: every analyzer configures generated-code analysis. Seeded twice, each reverted by its inverse edit: - CMTK0003 dropping the first element of an array's tasks: DroppedTaskVariantTests fails 2 rows under the new mode, so the sweep still catches its own defects. - CMTK0003 reporting a probe rule only below a leading '#pragma warning disable': IgnoredOutcomeAnalyzerTests fails 16 tests in the pragma pass, DroppedTaskVariantTests passes. The pass is now the focused tests'. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #245
Problem
Since #172, a full CI run takes 17-19 minutes instead of about 10.
DroppedTaskVariantTestsis one class of 1,838 rows, which xunit's default mode (one collection per class) ran one after another. In the test results of runs 41180296321 and 41178154299, every other test of every project has finished by 334-453 s. The theory then runs alone on one core for another two to seven minutes. Each of its reporting rows also ran a second time under#pragma warning disable: the same check for every shape, 28% of the theory's time.Change
xunit.runner.jsonwith"parallelMode": "all"forCodoMetis.TypeKit.Analyzers.Tests, copied to the output by the project file, which says why. Every row of every theory is now scheduled on its own. The[assembly: Parallelization]attribute the xunit docs show is not in the pinned 4.0.2. No test in the project shares state (each builds its own compilation), so none opts out.VariantSweep.Behaviors(AnalyzerTest.cs):TestBehaviors.SkipSuppressionCheck, set by the four sweeps:DroppedTaskVariantTests,TypeArgumentConstraintKindTests,QueryDelegateConditionTestsandTypeArgumentSiteRun(TypeArgumentConstraintVariantTests). The focused tests of each rule (IgnoredOutcomeAnalyzerTests,TypeArgumentConstraintAnalyzerTests,DefaultProducingCallAnalyzerTests) keep the pragma pass. The verifier's generated-code pass already never ran here: it is skipped for an analyzer that configures generated-code analysis, and every CMTK analyzer does.Test code only: no package, no generated member, no changelog entry.
Tests
release/2.3.1, then 1m 29s and 1m 27s on this branch, with 3,947 passing on both. The theory alone at 4 threads, as on a CI runner: 186 s down to 70 s. This PR's own run gives the CI before and after on both legs, which go into CI: run the analyzer variant sweeps' rows in parallel, and skip the harness's rule-wide passes there #245.DroppedTaskVariantTestsfails 2 rows under the new mode, so the sweep still catches its own defects.#pragma warning disable:IgnoredOutcomeAnalyzerTestsfails 16 tests in the pragma pass, andDroppedTaskVariantTestspasses. The pass is now the focused tests', as intended.🤖 Generated with Claude Code