Repository navigation
✨ Native code coverage from Ceedling's gcov plugin, with gcovr and ReportGenerator (supersedes #13) - #26
Merged
Conversation
The Testing view's Run with Coverage button now runs the suite through Ceedling's gcov: task and reports the result through VS Code's own coverage UI - the Test Coverage panel, per-file percentages, and inline line and branch hit counts. The data was already there. Ceedling's gcov plugin writes a gcovr Cobertura report with per-line hit counts, per-branch condition coverage and, from newer gcovr, per-method entries; nothing consumed it. This wires that report to the coverage API VS Code finalized in 1.88. - src/gcovrCoberturaReport.ts: pure, vscode-free normalization of the parsed report, unit-tested in plain Node like ceedlingXmlReport.ts. Every path comes out absolute. gcovr writes <filename> relative to <source> inside its root but absolute outside it, and before gcovr 8.3 <source> is itself relative to where gcovr ran (gcovr#1037). Uri.file() puts a relative path at the filesystem root, so a path left unresolved shows no coverage at all. - ceedlingEngine: a coverage variant of the run entry points, and reading the report against the project directory Ceedling runs in. A missing or unreadable report is "no coverage", not a failed run. - testController: a non-default Coverage run profile, since a coverage run rebuilds instrumented and is slower. Each project's previous report is deleted before the run, so a failed build can't pass the last run's coverage off as this one's. The fixtures are real reports from Ceedling 1.1.6 with gcovr 5.0 and 8.6, of a probe project with one source inside the Ceedling project and one above it. Requires raising the minimum VS Code from 1.71.0 to 1.88.0, the release that finalized the test coverage API.
Jake Kim's pull request #13 added native test coverage. It closed before it could merge, and its fork is gone. Its commit survives at refs/pull/13/head, and this merge brings it in unchanged, with its author, date, and message. The pull request targeted master. next_version has since reworked the engine and controller it changed, so conflicts resolve to next_version's side: - ceedlingEngine.ts, testController.ts: next_version's code. The next commits port the coverage wiring onto it. - package.json: next_version's, with this commit's VS Code 1.88 engine, @types/vscode (~1.88.0, next_version's convention), and coverageCommandArgs setting. - package-lock.json: regenerated from the merged package.json. - README.md: next_version's, with this commit's coverage section, feature bullet, setting row, and 1.88 requirement. - CHANGELOG.md, BreakingChanges.md: this commit's entries, folded into the existing [Unreleased] sections. The new coverage module, its tests, and its gcovr fixtures arrive exactly as written. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… Ceedling 1.2.0 capture-gcov.sh runs Ceedling's gcov plugin against a probe project in the madsciencelab image, with that image's gcov, gcovr, and ReportGenerator. The probe has one source inside the Ceedling project and one above it, as the gcovr 5.0 and 8.6 fixtures' probe did. Its mixins select the configurations the extension must read: both utilities, several report types, a renamed gcovr report, and reports deferred to report:gcov. Committed captures: - gcovr-7.2/ and reportgenerator-5.5.11/cobertura-probe.xml, the Cobertura reports. - ceedling-1.2.0/dumpconfig-gcov-*.yml, the resolved configurations, plugin defaults included. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs coverage through the engine as it now works, carrying forward Jake Kim's design from #13: - a non-default Coverage profile; - the coverageCommandArgs setting; - a missing report meaning "no coverage", not a failed run; - a stale report deleted before each coverage build. What moves where: - Coverage state: projectState reads, from the resolved configuration, whether the gcov plugin is enabled, whether reports wait for report:gcov, and where each enabled utility writes its Cobertura report. Nothing is hard-coded to gcovr's default path. - Engine: runFile, runFunction, and runProjectAll take a RunMode. A coverage build happens under the project directory's lock, as every build does: - it deletes stale Cobertura reports; - it clears earlier counters once per request, unless coverageResetsCounters is off; - it runs gcov: with the bare file name, which Ceedling's gcov task requires on every version; - it adds report:gcov when reports wait for that task; - it reads the report before releasing the lock. A project without the gcov plugin runs its tests as usual, and says why there is no coverage. - Parser: reads the parsed XML as `unknown` rather than `any`, and gives each file its candidate paths. The engine picks the candidate that exists. - Controller: - offers the Coverage profile only while some project enables the gcov plugin; - merges a request's reports with mergeCoverage; - keeps each FileCoverage's details for loadDetailedCoverage; - shows why a run has no coverage. Jake's tests run unchanged against the ported parser, apart from their call sites and two lint fixes. Co-authored-by: Jake Jooncheol Kim <227114254+jake-kim-sndk@users.noreply.github.com> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Ceedling's gcov plugin writes the same Cobertura format through ReportGenerator, which is common on Windows. The coverage state already lists ReportGenerator's report locations after gcovr's: - ReportGenerator/Cobertura.xml for one report type; - ReportGenerator/Cobertura/Cobertura.xml for several. The module is renamed coberturaReport.ts with git mv, so its history follows, since it no longer serves gcovr alone. Tests against the captures from Ceedling 1.2.0: - ReportGenerator 5.5.11 lists every source directory it was given, and names files relative to the directory it ran in. Each file gets a candidate per source, and the engine keeps the one that exists. That resolves both files, the one inside the project and the one above it. - Its <method> entries become functions, and partly taken branches carry through. - gcovr 7.2 with the default :report_root: "." reports only sources within that root. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the fake Ceedling Unit tests: - a function run's whole-file fallback keeps the counters its first attempt produced; - each build directory has its counters cleared once per run; - a branch line without condition-coverage carries no branch outcomes; - a line without a number or hit count is left out. The fake Ceedling enables the gcov plugin from its scenario. Its gcov: tasks reject a path-qualified name, as Ceedling's do, and write the CppUnit report under artifacts/gcov and a gcovr Cobertura report of the covered sources. Integration tests, in VS Code: - a project with the gcov plugin offers Run with Coverage; - a coverage run reports its tests, plus line, branch, and function coverage for src/calc.c, and the details load through the profile's loadDetailedCoverage; - the Coverage profile goes away when the plugin does, and comes back with it. The controller's coverageRunProfile getter lets the tests reach the profile. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rator
README:
- A "Code coverage" section replaces the gcovr-only notes. It covers:
- setting up the gcov plugin with gcovr or ReportGenerator and a Cobertura report, in
project.yml or a mixin;
- how coverage runs work: counters cleared per run, gcovr preferred, report:gcov, several
projects;
- what each tool reports, including gcovr's :report_root: for sources above the project;
- what to check when no coverage appears.
- The options table describes both coverage settings. The testCommandArgs row loses its stale
utils:gcov example.
- The Gcov plugin link points at the documentation that exists.
CHANGELOG and BreakingChanges: the coverage entries in house style, crediting @jake-kim-sndk and
#13, with the minimum VS Code raised from 1.74.0, the version next_version actually required.
docs/Acknowledgments.md: the README's account of the extension's origin, verbatim, then Jake
Kim's coverage contribution. The README keeps its section and links to the new file.
Development: the module table and a Coverage section map the flow to code. The manual
checklist covers gcovr, ReportGenerator, per-run coverage, and a project without a Cobertura
report, using mixins in tests/manual/mixins/.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ified gcov from 1.2.1 Four parallel reviews covered every change on this branch. Each finding was checked against the code, and each behavior change was made test-first. Engine: - The whole-file fallback after a --test-case collision clears that test's own counters first, so its hits aren't counted twice. - A failed coverage build still reports a configuration that can't give coverage. - A report covering no sources says where gcovr looks. - Ceedling 1.2.0's gcov plugin still prepends the test file prefix to the whole name, unlike its test: tasks. A new pathQualifiedCoverageTasks capability passes paths from Ceedling 1.2.1. Before that, the log warns when same-named test files make a coverage run ambiguous. - Stale files are deleted without a stat apiece. ENOENT is not a failure. - keyOfPath gives the controller the engine's own path rules for merging. Parser: - Line numbers must be integers from 1, and hits integers from 0, since VS Code positions can't be negative. - Implausible branch counts are dropped rather than rejecting the report. - Sources are trimmed, and an empty <coverage/> covers nothing. - The XML helpers move to parsedXml.ts, shared with testResults.ts. Controller: - Coverage and problems are shown from finally, so a run that throws partway keeps what its completed builds measured. - Each problem notifies once a session, and every run still logs it. - A named CoverageReport type replaces four inline copies. Tests: - nodeFileSystem.listFiles on a real disk; - the Windows coverage path; - a fallback to a file's first candidate; - the 1.2.1 path-qualified tasks; - the parser's edge cases; - a coverage run in VS Code against a project without a Cobertura report. The fake Ceedling accepts paths in gcov: tasks from 1.2.1. Docs: - README: the how-to's accuracy (clean also clears counters, ReportGenerator's 0% rows, two coverage runs at once share counters, same-named test files), its notification causes, and style. - KnownIssues: same-named test files in coverage runs before Ceedling 1.2.1. - Development.md: the coverage flow and its wording. - The nested ReportGenerator path is backed by the capture's file listing. Deferred, with reason: keeping two coverage runs at once in one project from clearing each other's counters would need the engine to track when each request ends. That is rare, and the README documents it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #13. Adds native code coverage to the Testing view. Run with Coverage runs tests through Ceedling's gcov plugin, then shows each source file's line, branch, and function coverage in VS Code's coverage views. It reads the Cobertura report from gcovr or ReportGenerator.
Credit
The design and the first implementation are by @jake-kim-sndk (Jake Jooncheol Kim), in #13. His pull request closed so it could be retargeted, and his fork has since been deleted. His commit survived at
refs/pull/13/head, and it is preserved two ways:aa07df0, with his author, date, and message. A merge commit brings it in, so GitHub lists it under his name.archive/pr-13-gcov-coverage, pushed to this repo as a durable copy.The commit that ports his engine and controller wiring onto the reworked code carries a
Co-authored-bytrailer for him. CHANGELOG credits him in its existing "contributed by" style. The newdocs/Acknowledgments.mdcredits him after the README's account of the extension's origin.Commits
aa07df0aa07df0.next_versionreworked the engine and controller it changed, so conflicts resolve tonext_version's side. His new module, tests, and fixtures arrive exactly as written.capture-gcov.shplus captures from Ceedling 1.2.0 with its own gcovr 7.2 and ReportGenerator 5.5.11. Each tool's Cobertura report, anddumpconfigwith the gcov plugin's defaults.coberturaReport.ts, so its history follows. Several<source>roots resolve to the candidate path that exists. Real-capture tests.tests/manual/mixins/.Behavior
ReportGenerator/Cobertura.xml, orReportGenerator/Cobertura/Cobertura.xmlwith several report types..gcda, plus ReportGenerator's.gcovinputs), once per build directory. The newceedlingExplorer.coverageResetsCounterssetting turns that off to accumulate coverage across runs.:report_task: TRUE: the run addsreport:gcovto its own command line.gcov:task names: Ceedling 1.2.0's gcov plugin still takes only a bare file name, unlike itstest:tasks. That is reported as gcov: tasks don't accept project-relative paths, unlike test: tasks in 1.2.0 Ceedling#1362. Coverage runs pass the bare name before 1.2.1, warning when same-named test files make that ambiguous, and the path from 1.2.1 (pathQualifiedCoverageTasks).Minimum VS Code is now 1.88.0, up from 1.74.0. 1.88 finalized the test coverage API. Typechecking runs against
@types/vscode1.88.0, so no newer API is used.Verification
Final review
Four parallel reviews covered the engine and coverage state, the controller, the parser and merge, and the tests and docs. Each finding was checked against the code, and each behavior change was made test-first.
Fixed:
ceedling cleanalso clears counters, ReportGenerator lists earlier builds' sources at 0%, and same-named test files are ambiguous. Plus style.Deferred, with reason: two coverage runs at once in one project share its counters. Keeping them apart would need the engine to track when each request ends. That is rare, and the README documents it.
Manual checks
docs/Development.md§5 gains coverage checks for gcovr, ReportGenerator, per-run coverage, and a project without a Cobertura report.🤖 Generated with Claude Code