Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,20 @@ This format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/)

---

# [Unreleased]

## ✨ Added

Native test coverage. The Testing view's **Run with Coverage** button runs the suite through Ceedling's `gcov:` task and reports the result through VS Code's own coverage UI: the Test Coverage panel, per-file percentages, and inline line and branch hit counts. Function-level entries follow when gcovr populates `<method>` — gcovr 5.0 leaves it empty, 8.6 fills it in.

New `ceedlingExplorer.coverageCommandArgs` setting, for projects whose coverage task is not the default `gcov:${TEST_ID}`.

## ⚠️ Changed

Minimum VS Code version is now 1.71.0 → **1.88.0**, the release that finalized the test coverage API this feature is built on.

---

# [2.0.1] — 2026-08-31

## ⚠️ Changed
Expand Down
14 changes: 13 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,13 @@ Ceedling and its complementary [ThrowTheSwitch] pieces and parts are and always
* Shows a failed test’s message and failing line directly in the Test Explorer and the editor.
* Runs a single test function in isolation, without running the rest of its file.
* Can be configured to report compiler and linker problems inline in the editor and in the Problems panel.
* Reports code coverage through VS Code’s built-in coverage UI, from Ceedling’s gcov plugin.

# Requirements

## VS Code

Requires VS Code 1.71.0 or later.
Requires VS Code 1.88.0 or later.

## Ceedling

Expand Down Expand Up @@ -67,6 +68,16 @@ Run and debug tests from the Testing view, or from the gutter next to a test in

The Testing view’s toolbar also has Clean and Clobber buttons. They run `ceedling clean` and `ceedling clobber` against the current project.

# Coverage

The Testing view’s **Run with Coverage** button runs the same tests through Ceedling’s `gcov:` task and reports the result in VS Code’s own coverage UI: the Test Coverage panel, per-file percentages, and — with Toggle Inline Coverage — line and branch hit counts in the editor.

It reads the report Ceedling’s [gcov plugin](https://throwtheswitch.github.io/Ceedling/plugins/gcov/) writes to `<build_root>/artifacts/gcov/gcovr/GcovCoverageCobertura.xml`. Configure that plugin as its own documentation describes; this extension only requires that `gcovr` is among its `:utilities:` and `Cobertura` among its `:reports:`.

Function rows in the Test Coverage panel depend on your gcovr version. gcovr 5.0 writes an empty `<methods/>` for every file, leaving only line and branch coverage; 8.6 populates it, and the rows appear with no change here.

Coverage is not the default run profile — a coverage run rebuilds instrumented, so it stays opt-in behind its own button. If your project’s coverage task is not `gcov:${TEST_ID}`, override `ceedlingExplorer.coverageCommandArgs`.

# Configuration

## Options
Expand All @@ -80,6 +91,7 @@ This table of options should be read as `ceedlingExplorer.<property>`.
| `.prettyTestLabel` | Shortens the test label in the _Testing_ explorer by dropping its leading prefix (e.g. inactive `test_BlinkTaskShouldToggleLed` vs. active `BlinkTaskShouldToggleLed`).<br><br>Inactive:<br><img src="img/prettyTestLabelInactive.png" alt="prettyTestLabelInactive" width="420"><br><br> Active:<br><img src="img/prettyTestLabelActive.png" alt="prettyTestLabelActive" width="420">|
| `.prettyTestFileLabel` | Shortens the test file label in the _Testing_ explorer by dropping its path, leading prefix, and file type (e.g. inactive `test/LEDs/test_BlinkTask.c` vs. active `BlinkTask`).<br><br>Inactive:<br><img src="img/prettyTestFileLabelInactive.png" alt="prettyTestFileLabelInactive" width="320"><br><br> Active:<br><img src="img/prettyTestFileLabelActive.png" alt="prettyTestFileLabelActive" width="320"> |
| `.testCommandArgs` | The command line arguments used to run Ceedling tests. The first argument has to literally contain the `${TEST_ID}` tag. The value `["test:${TEST_ID}"]` is used by default. For example, the arguments `"test:${TEST_ID}", "gcov:${TEST_ID}", "utils:gcov"` can be used to run tests and generate a gcov report. |
| `.coverageCommandArgs` | The command line arguments used by the Testing view's **Run with Coverage** button. The first argument has to literally contain the `${TEST_ID}` tag. The value `["gcov:${TEST_ID}"]` is used by default. See [Coverage](#coverage). |
| `.problemMatching` | Configuration of compiler/linker problem matching. See the [Problem matching](#problem%20matching) section for details. |
| `.testCaseMacroAliases` | An array of aliases for Unity’s parameterized test `TEST_CASE` macro. By default it is `["TEST_CASE"]`. |
| `.testRangeMacroAliases` | An array of aliases for Unity’s parameterized test `TEST_RANGE` macro. By default it is `["TEST_RANGE"]`. |
Expand Down
12 changes: 12 additions & 0 deletions docs/BreakingChanges.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,18 @@ Complemented by three other documents: [../CHANGELOG.md](../CHANGELOG.md), [Rele

---

# [Unreleased]

## Minimum VS Code version is now 1.88.0

This extension now requires VS Code 1.88.0 or later, up from 1.71.0. 1.88.0 is the release that finalized VS Code's test coverage API, which the new **Run with Coverage** support is built on.

**What changes for you:**
- On VS Code 1.88.0 or later, nothing. Coverage is additive and opt-in; every existing behavior and setting is unchanged.
- On anything older, VS Code will not offer this version. It keeps the last version you can run, which continues to work.

---

# [2.0.0] — 2026-08-31

## Native Testing API replaces the Test Explorer UI extension dependency
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

15 changes: 13 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@
"devDependencies": {
"@types/glob": "^7.1.3",
"@types/mocha": "^8.2.0",
"@types/vscode": "^1.71.0",
"@types/vscode": "^1.88.0",
"@vscode/test-electron": "^3.1.0",
"@vscode/vsce": "^3.9.2",
"esbuild": "^0.28.1",
Expand All @@ -82,7 +82,7 @@
"typescript": "^6.0.3"
},
"engines": {
"vscode": "^1.71.0",
"vscode": "^1.88.0",
"node": ">=22"
},
"activationEvents": [
Expand Down Expand Up @@ -152,6 +152,17 @@
],
"scope": "resource"
},
"ceedlingExplorer.coverageCommandArgs": {
"description": "The command line arguments used to run Ceedling tests with coverage, from the Testing view's Run with Coverage button. The first argument has to literally contain the ${TEST_ID} tag. Requires Ceedling's gcov plugin with its 'Cobertura' report enabled.",
"type": "array",
"items": {
"type": "string"
},
"default": [
"gcov:${TEST_ID}"
],
"scope": "resource"
},
"ceedlingExplorer.problemMatching": {
"markdownDescription": "Configuration of mechanism that scans Ceedling output text for known error/warning/info strings and reports these inline in the editor and in the Problems panel.&nbsp;The mechanism tries to resemble the one used in VSCode Tasks problemMatchers.",
"type": "object",
Expand Down
69 changes: 59 additions & 10 deletions src/ceedlingEngine.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import {
testCaseFilterMatchedExactly,
} from './ceedlingOutputParsing';
import { getTestListFromXmlReport } from './ceedlingXmlReport';
import { CoveredFile, getCoverageFromCoberturaReport } from './gcovrCoberturaReport';
import { Logger } from './logger';
import { FileDiagnostic, ProblemMatcher, ProblemMatchingPattern } from './problemMatcher';

Expand Down Expand Up @@ -202,21 +203,22 @@ export class CeedlingEngine {
return this.getExistingProjectKeys().map((projectKey) => this.getYmlProjectPath(projectKey));
}

async runFile(projectKey: string, file: CeedlingFileNode, reporter: RunReporter, token: vscode.CancellationToken): Promise<void> {
await this.runInternal(projectKey, file.id, this.getTestCommandArgs(file.id), getAllIds(file), reporter, token);
async runFile(projectKey: string, file: CeedlingFileNode, reporter: RunReporter, token: vscode.CancellationToken, coverage = false): Promise<void> {
await this.runInternal(projectKey, file.id, this.getTestCommandArgs(file.id, undefined, coverage), getAllIds(file), reporter, token);
}

async runProjectAll(projectKey: string, files: CeedlingFileNode[], reporter: RunReporter, token: vscode.CancellationToken): Promise<void> {
async runProjectAll(projectKey: string, files: CeedlingFileNode[], reporter: RunReporter, token: vscode.CancellationToken, coverage = false): Promise<void> {
const allIds = files.flatMap((file) => getAllIds(file));
await this.runInternal(projectKey, projectKey, this.getTestCommandArgs('all'), allIds, reporter, token);
await this.runInternal(projectKey, projectKey, this.getTestCommandArgs('all', undefined, coverage), allIds, reporter, token);
}

// Runs a single test function (and, if parametrized, all its cases together - Ceedling's
// --test-case filter has function-level granularity only, it cannot isolate one case from
// its siblings) instead of every test in the file. Ceedling still recompiles the whole file
// regardless - --test-case narrows what runs and gets reported, not what gets built.
async runFunction(
projectKey: string, file: CeedlingFileNode, fn: CeedlingTestFunction, reporter: RunReporter, token: vscode.CancellationToken
projectKey: string, file: CeedlingFileNode, fn: CeedlingTestFunction, reporter: RunReporter, token: vscode.CancellationToken,
coverage = false
): Promise<void> {
const functionName = extractTestFunctionName(fn.id);
const ids = [fn.id, ...fn.cases.map((c) => c.id)];
Expand All @@ -226,14 +228,14 @@ export class CeedlingEngine {
// it stops being used, so a since-fixed error would linger under fn.id after a later
// whole-file run replaced only file.id's own entry.
const collided = await this.runInternal(
projectKey, file.id, this.getTestCommandArgs(file.id, functionName), ids, reporter, token, functionName
projectKey, file.id, this.getTestCommandArgs(file.id, functionName, coverage), ids, reporter, token, functionName
);
if (collided) {
this.logger.warn(
`runFunction: --test-case=${functionName} also matched an unrelated test in '${file.id}' - ` +
`Ceedling's filter is a substring match, not exact. Falling back to a full-file run.`
);
await this.runFile(projectKey, file, reporter, token);
await this.runFile(projectKey, file, reporter, token, coverage);
}
}

Expand Down Expand Up @@ -510,16 +512,63 @@ export class CeedlingEngine {
// `testCaseFilter`, when given, appends Ceedling's --test-case flag to narrow the run to one
// function (and, if parametrized, all its cases - see runFunction). Ceedling auto-enables
// :test_runner: :cmdline_args itself when this flag is present; nothing here needs to.
private getTestCommandArgs(testToExec: string, testCaseFilter?: string): Array<string> {
// `coverage` swaps in the instrumented task set, which is a separate setting because the
// gcov task is not a drop-in for the test task: it needs Ceedling's gcov plugin enabled.
private getTestCommandArgs(testToExec: string, testCaseFilter?: string, coverage = false): Array<string> {
// Keep only the filename of the test 'test/test_foo.c' -> 'test_foo.c'
const testSuiteFilename = testToExec.replace(/^.*[\\/]/, "");
const defaultTestCommandArgs = ["test:${TEST_ID}"];
const setting = coverage ? 'coverageCommandArgs' : 'testCommandArgs';
const fallback = coverage ? ["gcov:${TEST_ID}"] : ["test:${TEST_ID}"];
const args = this.getConfiguration()
.get<Array<string>>('testCommandArgs', defaultTestCommandArgs)
.get<Array<string>>(setting, fallback)
.map(x => x.replace("${TEST_ID}", testSuiteFilename));
return testCaseFilter === undefined ? args : [...args, `--test-case=${testCaseFilter}`];
}

// Where Ceedling's gcov plugin leaves its gcovr Cobertura report. Fixed by the plugin, not
// configurable: only the enclosing build directory varies per project.
private getCoverageReportPath(projectKey: string): string {
return path.resolve(
this.projectData[projectKey].absPath, this.buildDirectories[projectKey],
'artifacts', 'gcov', 'gcovr', 'GcovCoverageCobertura.xml'
);
}

// Returns [] rather than throwing when the report is absent or unreadable: a coverage run
// whose report never appeared is a misconfiguration to report as "no coverage", not a reason
// to fail a test run whose results are otherwise perfectly good.
async readCoverageReport(projectKey: string): Promise<CoveredFile[]> {
const reportPath = this.getCoverageReportPath(projectKey);
if (!fs.existsSync(reportPath)) {
this.logger.warn(
`readCoverageReport: no report at '${reportPath}' - the build failed, or Ceedling's gcov ` +
`plugin is not enabled with the 'Cobertura' report.`
);
return [];
}
try {
const parser = new xml2js.Parser({ explicitArray: false });
const parsed = await parser.parseStringPromise(fs.readFileSync(reportPath, 'utf8'));
// absPath is where execCeedling runs Ceedling, and so where Ceedling runs gcovr.
const files = getCoverageFromCoberturaReport(parsed, this.projectData[projectKey].absPath);
this.logger.debug(`readCoverageReport: ${files.length} file(s) from '${reportPath}'`);
return files;
} catch (e) {
this.logger.error(`readCoverageReport: failed to read '${reportPath}': ${util.format(e)}`);
return [];
}
}

// A coverage build that fails never reaches gcovr, which would leave the previous run's report
// to be shown as this one's - the same trap deleteXmlReport avoids for test results.
async deleteCoverageReport(projectKey: string): Promise<void> {
try {
await fs.promises.rm(this.getCoverageReportPath(projectKey), { force: true });
} catch (e) {
this.logger.error(`deleteCoverageReport: ${util.format(e)}`);
}
}

// Picks the diagnostic that best explains a compile failure. Patterns are evaluated
// warning-then-error-then-catchall. The first entry in `diagnostics` is not reliably the
// most severe one. An early warning can otherwise outrank the actual fatal error.
Expand Down
96 changes: 96 additions & 0 deletions src/gcovrCoberturaReport.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
// Pure, `vscode`-free normalization of a parsed gcovr Cobertura XML report (as produced by
// xml2js's `new Parser({ explicitArray: false })` against the output of Ceedling's gcov plugin
// with the `Cobertura` report enabled). Kept free of any `vscode`/`fs` dependency so it can be
// unit-tested directly in plain Node (see tests/unit/gcovrCoberturaReport.test.ts).

import * as path from 'path';

export interface CoveredLine {
line: number;
hits: number;
/** One entry per branch outcome on this line, true when taken. Empty for a plain statement. */
branches: boolean[];
}

export interface CoveredFunction {
name: string;
line: number;
hits: number;
}

export interface CoveredFile {
/** Absolute. */
path: string;
lines: CoveredLine[];
functions: CoveredFunction[];
}

// With `explicitArray: false`, a container holding exactly one child parses to a single object
// rather than a one-element array, and an empty self-closing container parses to an empty string.
// Both are normalized to an array here.
function toArray(value: any): any[] {
if (value === undefined || value === null || value === '') return [];
return Array.isArray(value) ? value : [value];
}

// `condition-coverage` looks like "50% (1/2)". Only the covered/total pair is usable: gcovr does
// not say *which* outcomes were taken, so the branches are reported as N taken followed by
// (total - N) not taken. That is enough for a covered/total ratio, which is all the consumer needs.
function parseBranches(conditionCoverage: unknown): boolean[] {
if (typeof conditionCoverage !== 'string') return [];
const match = /\((\d+)\/(\d+)\)/.exec(conditionCoverage);
if (match === null) return [];
const covered = Number(match[1]);
const total = Number(match[2]);
if (!Number.isFinite(covered) || !Number.isFinite(total) || total <= 0) return [];
return Array.from({ length: total }, (_, i) => i < covered);
}

// Returns one entry per <class> in the report. gcovr emits a <class> per source file; a file that
// was compiled but never executed still appears, with every line at zero hits. `baseDir` is the
// directory gcovr ran in. Throws if the report doesn't have the expected shape; callers are
// expected to catch and log.
export function getCoverageFromCoberturaReport(xmlReportData: any, baseDir: string): CoveredFile[] {
const coverage = xmlReportData['coverage'];
const sourceRoot = toArray(coverage['sources']?.['source'])[0] ?? '';
const files: CoveredFile[] = [];

for (const pkg of toArray(coverage['packages']?.['package'])) {
for (const cls of toArray(pkg['classes']?.['class'])) {
const filename = cls['$']?.['filename'] ?? '';
if (filename === '') continue;

const lines: CoveredLine[] = [];
for (const line of toArray(cls['lines']?.['line'])) {
const number = Number(line['$']?.['number']);
const hits = Number(line['$']?.['hits']);
if (!Number.isFinite(number) || !Number.isFinite(hits)) continue;
lines.push({
line: number,
hits,
branches: line['$']?.['branch'] === 'true'
? parseBranches(line['$']?.['condition-coverage'])
: [],
});
}

const functions: CoveredFunction[] = [];
for (const method of toArray(cls['methods']?.['method'])) {
const name = method['$']?.['name'] ?? '';
// gcovr does not give a method a hit count of its own. Its first line carries it.
const methodLines = toArray(method['lines']?.['line']);
const first = methodLines[0];
const line = Number(first?.['$']?.['number']);
const hits = Number(first?.['$']?.['hits']);
if (name === '' || !Number.isFinite(line)) continue;
functions.push({ name, line, hits: Number.isFinite(hits) ? hits : 0 });
}

// <filename> is relative to <source> inside gcovr's root and absolute outside it; <source>
// is relative to baseDir before gcovr 8.3 (gcovr#1037). path.join would mangle the absolutes.
files.push({ path: path.resolve(baseDir, String(sourceRoot), String(filename)), lines, functions });
}
}

return files;
}
Loading