From 4c9ce1005eccd34ce97c7fbdcd263e9b5f59fce1 Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Thu, 15 Oct 2026 10:03:58 +0000 Subject: [PATCH 1/2] Report unit test coverage across modules Per-module JaCoCo reports credit a class only with the tests of its own module, so classes tested from another module show as uncovered. For example core's CalciteRelNodeVisitor is tested almost only by ppl's CalcitePPL*Test, and its report showed 11 of 2,622 lines covered, which made codecov/patch report about 0% for any change to it. Add a root jacocoAggregatedReport task that merges the unit test execution data of every module that applies jacoco and reports it against the same classes, with the same excludes, as the per-module reports that CI uploaded. CI now uploads this one report to Codecov. Line coverage of those classes goes from 27,094 / 41,033 (66.0%) to 32,135 / 41,033 (78.3%); core alone from 56.1% to 80.9%, and CalciteRelNodeVisitor to 2,271 of 2,622 lines. The per-module reports and jacocoTestCoverageVerification rules are unchanged. Signed-off-by: Lantao Jin --- .../workflows/sql-test-and-build-workflow.yml | 4 +- DEVELOPER_GUIDE.rst | 4 +- build.gradle | 47 +++++++++++++++++++ 3 files changed, 52 insertions(+), 3 deletions(-) diff --git a/.github/workflows/sql-test-and-build-workflow.yml b/.github/workflows/sql-test-and-build-workflow.yml index ccc7f11d605..18b0cdfc232 100644 --- a/.github/workflows/sql-test-and-build-workflow.yml +++ b/.github/workflows/sql-test-and-build-workflow.yml @@ -106,7 +106,7 @@ jobs: - name: Build and Test run: | if [ "${{ matrix.test-type }}" = "unit" ]; then - ./gradlew --continue build -x spotlessCheck -x integTest -x tracingIntegTest -x yamlRestTest -x doctest + ./gradlew --continue build jacocoAggregatedReport -x spotlessCheck -x integTest -x tracingIntegTest -x yamlRestTest -x doctest elif [ "${{ matrix.test-type }}" = "integration" ]; then ./gradlew --continue integTest tracingIntegTest yamlRestTest else @@ -124,7 +124,7 @@ jobs: uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 with: name: jacoco-coverage-reports - path: '**/build/reports/jacoco/test/jacocoTestReport.xml' + path: build/reports/jacoco/jacocoAggregatedReport/jacocoAggregatedReport.xml retention-days: 1 if-no-files-found: error diff --git a/DEVELOPER_GUIDE.rst b/DEVELOPER_GUIDE.rst index a179b1fc64d..821565412f6 100644 --- a/DEVELOPER_GUIDE.rst +++ b/DEVELOPER_GUIDE.rst @@ -243,6 +243,8 @@ For faster local iterations, skip integration tests. ``./gradlew build -x integT - Compile all Java source files. * - ./gradlew test - Run all unit tests. + * - ./gradlew jacocoAggregatedReport + - Run all unit tests and generate a code coverage report across all modules in build/reports/jacoco/jacocoAggregatedReport. * - ./gradlew :integ-test:integTest - Run all integration test (this takes time). * - ./gradlew :integ-test:yamlRestTest @@ -357,7 +359,7 @@ Here is a sample for integration test for your reference: } } -Finally thanks to JaCoCo library, you can check out the test coverage in ``/build/reports/jacoco`` for your changes easily. +Finally thanks to JaCoCo library, you can check out the test coverage in ``/build/reports/jacoco`` for your changes easily. A module's report only counts the tests of that module. To also count tests in other modules (for example ``ppl`` tests of ``core`` classes), which is the coverage that Codecov reports, run ``./gradlew jacocoAggregatedReport`` and open ``build/reports/jacoco/jacocoAggregatedReport/html/index.html``. Deploying Locally ----------------- diff --git a/build.gradle b/build.gradle index 348ff34563b..0b8d794773e 100644 --- a/build.gradle +++ b/build.gradle @@ -300,6 +300,53 @@ jacocoTestCoverageVerification { } check.dependsOn jacocoTestCoverageVerification +// Each module's jacocoTestReport credits a class only with the tests of its own module, but many +// classes are mostly tested from other modules (for example core's CalciteRelNodeVisitor is tested +// by ppl's CalcitePPL*Test). This report merges the unit test execution data of every module that +// applies jacoco and reports it against the classes of the modules below, with the same excludes as +// their own jacocoTestReport. CI uploads it to Codecov. The per-module +// jacocoTestCoverageVerification rules still measure each module's own tests only. +def aggregatedCoverageModules = [ + ':api', + ':async-query', + ':async-query-core', + ':core', + ':datasources', + ':direct-query', + ':direct-query-core', + ':opensearch', + ':ppl', + ':prometheus', + ':protocol', + ':sql', +] +def jacocoAggregatedReport = tasks.register('jacocoAggregatedReport', JacocoReport) { + group = 'verification' + description = 'Generates a code coverage report of all modules from the unit tests of all modules.' + reports { + html.required = true + xml.required = true + csv.required = false + } +} +// The module reports set their excludes in afterEvaluate, so read them once every project is evaluated. +gradle.projectsEvaluated { + jacocoAggregatedReport.configure { + subprojects.findAll { it.plugins.hasPlugin('jacoco') }.each { p -> + // 'junit4' is the JUnit 4 test task of async-query, direct-query and direct-query-core. + p.tasks.withType(Test).matching { it.name in ['test', 'junit4'] }.each { testTask -> + dependsOn testTask + executionData testTask + } + } + aggregatedCoverageModules.each { path -> + def moduleReport = project(path).tasks.named('jacocoTestReport', JacocoReport).get() + classDirectories.from(moduleReport.classDirectories) + sourceDirectories.from(moduleReport.sourceDirectories) + } + } +} + configurations.all { resolutionStrategy.force 'junit:junit:4.13.2' From e2fdc209cd431176e92774fcfb2499901951f0ab Mon Sep 17 00:00:00 2001 From: Lantao Jin Date: Fri, 16 Oct 2026 02:46:05 +0000 Subject: [PATCH 2/2] Generate the aggregated coverage report when a unit test fails jacocoAggregatedReport depended on every unit test task, so with --continue a single failing test skipped the report, and the CI coverage upload failed with no file to upload. The per-module reports it replaces are finalizers of the test task and are still generated when a test fails, because JaCoCo writes the execution data anyway. Order the report after the test tasks instead of depending on them, as each module's jacocoTestReport does. It no longer runs the tests by itself, so the developer guide now says ./gradlew test jacocoAggregatedReport. The CI command already runs build first. Signed-off-by: Lantao Jin --- DEVELOPER_GUIDE.rst | 4 ++-- build.gradle | 7 +++++-- 2 files changed, 7 insertions(+), 4 deletions(-) diff --git a/DEVELOPER_GUIDE.rst b/DEVELOPER_GUIDE.rst index 821565412f6..4c75cfdeaa7 100644 --- a/DEVELOPER_GUIDE.rst +++ b/DEVELOPER_GUIDE.rst @@ -243,7 +243,7 @@ For faster local iterations, skip integration tests. ``./gradlew build -x integT - Compile all Java source files. * - ./gradlew test - Run all unit tests. - * - ./gradlew jacocoAggregatedReport + * - ./gradlew test jacocoAggregatedReport - Run all unit tests and generate a code coverage report across all modules in build/reports/jacoco/jacocoAggregatedReport. * - ./gradlew :integ-test:integTest - Run all integration test (this takes time). @@ -359,7 +359,7 @@ Here is a sample for integration test for your reference: } } -Finally thanks to JaCoCo library, you can check out the test coverage in ``/build/reports/jacoco`` for your changes easily. A module's report only counts the tests of that module. To also count tests in other modules (for example ``ppl`` tests of ``core`` classes), which is the coverage that Codecov reports, run ``./gradlew jacocoAggregatedReport`` and open ``build/reports/jacoco/jacocoAggregatedReport/html/index.html``. +Finally thanks to JaCoCo library, you can check out the test coverage in ``/build/reports/jacoco`` for your changes easily. A module's report only counts the tests of that module. To also count tests in other modules (for example ``ppl`` tests of ``core`` classes), which is the coverage that Codecov reports, run ``./gradlew test jacocoAggregatedReport`` and open ``build/reports/jacoco/jacocoAggregatedReport/html/index.html``. Deploying Locally ----------------- diff --git a/build.gradle b/build.gradle index 0b8d794773e..be38845404d 100644 --- a/build.gradle +++ b/build.gradle @@ -322,7 +322,8 @@ def aggregatedCoverageModules = [ ] def jacocoAggregatedReport = tasks.register('jacocoAggregatedReport', JacocoReport) { group = 'verification' - description = 'Generates a code coverage report of all modules from the unit tests of all modules.' + description = 'Generates a code coverage report of all modules from the unit tests of all modules. ' + + 'Does not run the tests; use ./gradlew test jacocoAggregatedReport.' reports { html.required = true xml.required = true @@ -335,7 +336,9 @@ gradle.projectsEvaluated { subprojects.findAll { it.plugins.hasPlugin('jacoco') }.each { p -> // 'junit4' is the JUnit 4 test task of async-query, direct-query and direct-query-core. p.tasks.withType(Test).matching { it.name in ['test', 'junit4'] }.each { testTask -> - dependsOn testTask + // Like each module's jacocoTestReport, only run after the tests rather than depend on + // them, so that a failing test still leaves a report for CI to upload. + mustRunAfter testTask executionData testTask } }