[schlessera]

The Behat job with code coverage ran every scenario one after the other under Xdebug, and took most packages far longer than any other job. In wp-cli/wp-cli it took 45–48 min, while every other job finished in under 6.

Changes

• PCOV instead of Xdebug by default, through the new coverage-driver input. PCOV only instruments the files under pcov.directory, set to the checkout. Xdebug instruments every file PHP loads, all of WordPress included. PCOV alone halved the job time.
• Coverage now runs as a job of its own, split across runners by WP_CLI_TEST_SHARD, each runner with 4 parallel workers.
  • Packages get one runner per 50 scenarios, up to the new coverage-shards input (default 4), because every runner spends about a minute on its setup.
  • A matrix entry can set its own number with coverage_shards.
• One Codecov upload. The shards keep their reports as artifacts, and a final job uploads all of them in one go once every shard has passed. Codecov still sees one feature upload per commit, so settings like wp-cli/wp-cli's after_n_builds: 2 keep working.
• Older wp-cli-tests: with versions that can't shard, the first runner runs every scenario and the others skip, so coverage stays complete and each scenario runs once.

Results in wp-cli/wp-cli

Measured with wp-cli/wp-cli-tests#420:

Setup	Coverage job (slowest shard)
Before: Xdebug, one process	45–48 min
PCOV, 4 workers, 1 runner	8–12 min
PCOV, 4 workers, 4 runners	3.7–4.2 min

• Workflow time: the whole Testing workflow went from ~49 min to ~6 min, so coverage no longer sets the overall time.
• Coverage numbers: feature coverage was 64.97% against 65.13% before. The difference is mostly global statements, which Xdebug counts and PCOV doesn't.
• Fallback on CI: tested in media-command, which is still on a wp-cli-tests release without sharding (run). It got 2 runners: the first ran all 75 scenarios, the second skipped in 44 s, and one report was uploaded.

actionlint is clean.

🤖 Generated with Claude Code

[coderabbitai]

Important

• Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 6685f7dd-44c0-4858-8e63-5ed83f200e6f

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

• X
• Mastodon
• Reddit
• LinkedIn