Skip prek cache re-upload on cache hit and drop skill-eval envs from CI cache

[kaxil]

The Static checks job spends about five minutes of every pull request run re-compressing and re-uploading a prek cache it has just downloaded. This PR removes that work on cache hits and shrinks the cache itself, for roughly a 6 to 7 minute saving on a job that currently takes 25 to 40 minutes.

Timings from a typical run (job 106662807685, 25.3 min total):

Step	Time
Download prek cache stash (2.7 GB gzipped, 7.2 GB extracted)	3.0 min
Extract	0.7 min
prek install-hooks (cache hit)	1 s
Re-tar the same cache	3.5 min
Re-upload it	1.8 min

Save only when there is something new to save. The stash restore looks for the PR branch first and then falls back to the base branch, so a PR on a cache hit is holding the main stash and would upload a byte-identical copy under its own branch name. The install-prek action now skips the tar and upload steps when the tarball was restored and the event is a pull_request. Cache misses still save, and non-PR runs (the canary and scheduled runs on main) still save on every run so the base-branch stash stays inside its two-day retention window. save-cache: true is only set by the two static-check jobs, and both are reached only from ci-amd.yml and ci-arm.yml, so pull_request is the only PR-shaped event that reaches this condition.

Drop the skill-eval environments from the cache. prek install-hooks builds environments for every configured hook, including manual-stage ones that CI never runs. The three skill-eval hooks pull promptfoo plus the Anthropic and OpenAI agent SDKs into two node environments, which are about 3.5 GB (measured on a local prek cache; the CI log has no per-environment breakdown, only the 7.2 GB total). Skipping them in the CI install roughly halves what every job downloads and extracts. Because --skip only avoids building environments and does not remove restored ones, the cache key moves from cache-prek-v9 to cache-prek-v10, so the first run after merge builds the cache from scratch without them instead of re-saving the restored oversized one.

The miss path also compresses with pigz when it is on the runner (it ships in the ubuntu-22.04 image) and falls back to gzip otherwise. The output is plain gzip, so the restore side is unchanged and the cache key does not need a bump.

---

• Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
• For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
• When adding dependency, check compliance with the ASF 3rd Party License Policy.
• For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

[Andrushika]

Left two comments, thanks!
This overlaps with #73182, which skips the same re-upload on a cache hit. It might be worth a look so the two do not resolve the hit rule differently.

[kaxil]

Thanks for the pointer to #73182. This PR is the narrow subset of that change (skip the save on a hit, faster compression, no double compression, plus dropping the skill-eval environments and bumping the key so the slim cache actually lands). The hit rule here is deliberately simple: save on a miss or on non-PR runs, so the main stash keeps refreshing. #73182's marker-based detection and the narrower ~/.cache/prek wipe would layer on top of this cleanly if it is rebased.

[github-actions]

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status	Branch	Result
❌	v3-3-test

You can attempt to backport this manually by running:

cherry_picker b65c145 v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.