Skip to content

Run migration tests once per backend instead of in every core test shard - #73185

Merged
potiuk merged 5 commits into
apache:mainfrom
zozo123:split/migration-validation
Sep 22, 2026
Merged

potiuk merged 5 commits into
apache:mainfrom
zozo123:split/migration-validation

Conversation

@zozo123

@zozo123 zozo123 commented Sep 15, 2026 •

Copy link
Copy Markdown
Contributor

Run migration validation only in the first existing core-test shard for each eligible Python/backend combination. Other shards continue running their normal tests. Migration failures still fail that shard, and the Python 3.14 exemption is unchanged.

The final diff is one workflow condition. Review feedback is addressed: the redundant test file is removed and the original comment explaining when to exclude new Python versions is restored.

Validation: actionlint and applicable pre-commit checks pass locally. Updated onto current main; fresh CI is running. Related: #73124.


Was generative AI tooling used to co-author this PR?
  • Yes — Codex (GPT-6)

Generated-by: Codex (GPT-6) following the guidelines

@Andrushika Andrushika left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the optimization concept itself is good.
When using AI tools to contribute, please make sure you fully understand every line of the change and that every line matters.

Comment thread scripts/tests/ci/test_migration_validation_optimizations.py Outdated
Comment thread scripts/tests/ci/test_migration_validation_optimizations.py Outdated
Comment thread .github/workflows/run-unit-tests.yml
@zozo123
zozo123 force-pushed the split/migration-validation branch from 93f6220 to 7fbf260 Compare September 22, 2026 11:09
@zozo123
zozo123 requested a review from Andrushika September 22, 2026 11:28

@Andrushika Andrushika left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM after change, thanks.
The PR title could be more clear, something like:
Run migration tests once per backend instead of in every core test shard

@zozo123 zozo123 changed the title Run migration validation in an existing test shard Run migration tests once per backend instead of in every core test shard Sep 22, 2026

@potiuk potiuk left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks — good saving. Migration tests don't depend on the test type, so running them in every core shard repeated identical work. I checked the one way this could go wrong: the matrix exclude lists (postgres_exclude / mysql_exclude / sqlite_exclude in selective_checks.py) only ever drop python/backend pairs, never test types, so every surviving python/backend combination still runs migrations exactly once. Approving.

One optional nit: the fromJSON(inputs.test-types-as-strings-in-json)[0].description clause isn't self-explanatory — a one-line comment ("run once per python/backend: only in the first test-type shard") would stop someone from "simplifying" it away later.


This review was drafted by an AI-assisted tool and
confirmed by an Apache Airflow maintainer. The maintainer
approving this PR has read the findings and signed off. If
something feels off, please reply on the PR and a maintainer
will follow up.

More on how Apache Airflow handles maintainer review:
contributing-docs/05_pull_requests.rst.


Drafted-by: Claude Code (Opus 5); reviewed by @potiuk before posting

@potiuk
potiuk merged commit 8303d28 into apache:main Sep 22, 2026
156 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Backport failed to create: v3-3-test. View the failure log Run details

Note: As of Merging PRs targeted for Airflow 3.X
the committer who merges the PR is responsible for backporting the PRs that are bug fixes (generally speaking) to the maintenance branches.

In matter of doubt please ask in #release-management Slack channel.

Status Branch Result
❌ v3-3-test Commit Link

You can attempt to backport this manually by running:

cherry_picker 8303d28 v3-3-test

This should apply the commit to the v3-3-test branch and leave the commit in conflict state marking
the files that need manual conflict resolution.

After you have resolved the conflicts, you can continue the backport process by running:

cherry_picker --continue

If you don't have cherry-picker installed, see the installation guide.

@zozo123
zozo123 deleted the split/migration-validation branch September 24, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants