Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 11 additions & 1 deletion .github/fork-ci-gate.toml
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ enabled = false
enabled_since = 2026-01-01T00:00:00Z

# Days without any fork CI activity after which a gated pull request is closed.
close_after_days = 7
close_after_days = 30

# Report each job of a gated pull request's fork run as its own commit status. Set to false when
# the fork CI monitor exceeds GitHub's rate limits (see dev/AGENTS.md); the `Fork CI / Tests (AMD)`
Expand All @@ -50,4 +50,14 @@ job_statuses = true
# check-fork-ci-gate-config hook rejects a new or changed instant that is in the past.
[pilot]
# "login-three" = 2026-10-20T00:00:00Z
"aaron-y-chen" = 2026-11-16T00:00:00Z
"Andrushika" = 2026-11-16T00:00:00Z
"Dev-iL" = 2026-11-16T00:00:00Z
"justinpakzad" = 2026-11-16T00:00:00Z
"Prab-27" = 2026-11-16T00:00:00Z
"SameerMesiah97" = 2026-11-16T00:00:00Z
"sreecharan-desu" = 2026-11-16T00:00:00Z
"stephen-bracken" = 2026-11-16T00:00:00Z
"raphaelauv" = 2026-11-16T00:00:00Z
"yuseok89" = 2026-11-16T00:00:00Z
"meel-ayush" = 2026-11-11T00:00:00Z
5 changes: 3 additions & 2 deletions .github/workflows/fork-ci-monitor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
name: "Fork CI monitor"
on: # yamllint disable-line rule:truthy
schedule:
- cron: '*/15 * * * *'
# MONITOR_INTERVAL in fork_ci_reconcile.py, whose per-run write budgets assume this cadence.
- cron: '*/5 * * * *'
workflow_dispatch:
inputs:
dry-run:
Expand Down Expand Up @@ -59,7 +60,7 @@ jobs:
timeout-minutes: 20
steps:
# What earlier runs reported, so that a problem that persists is posted to Slack once
# rather than every 15 minutes. Losing it, for example to cache eviction, re-posts once.
# rather than on every run. Losing it, for example to cache eviction, re-posts once.
# Restored before anything that can fail, so a failing setup is reported once too, and
# outside the checkout, where nothing but the steps below reads it.
- name: "Restore what earlier runs reported"
Expand Down
16 changes: 14 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -331,8 +331,9 @@ repos:
PYTHONPYCACHEPREFIX: .build/pycache
entry: ./scripts/ci/prek/check_fork_ci_gate_config.py
language: python
files: ^\.github/fork-ci-gate\.toml$|^dev/breeze/src/airflow_breeze/utils/fork_ci_gate\.py$
files: ^\.github/fork-ci-gate\.toml$|^dev/breeze/src/airflow_breeze/utils/fork_ci_gate\.py$|^\.asf\.yaml$
pass_filenames: false
additional_dependencies: ['pyyaml']
- id: check-java-sdk-version-in-sync
name: Check Java SDK JDK version is consistent across build files
env:
Expand Down Expand Up @@ -1036,7 +1037,8 @@ repos:
^airflow-core/docs/.*/diagram_[^/]*\.py$|
^contributing-docs/images/diagram_[^/]*\.py$|
^docs/images/.*\.py$|
^airflow-ctl/docs/images/diagrams/.*\.py$
^airflow-ctl/docs/images/diagrams/.*\.py$|
^dev/breeze/doc/ci/images/diagram_[^/]*\.py$
pass_filenames: true
- id: prevent-deprecated-sqlalchemy-usage
name: Prevent deprecated sqlalchemy usage
Expand Down Expand Up @@ -1725,6 +1727,16 @@ repos:
language: python
pass_filenames: true
files: ^\.github/workflows/.*\.yml$
- id: check-scheduled-workflows-skip-forks
name: Check scheduled workflows skip their cron runs in forks
env:
PYTHONPYCACHEPREFIX: .build/pycache
description: Forks that enable Actions get every cron, so each scheduled job must skip outside apache/airflow
entry: ./scripts/ci/prek/check_scheduled_workflows_skip_forks.py
language: python
pass_filenames: true
require_serial: true
files: ^\.github/workflows/.*\.yml$
- id: check-docstring-param-types
name: Check that docstrings do not specify param types
env:
Expand Down
2 changes: 1 addition & 1 deletion contributing-docs/05_pull_requests.rst
Original file line number Diff line number Diff line change
Expand Up @@ -272,7 +272,7 @@ and approves them automatically when all of these hold:

If your PR's CI runs in your fork (it carries the ``fork ci required`` label), the CI run in
``apache/airflow`` only checks that, so none of the rules above apply. The ``CI run started`` check
waits until the fork CI monitor sees your fork's run, usually within 15 minutes, and then links it. Its runs are approved
waits until the fork CI monitor sees your fork's run, usually within 5 to 10 minutes, and then links it. Its runs are approved
automatically unless the PR changes files that decide which CI jobs run and on which runners:
anything under ``.github/``, ``dev/breeze/src/`` or ``scripts/ci/prek/``, or
``dev/breeze/pyproject.toml``, ``dev/breeze/uv.lock`` or ``scripts/ci/install_breeze.sh``. Then
Expand Down
30 changes: 23 additions & 7 deletions contributing-docs/18_contribution_workflow.rst
Original file line number Diff line number Diff line change
Expand Up @@ -132,12 +132,14 @@ CI in your fork

Pull requests from contributors who are not committers or collaborators are moving to running
their CI in the contributor's own fork, on GitHub's free runners, instead of on the shared CI
capacity of the Apache Software Foundation (`AIP-120
<https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711/AIP-120+Run+CI+for+external+contributions+in+contributor+forks>`__).
capacity of the Apache Software Foundation (`ADIP-12, formerly AIP-120
<https://cwiki.apache.org/confluence/spaces/AIRFLOW/pages/451974711>`__).
CI on a public fork costs you nothing and has no minute limit; it runs on every push to your
fork, so you get feedback before a maintainer looks at your pull request. The change is being
piloted: until it is switched on for everyone, it applies to pilot participants (see below) and
to pull requests a maintainer labels ``use fork ci``.
to pull requests a maintainer labels ``use fork ci``. The `pull request lifecycle
<../dev/breeze/doc/ci/08_fork_ci_for_external_contributors.md#pull-request-lifecycle>`__ diagram
shows what your pull request shows at each step and who acts next.

Set up your checkout and fork once:

Expand All @@ -156,7 +158,11 @@ Set up your checkout and fork once:
gh workflow enable ci-amd.yml -R <you>/airflow

GitHub Actions must also be enabled in your fork and allow all actions: ``Tests (AMD)`` uses
actions from other repositories.
actions from other repositories. When your fork's Actions settings block the workflow, its run
ends in "Startup failure" before any job runs, and the fork CI monitor reports
``Fork CI / Tests (AMD)`` as failed with "could not start". Open the run in your fork for GitHub's
error, which can also be an invalid workflow file, fix the settings under
**Settings > Actions > General** in your fork, and push again.

Keep the workflow enabled for as long as you have open pull requests. If it gets disabled, from
your fork's Actions tab or by GitHub after 60 days without activity in your fork, the fork CI
Expand All @@ -178,15 +184,25 @@ Set up your checkout and fork once:
breeze ci audit

When fork CI applies to your pull request, it is labelled ``fork ci required`` and kept in draft
until the ``Tests (AMD)`` run in your fork for its latest commit is green. A
until the ``Tests (AMD)`` workflow runs in your fork for its latest commit. A failed run does not
draft it. A
``Fork CI / Tests (AMD)`` status on the pull request links to that run, and one comment on the pull
request explains the current state. Once the run is green, a pull request the monitor drafted is
marked ready for review and labelled ``ready for maintainer review``; a pull request you drafted
yourself stays a draft, and gets the label, once you mark it ready. A pull request whose fork CI
does not run for 7 days is closed with an explanation; you can reopen it after setting up fork CI. If your pull request changes
yourself stays a draft, and gets the label, once you mark it ready. Marking a draft ready needs a
free slot under the `limit on open pull requests <32_open_pull_request_limit.rst>`__: at the limit,
GitHub refuses it, also when a maintainer tries, so close or finish one of your other open pull
requests first. A pull request whose fork CI
does not run for 30 days is closed with an explanation; you can reopen it after setting up fork CI. If your pull request changes
files that decide which CI jobs run and where, such as ``.github/``, the breeze sources or
``scripts/ci/prek/``, a green fork run is not enough: a maintainer reviews the change and runs project CI for it.

Fork CI tests your branch as it is, while project CI tests it merged with the target branch. The
``CI passed`` check that merging requires is therefore reported only when your branch is fewer than
200 commits behind the target branch and the target branch has not changed any file your pull request
changes since your branch's base. Otherwise the pull request comment lists those files and asks you
to rebase on ``upstream/main`` (or the target branch) and push, which runs fork CI again.

**Joining the pilot.** Open a pull request that adds your GitHub login to the ``[pilot]`` table in
``.github/fork-ci-gate.toml``, with a join instant after the expected merge, for example the next day
at 00:00 UTC:
Expand Down
3 changes: 3 additions & 0 deletions contributing-docs/32_open_pull_request_limit.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,9 @@ How the limit works
* Draft Pull Requests do not count towards the limit yet, because GitHub does not support counting
them. Once GitHub adds that, we will start counting drafts as well - so do not rely on drafts to
get around the limit. Keep unfinished work on a branch in your fork instead.
* Marking a draft ready for review makes it count, so GitHub refuses it while you are at the limit -
also when a maintainer tries to do it for you. Close or finish one of your other open Pull
Requests first, then mark the draft ready.
* When you have 5 open Pull Requests, GitHub does not let you open another one. A slot frees up when
one of your Pull Requests is merged or closed.
* The limit counts Pull Requests you have open at the same time - there is no limit on how many
Expand Down
4 changes: 2 additions & 2 deletions dev/AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,9 @@ workflow of `apache/airflow` shares (see "Write budget" in

If the budget is exceeded, turn the job statuses off: set `job_statuses = false` in
`.github/fork-ci-gate.toml`. Signs that it is exceeded are the monitor's Slack note that job
statuses "have not fit in the budget" (posted only after three runs in a row deferred them)
statuses "have not fit in the budget" (posted only after job statuses were deferred for about 45 minutes in a row)
staying for hours, pull requests listed because their fork run finished more than an hour ago with
job statuses still not posted, its Slack note that GitHub rate-limited it in two runs in a row, the
job statuses still not posted, its Slack note that GitHub rate-limited it for about 30 minutes in a row, the
"Rate limits left" line of its run log approaching zero, or other workflows of the repository
failing on secondary rate limits. The aggregate `Fork CI / Tests (AMD)`
status and `CI passed` are posted either way, so turning job statuses off loses detail, not gating.
Expand Down
2 changes: 1 addition & 1 deletion dev/breeze/doc/08_ci_tasks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -185,7 +185,7 @@ Reconciling pull requests gated to fork CI
------------------------------------------

Pull requests from external contributors can run their CI in the contributor's fork instead of in
``apache/airflow`` (AIP-120). The ``fork-ci-reconcile`` command, run every 15 minutes by the
``apache/airflow`` (AIP-120). The ``fork-ci-reconcile`` command, run every 5 minutes by the
``fork-ci-monitor.yml`` workflow, is the only place that changes those pull requests: it labels newly
gated pull requests ``fork ci required``, keeps them in draft until the fork run is green, maintains
one state comment and a ``Fork CI / Tests (AMD)`` commit status linking to the run in the fork, closes
Expand Down
Loading
Loading