Skip to content

feat(ci): AgentCI owns the DAG — job outputs, run-time needs.*, one job per act (2.130.0) - #181

Merged
drknowhow merged 2 commits into
mainfrom
feat/ci-dag-ownership
Sep 12, 2026
Merged

drknowhow merged 2 commits into
mainfrom
feat/ci-dag-ownership

Conversation

@drknowhow

Copy link
Copy Markdown
Owner

Why

Yep's test workflow (drknowhow/Yep #1509: a changes gate, three shards on if: needs.changes.outputs.code == 'true', an aggregator reading ${{ needs.<job>.result }}) came back from c3_ci with every job UNSUP: "unresolved expression(s): github.event.before, needs.changes.result". That shape is most real workflows.

Measured before writing

Two-job probe, act 0.2.89, Docker Desktop 29.6.2:

  • act -j b runs a first when b needs it. C3 runs jobs one -j at a time, so every dependency ran twice and an aggregator replayed the whole workflow inside its container.
  • act logs GITHUB_OUTPUT writes as ::set-output:: k=v (no step id).
  • With a declared event and no payload, github.event.* is "".
  • act accepts -W outside the working tree.

What

  • Job outputs, both engines. Native: per-step GITHUB_OUTPUT file (k=v, k<<EOF). act: set-output lines read back. outputs: resolves via steps.<id>.outputs.<k>; dependents get needs.<job>.outputs.<name> in if: and run:. Declared-but-never-written is refused, not blanked. Jobs with outputs skip the verdict cache.
  • needs.* / steps.* deferred at DAG-build time, filled in per step at run time (late_substitute).
  • github.event.* is "" under a declared event (what GitHub and act do); a native blocker without one. secrets.*/vars.* still block both engines.
  • One job per act. act gets a derived copy of the workflow (temp file, -W): this job only, needs and if: stripped, needs.* spelled out from C3's recorded results/outputs. An unrecorded reference is reported before act starts.
  • Cached image is not pulled (docker image inspect first) — closes the issue c3_ci: act image-pull auth failure is reported as a job failure with 'unparsed' output #173 trap where act's stored credential turns a public image on disk into an auth failure.
  • Free artifact-server port per run. act binds 34567 by default; two runs on one box collided and the second died before its first step with unparsed.

Proof

  • tests/test_ci_dag_ownership.py: 19 tests (classification, derived workflow, log outputs, native outputs flow, pull policy, port).
  • Full suite: 4020 passed, 9 skipped (the codex transport test fails on untouched main too — environment). The act integration test now passes while another act run holds 34567.
  • Live on drknowhow/Yep's workflow (10,683 tests): 6/6 jobs ran — changes alone in its container (code=true captured), shards 517s/362s/277s green, the aggregator handed changes=success shards=success smokes=failure. Verdict FAIL, honestly: one smoke needs java, present on GitHub's runner and absent from the act image. Fix for that goes to the Yep repo.

Not in this PR

  • Pinning the runner image by digest (floating catthehacker/ubuntu:act-latest today). Cheap to add if wanted.
  • act's per-step ids for set-output: the union of a job's writes is used; two steps writing the same key is the one shape that cannot be told apart (last wins, as on GitHub).

https://claude.ai/code/session_019n1sv7vnKeRjwHMLmA7Tv2

…ob per act (2.130.0)

A workflow with a `changes` gate, shards on
`if: needs.changes.outputs.code == 'true'` and an aggregator reading
`${{ needs.<job>.result }}` could not run under c3_ci: every job UNSUP
with "unresolved expression(s): github.event.before, needs.changes.result".

Measured first, with a two-job probe: `act -j b` runs `a` when `b` needs
it, so C3's per-job act invocation ran every dependency twice and an
aggregator replayed the whole workflow; act logs GITHUB_OUTPUT writes as
`::set-output:: k=v`; `github.event.*` is "" under a declared event.

- Job outputs on both engines. Native steps get a GITHUB_OUTPUT file
  (k=v and k<<EOF); act's set-output lines are read back. A job's
  `outputs:` resolves from `steps.<id>.outputs.<k>` and dependents read
  `needs.<job>.outputs.<name>` in `if:` and `run:`. Declared-but-never-
  written is refused, not blanked. Jobs with outputs are never cached.
- `needs.*` / `steps.*` deferred at DAG-build time, filled in per step.
- `github.event.*` is "" with a declared event; a native blocker without.
- act receives a derived single-job workflow (temp file, -W outside the
  tree): `needs` and `if:` removed, `needs.*` spelled out from C3's own
  results and outputs. Unrecorded references are reported before act runs.
- A cached runner image is no longer "pulled" (issue #173's auth trap).
- Each act run gets a free artifact-server port; two runs on one box
  collided on 34567 and the second died before its first step.

Live on drknowhow/Yep's test workflow: 6/6 jobs ran, shards green, the
aggregator got real results. Verdict FAIL for an honest reason (a smoke
needs java; the act image has none). tests: 4020 passed; the codex
transport test fails on untouched main too (environment).

Claude-Session: https://claude.ai/code/session_019n1sv7vnKeRjwHMLmA7Tv2
…ved the job its gate reads

`--job smokes` deselected `changes`, then `if: needs.changes.outputs.code
== 'true'` was unjudgeable and the job came back UNSUP. A selector (and a
rerun) now brings the transitive `needs` closure along; the rest stays
DESELECTED and the verdict stays PARTIAL. Measured live: `--job smokes`
on Yep's workflow now runs `changes` (1.3s, code=true) then `smokes`.

Claude-Session: https://claude.ai/code/session_019n1sv7vnKeRjwHMLmA7Tv2
@drknowhow
drknowhow merged commit 0a686ac into main Sep 12, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant