Skip to content
30 changes: 24 additions & 6 deletions .claude/notes/agents.md
Original file line number Diff line number Diff line change
Expand Up @@ -855,12 +855,29 @@ shared `agents/_skills.py` resolver: the SDK wants ONE parent directory, not a l
ran 7 steps). The adapter counts calls from the event stream and abandons the host when call N+1
opens; `len(turnUsages)` from the `result` then replaces the estimate as `num_turns`.

**A call opens on its first text, or on its `usage` frame when it streams none — never on a tool
event.** The host streams a call's text, flushes the call's `usage` frame, then runs the call's
tools, and it runs a batched reply's tools one at a time: `tool_call`, `tool_result`, `tool_call`,
`tool_result`. The adapter once opened call N+1 when every open tool had returned. With that order
the open set empties after each tool, so every tool counted as a call: a downstream suite saw a turn
with 16 round-trips and 96 tool calls cut at `max_turns: 80`. Now a `thinking` or `message` event
opens a call unless the open call has already streamed text, and a `usage` frame opens one only for
a call that streamed none. A crossing call that streams text is cut at that text, before any of its
tools. A tool-only crossing call is cut at its own frame, and its first tool can start before the kill:
the host starts a call's tools right after the frame, and the stream gives no earlier boundary. A host
without `usage` frames counts only the calls that stream text; such a host is below the 1.203.0 floor,
which is also the first release that takes the `auth` init option.
Only `_TurnState.on_text` / `on_tool_event` / `on_usage_frame` / `on_result` write the count, so the rule
lives in one place. `on_result` compares the live count with `len(turnUsages)` and warns when they
differ: the order is read from the 1.203.1 host and SDK source, not from a live transcript, so a host
release that reorders the frames shows up in the logs instead of silently moving the cap.

**A cut turn keeps the usage of its finished calls.** A turn cut by `max_turns` or a stop gets no
`result`, so the adapter sums the per-round-trip `usage` frames; the `result`'s total replaces the sum
when it arrives. Ordering (from SDK/backend source, not a live transcript): call N's `usage` frame
always precedes the `tool_result` that opens call N+1. The frame is NOT a call boundary — it arrives
before its call's tools run — so the cap stays on "the previous call's tools have all returned". The
adapter warns when a cut turn had finished calls but no usage.
when it arrives. Ordering (from the host and SDK source, not a live transcript): call N's `usage` frame
precedes every tool event of call N. A crossing call that streams text is cut at that text, so its usage
is dropped; a tool-only crossing call is cut at its own frame, so its usage is booked. The adapter warns
when a cut turn had finished calls but no usage.

**One `AssistantMessage` per `communicate()`.** `isStepStart` and `turnUsages` would allow a
per-round-trip split; today `isStepStart` only merges streamed deltas. `close_window` opens the one
Expand Down Expand Up @@ -957,8 +974,9 @@ introduced or should silently "fix" by picking one number over the other.

### Delegate agent golden-master coverage

`tests/_fixtures/golden_streams/delegate_fixtures.py` scripts the host's stdout frames for four
scenarios: a tool call, a `max_turns` cut, an `error`-frame crash and an orphaned tool. The `event`,
`tests/_fixtures/golden_streams/delegate_fixtures.py` scripts the host's stdout frames for six
scenarios: a tool call, a `max_turns` cut, an `error`-frame crash, an orphaned tool, a tool-only call
that crosses the cap, and a batched reply whose tools run one at a time. The `event`,
`result` and `error` frames mirror the live `1.202.1` transcript. The `usage` frame and the `result`
fields new in `1.203.0` follow the host source of UiPath/Autopilot#6656 (`src/usage.ts` and
`handleSend`): every usage object carries all four buckets with `input_tokens` exclusive of cache,
Expand Down
2 changes: 1 addition & 1 deletion .claude/notes/lint-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -845,7 +845,7 @@ assertion for each. A defaulted `AgentStartEvent` lands ~365 days before the clo
first window, so the head blows any sane UPPER bound by the whole offset. A defaulted
`AgentEndEvent` lands ~365 days BEFORE its own last message, so `decompose_turn` clamps the
negative to `0.0` — "measured, and instant", which passes an upper bound. Only a strict
`> 0.0` catches it. It holds on all three clocked harnesses because real work separates a
`> 0.0` catches it. It holds on every clocked harness because real work separates a
turn's last flush from its end event. The margin is smallest on antigravity, which holds
its process across turns: it measures 0.007-0.035 ms there, 7-35 ticks of the 1 us
resolution that both `datetime` and `time.monotonic()` have on Linux, macOS and Windows.
Expand Down
7 changes: 6 additions & 1 deletion .claude/notes/timing.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@ removes:
transition or an NTP step inside a turn lands directly in a generation window — an
hour-long jump in a millisecond field. Nightly runs start at 04:18 and run for hours,
so it is reachable rather than theoretical. A monotonic-derived stamp cannot express it.
- Delegate stamped its one window and its tool spans with raw `datetime.now()` but measured
`duration_seconds` with `time.monotonic()`. On Windows the wall clock moved 0.24-0.37 ms
further than the monotonic one inside a sub-millisecond turn, so the tail alone came out
longer than the whole turn and `test_delegate_golden[a_tool_call]` failed at random. One
window has no inter-window drift to correct, but it still shares the turn's duration.

It is an EXTRACTION, not an invention: antigravity already captured this exact pair at
the top of `communicate` and simply did not use it for later stamps.
Expand Down Expand Up @@ -205,7 +210,7 @@ Magnitudes are only real where a scripted clock makes them real, so each case dr
harness's own reducer with a hand-moved clock and feeds its output through a real
`EventCollector`, the seam production uses to compute the head and the tail. The module
reuses the three clock-injection idioms the per-harness suites already use rather than
inventing a fourth: an injected `TurnClock` stand-in (pi, antigravity, claude-code), a
inventing a fourth: an injected `TurnClock` stand-in (pi, antigravity, claude-code, delegate), a
`datetime` subclass patched onto the module (opencode), and scripted SDK epoch-millisecond
stamps (codex). OpenCode needs a `datetime` SUBCLASS rather than a stub because the
reducer also calls `datetime.fromtimestamp` through the same module global to convert the
Expand Down
4 changes: 2 additions & 2 deletions docs/agents/DELEGATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -168,8 +168,8 @@ Run-limit semantics per harness: [Run-Limit Parity](HARNESS_PARITY.md).
## Known Limitations

1. **No multi-generation transcript splitting.** This agent builds one `AssistantMessage` per `communicate()` call, not one per backend round-trip. The host does send the signals a split needs (`isStepStart` on `message` events and per-round-trip `turnUsages` on `result`), but this agent does not use them yet.
2. **`max_turns` is enforced by coder_eval, not by the host.** The host's `maxSteps` option on `send` does not stop the turn (confirmed live: `maxSteps: 2` ran 7 steps and only reported `maxStepsReached: true`), so this agent does not send it. See [Run-Limit Parity](HARNESS_PARITY.md).
A turn cut at `max_turns`, or by a cooperative early stop, keeps the token usage and cost of every model call that finished before the cut, as on the other agents. The host sends a `usage` frame for each call before that call's tool results. The call in progress at the cut has no usage. A host that does not send `usage` frames reports usage only on its final `result` frame, which a cut turn never gets: that turn has no usage, and the agent logs a warning.
2. **`max_turns` is enforced by coder_eval, not by the host.** The host's `maxSteps` option on `send` does not stop the turn (confirmed live: `maxSteps: 2` ran 7 steps and only reported `maxStepsReached: true`), so this agent does not send it. It counts a call on the call's first streamed text, or on its `usage` frame when the call streams none. The host runs a batched reply's tools one at a time, so tool events do not mark a call. See [Run-Limit Parity](HARNESS_PARITY.md).
A turn cut at `max_turns`, or by a cooperative early stop, keeps the token usage and cost of every model call that finished before the cut, as on the other agents. The host sends a `usage` frame for each call before that call's tool results. Which side of the cut the crossing call lands on depends on its shape. A call that streams text is cut at that text, before its `usage` frame and before any of its tools, so its usage is dropped. A tool-only call is cut at its own `usage` frame, so its usage is booked, and its first tool can start before the kill: the host starts a call's tools right after the frame, and the stream gives no earlier boundary. When the `result` arrives, the agent compares its own call count with the host's `turnUsages` and logs a warning if they differ, because a change in the host's frame order would move the cap. A host that does not send `usage` frames reports usage only on its final `result` frame, which a cut turn never gets: that turn has no usage, and the agent logs a warning.
3. **Only three backend failures get a specific diagnosis.** A Cloudflare WAF block page is reported as a content-filter failure and is not retried: the same prompt or tool result is blocked again. An SSE connect timeout is reported as a connection failure and is retried. A session conflict ("A reply is already being generated") is retried in a new conversation when no turn of that conversation has finished. After a finished turn, or with a `session_id` set in the config, the retry stays in the same conversation, so the agent never continues without its earlier turns: the retry succeeds once the backend releases the conversation (it stops the reply when the host disconnects), or the task ends as an error. There is no first-response stall detection: a stalled turn ends at its turn timeout. Every other crash ends the turn as an `AgentCrashError`, which is retried unless its message matches a category that is not retried (a timeout, or an auth, billing or content-filter error).
4. **`sdk_options` accepts only `effort`.** Reasoning effort uses the same `sdk_options.effort` key as Claude Code, so `-D agent.sdk_options.effort=high` works for both agents. Any other key, or an `effort` that is not a string, is a validation error. The host checks the tier itself: it logs a tier it does not recognize and uses the model's default. `project_id`, `session_id` and `enable_computer_use` are typed config fields.
5. **`enable_computer_use: true` requires local permissions and is unavailable on Linux.** macOS needs Accessibility + Screen Recording grants; the SDK throws unconditionally on Linux when this is enabled.
Expand Down
32 changes: 17 additions & 15 deletions docs/agents/HARNESS_PARITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ This page is the contract for what each run limit means per harness, plus the sh

| Limit | claude-code | codex | antigravity | opencode | pi | delegate |
|---|---|---|---|---|---|---|
| `run_limits.max_turns` (main-thread model API calls on every harness, see below) | native CLI cap, plus a harness backstop when call N+1 begins | a call runs from its first item to its `thread/tokenUsage/updated`, and one that ran tools opens the next call there | a call runs from its first new MODEL step to the step carrying its usage | one `step_start` step | one `turn_start` turn | a call opens when the previous call's tools have all returned |
| `run_limits.max_turns` (main-thread model API calls on every harness, see below) | native CLI cap, plus a harness backstop when call N+1 begins | a call runs from its first item to its `thread/tokenUsage/updated`, and one that ran tools opens the next call there | a call runs from its first new MODEL step to the step carrying its usage | one `step_start` step | one `turn_start` turn | a call opens on its first streamed text, or on its `usage` frame when it streams none; a tool-only call N+1's first tool can start before the kill |
| `run_limits.turn_timeout` | watchdog, SIGKILL on the CLI subprocess | watchdog + cooperative interrupt | watchdog, plus an earlier internal poll deadline at 80% of it (see below) | deadline enforced in-loop and on the final reap; SIGTERM→SIGKILL on the CLI's whole process group | deadline enforced in-loop and on the final reap; SIGTERM→SIGKILL on the CLI's whole process group | deadline checked both between reads and while blocked inside one (`asyncio.wait_for`); force-kills the host subprocess and drops the handle so the next turn respawns |
| `run_limits.task_timeout` | orchestrator-level, agent-agnostic | orchestrator-level, agent-agnostic | orchestrator-level, agent-agnostic | orchestrator-level, agent-agnostic | orchestrator-level, agent-agnostic | orchestrator-level, agent-agnostic |
| `run_limits.stop_early` | cooperative `should_stop` | cooperative `should_stop` | cooperative `should_stop` | cooperative `should_stop` (event granularity) | cooperative `should_stop` (event granularity — Pi streams incrementally) | cooperative `should_stop`, polled per forwarded SDK event; the host is abandoned (no interrupt command exists) and a fresh one spawns for the next turn |
Expand All @@ -34,8 +34,8 @@ wall clock its numbers account for.
| `generation_completed_at` | set | `None` — see below | `None` | `None` | `None` | set |
| `message_id` source | SDK `message_id`; `None` when the stream carries none; `subagent-<tool_use_id>` for a synthesized sub-agent terminal | synthetic `turn_id-msg-N`, shared across the sub-messages of one generation; `turn_id-subagent-N` for recovered sub-agent generations | synthetic `turn_id-msg-N`, one per generation | CLI `messageID`; `None` when absent | CLI `responseId`; `None` when absent | synthetic `<turn_id>-msg-0` (exactly one per turn) |
| `Σ generation + ∪ tool + head + tail ≈ turn duration` | yes [^identity] | yes [^identity] | yes [^identity] | yes [^identity] | yes [^identity] | yes, trivially — one window IS the turn |
| clock basis for recorded stamps | one `TurnClock` per turn | SDK epoch ms (`_ms_to_dt`) — the subprocess's own clock, unreachable from the host, for BOTH window bounds and tool spans | one `TurnClock` per turn | **MIXED**: window bounds on the host `datetime.now()` (`:362`, `:696`); tool spans on CLI epoch ms (`_epoch_ms_to_dt`, `:406`/`:462`) | one `TurnClock` per turn | raw `datetime.now()` (no `TurnClock` — CE064 does not require one, since there is only one window and no inter-window drift it could correct) |
| turn bracket (`AgentStartEvent` / `AgentEndEvent`) stamp | the same `TurnClock` (**CE064**) | raw `datetime.now()` — consistent with its epoch-ms bounds | the same `TurnClock` (**CE064**) | raw `datetime.now()` — consistent with its epoch-ms tool spans | the same `TurnClock` (**CE064**) | raw `datetime.now()` — consistent with its own single-window bounds |
| clock basis for recorded stamps | one `TurnClock` per turn | SDK epoch ms (`_ms_to_dt`) — the subprocess's own clock, unreachable from the host, for BOTH window bounds and tool spans | one `TurnClock` per turn | **MIXED**: window bounds on the host `datetime.now()` (`:362`, `:696`); tool spans on CLI epoch ms (`_epoch_ms_to_dt`, `:406`/`:462`) | one `TurnClock` per turn | one `TurnClock` per turn (its raw `datetime.now()` bounds drifted from the monotonic `duration_seconds` on Windows and put the tail past the whole turn) |
| turn bracket (`AgentStartEvent` / `AgentEndEvent`) stamp | the same `TurnClock` (**CE064**) | raw `datetime.now()` — consistent with its epoch-ms bounds | the same `TurnClock` (**CE064**) | raw `datetime.now()` — consistent with its epoch-ms tool spans | the same `TurnClock` (**CE064**) | the same `TurnClock` (**CE064**) |
| window built by `timing.py::close_window` | yes | yes | yes | yes | yes | yes |

[^identity]: "yes" is load-bearing, and THREE sensors check it, each seeing
Expand Down Expand Up @@ -126,8 +126,8 @@ Two consequences worth stating, because both are behaviour changes:
(not on `message_id`, which OpenCode and Pi can legitimately leave `None`),
subtracts the overlap once, and re-apportions so the parts still sum.

**Two clock bases remain, and the row above says which.** Antigravity, Pi and
claude-code derive every recorded wall stamp from one `TurnClock` per turn, so
**Two clock bases remain, and the row above says which.** Antigravity, Pi,
claude-code and Delegate derive every recorded wall stamp from one `TurnClock` per turn, so
a turn's bounds and the tool spans subtracted from them cannot disagree, and
neither can be moved by a DST transition or an NTP step inside the turn.
Antigravity needed it first: its span was monotonic while its tool intervals
Expand Down Expand Up @@ -555,16 +555,18 @@ Each harness finds the call boundary in its own stream (the table above):
step, so a rise in that total closes the call, and the next call opens with a MODEL
step at a new `step_index`.
- **OpenCode** and **Pi** stream one `step_start` or `turn_start` per call.
- **Delegate**'s event stream marks a round-trip only on a text reply
(`isStepStart`), and a tool-only reply streams only its tool call, with no text
before it. The host's own `maxSteps` option does not stop the turn, so the
adapter does not send it and enforces the cap itself. So the next call opens when every tool the
previous call announced has returned, since those results go back to the model,
and the cap fires before call N+1 can run anything. A reply that announces
several tools before their results counts once. If a tool never returns, the
next call opens at the model's next text or new tool call instead.
The host sends each call's `usage` frame before that call's tool results, so
the calls under the cap keep their tokens and cost.
- **Delegate** streams a call's text, then writes the call's `usage` frame, then
runs the call's tools. It runs a batched reply's tools one at a time, so one call
streams `tool_call`, `tool_result`, `tool_call`, `tool_result`, and a tool event
cannot mark a call boundary. A call therefore opens on its first `thinking` or
`message` event. A tool-only reply streams no text, so it opens on its `usage`
frame. A call N+1 that streams text is cut at that text, before any of its tools.
A tool-only call N+1 is cut at its `usage` frame, and its first tool can start
before the kill: the host starts a call's tools right after the frame, and the
stream gives no earlier boundary. The host's own `maxSteps` option does not stop
the turn, so the adapter does not send it and enforces the cap itself. A host that
sends no `usage` frames counts only the calls that stream text. The calls under
the cap keep their tokens and cost.

Every harness enforces the cap on the same loop boundary as the
cooperative early stop, then kills or cancels the in-flight turn so the cap stops
Expand Down
Loading
Loading