Skip to content

fix(agbossjs/agboss#720): the child queue's claim marks exactly the child it answers — a MATERIALIZED CTE, run once; core at 06c4bc1d - #40

Merged
PavelRavvich merged 3 commits into
mainfrom
fix/720-claim-one-row
Nov 4, 2026
Merged

PavelRavvich merged 3 commits into
mainfrom
fix/720-claim-one-row

Conversation

@PavelRavvich

Copy link
Copy Markdown
Member

Refs agbossjs/agboss#720

What

The child queue's claim, UPDATE … FROM (SELECT … LIMIT 1 FOR UPDATE SKIP LOCKED) first, could mark many children running in one statement and return one. When the queue was analyzed while it held about one row, PostgreSQL ran it as a nested loop with the subquery on the inner side, rescanned once per row of the table. Each rescan skipped the rows the statement had already updated and locked the next queued one.

That is the queue right after a batch of one, such as the Kubernetes blueprint's frozen-pod step. On kind, the next batch's children were then left running for a pod that never heard of them. With the core's reclaim bug (fixed in agbossjs/agboss#758), they stayed stuck under a live lease.

The choice is now WITH first AS MATERIALIZED (SELECT … FOR UPDATE SKIP LOCKED LIMIT 1) UPDATE … FROM first. The CTE runs once, and a rescan reads its one stored row. The README's table of atomic operations and the store's doc comments say so.

The core is pinned at 06c4bc1d (agbossjs/agboss#758). It brings:

  • the queue contract case "a claim marks exactly the child it answers: every other stays queued";
  • fault scenario s16-claim-answer-lost.

Evidence

  • tests/stores/child-execution-queue.test.ts (new case): analyze a one-row queue, queue 48 children, claim once. Before the fix, 16 rows were marked running by that one statement. Now exactly one.
  • At the SQL level (psql, the same state): Subquery Scan on first (loops=48), Update … (actual rows=48) before; the CTE form gives CTE Scan (loops=48), Update … (actual rows=1).
  • Captured live on 4 local processes of the example app, after a SIGSTOP/SIGCONT step: rowCount 2–5 from single claims. The session was plain READ COMMITTED, no open transaction, with the plan above (AnyInstance children stuck forever: a pod reclaims its own fresh claim and holds the child's lease with no pass running it agboss#720, comment).

Local gate

On a dedicated PostgreSQL 16 container, under the shared lock:

  • npm run test:faults: 32/32, including 16: a pod's claims of a child answered but never heard, twice.

  • npm run check: everything passes except two cases that need more CPU than this loaded machine gives:

    • the lease interleaving explorer, cut at its 900 s limit;
    • one clock-authority real-time bite, which passes run alone (all six bites green).

    Neither touches the child queue, and neither changes between the old and new pin. On GitHub the whole check takes about 14 minutes. CI is the judge for those two.

🤖 Generated with Claude Code

https://claude.ai/code/session_012Kh8JmysuhZyQRSpNyJdPg

PavelRavvich and others added 3 commits November 4, 2026 16:02
…hild it answers — the choice is a MATERIALIZED CTE, run once

`UPDATE … FROM (SELECT … LIMIT 1 FOR UPDATE SKIP LOCKED) first` was planned, on statistics taken
while the queue held about one row, as a nested loop rescanning the subquery once per row of the
table: each rescan skipped the rows the statement had already marked and locked the next, so one
claim marked many children `running` for the pod and returned one. The others ran nowhere, and on
the core's side a reclaim of them could keep a lease with no pass behind it (agbossjs/agboss#720).
That is the queue right after a batch of one, such as the Kubernetes blueprint's frozen-pod step.

The choice is now `WITH first AS MATERIALIZED (…) UPDATE … FROM first`: the CTE runs once and its
rescans read its one row. tests/stores/child-execution-queue.test.ts analyzes a one-row queue,
queues 48 children and claims once: 16 rows were marked before, exactly one now.

Refs agbossjs/agboss#720

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Kh8JmysuhZyQRSpNyJdPg
…ract case and fault scenario 16

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Kh8JmysuhZyQRSpNyJdPg
…laim's doc comment

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Kh8JmysuhZyQRSpNyJdPg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant