Skip to content

fix(atomic): the WinError 5 race had four more copies, including the lock store (2.118.2) - #158

Merged
drknowhow merged 1 commit into
mainfrom
fix/atomic-json-shared-writer
Sep 5, 2026
Merged

drknowhow merged 1 commit into
mainfrom
fix/atomic-json-shared-writer

Conversation

@drknowhow

Copy link
Copy Markdown
Owner

2.118.1 (#157) fixed the publish step in the shell job store: a shared <name>.tmp plus a single os.replace, which on Windows can publish a half-written file or raise PermissionError: [WinError 5] Access is denied. That shape had been copied into four other small JSON stores, and fixing one copy of a bug is not fixing the bug.

What changed

All five now go through one helper, services/atomic_json.py — unique temp (<name>.tmp<pid>-<random>), four os.replace attempts with backoff, a finally-unlink, and an fsync.

Call site Why it matters
services/agent_locks.py:_save The lock store, and the most exposed of the four: written from a threaded server, locks.json.tmp is a name every writer on the box picks, and a lost write drops live leases — exactly what _load refuses to do when it quarantines a corrupt file rather than resetting it.
services/access_guard._write_scope_config The CLI's config.json writer.
cli/hub_server.py (config route) The hub, writing the same file.
oracle/services/mobile_api.py (_write_override_section) The mobile API, ditto.

The last three write one file from three processes, one of them threaded.

None had a reported failure. That is a statement about how often they are written, not about whether the race is real — the job store's version was also unreported until it started failing windows-latest CI.

Also: durable config is flushed before it is published

write_json_atomic fsyncs the temp before the replace, so a crash between the two cannot leave a zero-length config.json. The Access Guard reads a truncated config as <corrupt-config> and fails every path closed, which wedges the session that would otherwise fix it. The shell job store opts out (fsync=False) — its records are ephemeral spill state that a crash invalidates anyway.

Serialisation is unchanged at all five sites: indent, ensure_ascii and the trailing newline are per-caller arguments, so no file on disk reformats.

cli/_hook_utils._atomic_write_json keeps its own copy on purpose — a PreToolUse hook is a separate short-lived single-threaded process that imports nothing from services, so pid-only is genuinely enough there. That is now the only remaining duplication, and it is documented as deliberate in both places.

Tests

tests/test_atomic_json.py: temp uniqueness (per write, not per pid — threads share a pid), retry on transient PermissionError vs. re-raise on a persistent one, no retry for a non-PermissionError OSError, no orphaned temps on the failure path, serialisation parity for each caller's flags, the fsync policy in both directions, a 6-thread hammer, and a source scan asserting no call site re-derives the shape.

Verification

  • Full suite on Windows: 3675 passed, 6 skipped
  • ruff check .: clean
  • Local container CI: lint PASS, jsx PASS, test (ubuntu-latest, 3.12) PASS. Not a FULL_CI_PASS — macOS cells cannot run on this host and the Windows cells are refused by the native engine because they would pip install onto the box.

https://claude.ai/code/session_01JjAQC4Uv3aBnv9oQLPMBMy

…lock store (2.118.2)

2.118.1 fixed the publish step in the shell job store: a shared `<name>.tmp`
plus a single `os.replace`, which on Windows can publish a half-written file
or raise `PermissionError: [WinError 5] Access is denied`. That shape had
been copied into four other small JSON stores, and fixing one copy of a bug
is not fixing the bug.

All five now go through one helper, `services/atomic_json.py`:

- `services/agent_locks.py` — the lock store, and the most exposed of the
  four: written from a threaded server, `locks.json.tmp` is a name every
  writer on the box picks, and a lost write drops live leases. That is
  exactly what its loader refuses to do when it quarantines a corrupt file
  rather than resetting it to empty.
- `services/access_guard._write_scope_config` — the CLI's `config.json`.
- `cli/hub_server.py` — the hub's config route, writing the same file.
- `oracle/services/mobile_api.py` — the mobile override-config writer, ditto.
  The last three write one file from three processes, one of them threaded.

None had a reported failure. That is a statement about how often they are
written, not about whether the race is real — the job store's version was
also unreported until it started failing CI.

`write_json_atomic` also fsyncs the temp before the replace, so a crash
between the two cannot leave a zero-length `config.json`. The Access Guard
reads a truncated config as `<corrupt-config>` and fails every path closed,
which wedges the session that would otherwise fix it. The shell job store
opts out (`fsync=False`): its records are ephemeral spill state that a crash
invalidates anyway.

Serialisation is unchanged at all five sites — indent, `ensure_ascii` and the
trailing newline are per-caller arguments, so no file on disk reformats.

`cli/_hook_utils._atomic_write_json` keeps its own copy on purpose: a
PreToolUse hook is a separate short-lived single-threaded process that
imports nothing from `services`, so pid-only is genuinely enough there.

tests/test_atomic_json.py covers temp uniqueness (per write, not per pid —
threads share a pid), retry vs. re-raise, no orphaned temps, serialisation
parity, the fsync policy, a 6-thread hammer, and a source scan asserting no
call site re-derives the shape.

Local: 3675 passed / 6 skipped on Windows; ruff clean; container CI lint,
jsx and test (ubuntu-latest, 3.12) all pass.

Claude-Session: https://claude.ai/code/session_01JjAQC4Uv3aBnv9oQLPMBMy
@drknowhow
drknowhow merged commit c71842d into main Sep 5, 2026
12 checks passed
@drknowhow
drknowhow deleted the fix/atomic-json-shared-writer branch September 5, 2026 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant