Repository navigation
fix(atomic): the WinError 5 race had four more copies, including the lock store (2.118.2) - #158
Merged
Merged
Conversation
…lock store (2.118.2) 2.118.1 fixed the publish step in the shell job store: a shared `<name>.tmp` plus a single `os.replace`, which on Windows can publish a half-written file or raise `PermissionError: [WinError 5] Access is denied`. That shape had been copied into four other small JSON stores, and fixing one copy of a bug is not fixing the bug. All five now go through one helper, `services/atomic_json.py`: - `services/agent_locks.py` — the lock store, and the most exposed of the four: written from a threaded server, `locks.json.tmp` is a name every writer on the box picks, and a lost write drops live leases. That is exactly what its loader refuses to do when it quarantines a corrupt file rather than resetting it to empty. - `services/access_guard._write_scope_config` — the CLI's `config.json`. - `cli/hub_server.py` — the hub's config route, writing the same file. - `oracle/services/mobile_api.py` — the mobile override-config writer, ditto. The last three write one file from three processes, one of them threaded. None had a reported failure. That is a statement about how often they are written, not about whether the race is real — the job store's version was also unreported until it started failing CI. `write_json_atomic` also fsyncs the temp before the replace, so a crash between the two cannot leave a zero-length `config.json`. The Access Guard reads a truncated config as `<corrupt-config>` and fails every path closed, which wedges the session that would otherwise fix it. The shell job store opts out (`fsync=False`): its records are ephemeral spill state that a crash invalidates anyway. Serialisation is unchanged at all five sites — indent, `ensure_ascii` and the trailing newline are per-caller arguments, so no file on disk reformats. `cli/_hook_utils._atomic_write_json` keeps its own copy on purpose: a PreToolUse hook is a separate short-lived single-threaded process that imports nothing from `services`, so pid-only is genuinely enough there. tests/test_atomic_json.py covers temp uniqueness (per write, not per pid — threads share a pid), retry vs. re-raise, no orphaned temps, serialisation parity, the fsync policy, a 6-thread hammer, and a source scan asserting no call site re-derives the shape. Local: 3675 passed / 6 skipped on Windows; ruff clean; container CI lint, jsx and test (ubuntu-latest, 3.12) all pass. Claude-Session: https://claude.ai/code/session_01JjAQC4Uv3aBnv9oQLPMBMy
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
2.118.1 (#157) fixed the publish step in the shell job store: a shared
<name>.tmpplus a singleos.replace, which on Windows can publish a half-written file or raisePermissionError: [WinError 5] Access is denied. That shape had been copied into four other small JSON stores, and fixing one copy of a bug is not fixing the bug.What changed
All five now go through one helper,
services/atomic_json.py— unique temp (<name>.tmp<pid>-<random>), fouros.replaceattempts with backoff, afinally-unlink, and an fsync.services/agent_locks.py:_savelocks.json.tmpis a name every writer on the box picks, and a lost write drops live leases — exactly what_loadrefuses to do when it quarantines a corrupt file rather than resetting it.services/access_guard._write_scope_configconfig.jsonwriter.cli/hub_server.py(config route)oracle/services/mobile_api.py(_write_override_section)The last three write one file from three processes, one of them threaded.
None had a reported failure. That is a statement about how often they are written, not about whether the race is real — the job store's version was also unreported until it started failing windows-latest CI.
Also: durable config is flushed before it is published
write_json_atomicfsyncs the temp before the replace, so a crash between the two cannot leave a zero-lengthconfig.json. The Access Guard reads a truncated config as<corrupt-config>and fails every path closed, which wedges the session that would otherwise fix it. The shell job store opts out (fsync=False) — its records are ephemeral spill state that a crash invalidates anyway.Serialisation is unchanged at all five sites: indent,
ensure_asciiand the trailing newline are per-caller arguments, so no file on disk reformats.cli/_hook_utils._atomic_write_jsonkeeps its own copy on purpose — a PreToolUse hook is a separate short-lived single-threaded process that imports nothing fromservices, so pid-only is genuinely enough there. That is now the only remaining duplication, and it is documented as deliberate in both places.Tests
tests/test_atomic_json.py: temp uniqueness (per write, not per pid — threads share a pid), retry on transientPermissionErrorvs. re-raise on a persistent one, no retry for a non-PermissionErrorOSError, no orphaned temps on the failure path, serialisation parity for each caller's flags, the fsync policy in both directions, a 6-thread hammer, and a source scan asserting no call site re-derives the shape.Verification
ruff check .: cleanlintPASS,jsxPASS,test (ubuntu-latest, 3.12)PASS. Not aFULL_CI_PASS— macOS cells cannot run on this host and the Windows cells are refused by the native engine because they wouldpip installonto the box.https://claude.ai/code/session_01JjAQC4Uv3aBnv9oQLPMBMy