Skip to content

Revisions batch: on-ramp + model currency + D13 scope row (spec), six guide rules, first full live-AC run (6/6 after the thread-identity fix) - #25

Merged
Carr1005 merged 14 commits into
mainfrom
guide-revisions-batch
Sep 10, 2026
Merged

Revisions batch: on-ramp + model currency + D13 scope row (spec), six guide rules, first full live-AC run (6/6 after the thread-identity fix)#25
Carr1005 merged 14 commits into
mainfrom
guide-revisions-batch

Conversation

@Carr1005

@Carr1005 Carr1005 commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

The revisions batch from the tester-feedback triage (#24) plus the promoted #18 findings — spec edits and their guide generalizations, validated Option-B style (surgical edits + clean-room Opus regens; no adoption, no rebuild). Also includes the lab's first full keyed live-AC run, which caught and fixed a real design gap.

Spec (3 commits)

  • 2835655on-ramp + currency: the opening blockquote gains the orientation lead ("what you're building" as a user-visible scene) and the fastest-path/completion sentence (offline suite green = the recommended baseline is complete; keys unlock only live tests/live use); inline heavy-setup notes on D11 Oracle and D12 Tavily; D4 Options name providers, never "current" model names ("expect the build agent to recommend current ones at the gate"); CTX-D gains the request-parameter-compatibility perishable (the temperature-rejection class); fixtures/llm-stubs.pyllm_stubs.py.
  • 809dea8D13 summary-memory-scope row (owner decision, first contradicted-category row): thread-scoped default (the app path) vs the Lesson-5 notebook's unscoped global pool — both course-demonstrated; stakes test passes because the choice changes what a read returns. Side edits: contradicted-group paragraph, R18 defers to D13, §5 coverage note, CTX-C11 cross-ref.
  • R11 harness-known-identity sentence — added mid-batch after the live AC21 run caught the gap (below), per the spec's own living-document rule.

Guide (1 commit, 0cd5ff2)

Six course-agnostic rules: G1 orientation-block template + reader test; G2 heavy options state setup weight inline at choice time; G3 model currency is the generation-vs-gate timing split, CTX-D parameter-constraint category; G4 identifier formats are Pass-A parameters, alphabet included; G5 fixture module filenames must be importable identifiers; G6 the contradicted route's worked storage-scoping example. Reference-style §14 lines.

Validation

  • 3/3 clean-room Opus regens: all six new V-properties present in every regen (B full-scored 30/30 PRESENT, 0 PARTIAL with quoted evidence + orchestrator spot-audits; A and C grep-verified). G4 closed the id-alphabet cross-model gap 3/3 (was 0/3); G6 held the scope row 3/3; regen-B is the first exact-canon 13/13 Ledger in any pass. Carried residual expressed as documented (A +1 row, C −1) — logged, not fix-rounded.
  • Stage-1 gate probe (Opus): cold-start build agent still stops at the gate with the new blockquote; relayed the completion criterion correctly.
  • Matrix: docs/research/guide-revisions-batch-matrix.md (decision log incl. F8 parked and Recording: Claude Code transcript compaction can erase the /prepare-build announcement bookend #19 excluded by owner).

Live-AC run (run-09, owner-supplied key) — the headline

First keyed run: AC4, AC9, AC19, AC22, AC24 passed live. AC21 — the course's cross-session continuity demo — failed deterministically: the assembled context never tells the model its thread id, so it called summarize_and_store with a guessed "current_thread" and the tool honestly consolidated nothing. The oracle worked exactly as designed: the spec gained the missing constraint (R11: harness-known identities are injected, never model-supplied), the run-09 build was conformed (~20 lines), offline suite re-green 21/21, and AC21 passed on rerun — live suite 6/6. The course-agnostic guide rule ("harness-known identities are injected, never model-guessed") is recorded in the matrix as the next pass's first candidate, per fix-and-test-travel-together.

Issue routing

🤖 Generated with Claude Code

Carr1005 and others added 10 commits September 9, 2026 12:20
…ency updates, importable fixture filename

Owner-approved texts from the #24 batch: blockquote gains 'what you're
building' + fastest-path/completion sentences (R16/R18 density feedback;
keyless finish blessed); D11/D12 course-default entries carry inline setup
weight; D4 Options name providers not current model names, with gate-time
recommendation deferred to the build agent; CTX-D gains the perishable
request-parameter-behavior category (temperature rejection); §2 key wording
acknowledges compatible endpoints; fixtures/llm-stubs.py -> llm_stubs.py.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…o canon)

First contradicted-category row: thread-scoped default (the app path) vs the
Lesson-5 notebook's unscoped global pool, both course-demonstrated; stakes
test passes because the choice changes what a summary read returns. Side
edits: contradicted-group paragraph updated; R18 defers to D13 (no deciding
twice); §5 coverage note and CTX-C11 cross-reference D13; provenance note
records the batch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…nline heavy-option weights, model-currency timing, id alphabets, importable fixture filenames, scoping-contradiction example

G1 blockquote anatomy: orientation lead (goal-row default as a user-visible
interaction, never a component inventory) + fastest-path/completion sentence.
G2 §5.5 Options: heavy options state setup weight inline at choice time.
G3 §6.0 + CTX-D anatomy: model currency is the canonical generation-vs-gate
timing split; request-parameter-compatibility perishables.
G4 Pass A: identifier formats are working parameters, alphabet included.
G5 §5 anatomy: fixture module filenames must be importable identifiers.
G6 §5.5 contradicted route: worked storage-scoping example that passes the
stakes test (stabilizes the D13-class subject).
§14: reference-style audit lines for G1/G2/G3/G5; G4 rides the existing
Pass-A coverage line.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…1-V6 properties

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…1 residual class recurrence

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… (augmentation fold)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…marization (R11)

AC21's live run caught the gap deterministically: the model, never told its
thread id, called summarize_and_store with a guessed 'current_thread' and the
tool honestly consolidated nothing. The constraint is now stated; AC21 is its
end-to-end test. Living-document rule applied; guide-level generalization
(harness-known identities are injected, never model-guessed) queued as a
pass finding.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ct PASS; AC21 finding + live-run record

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Carr1005 and others added 2 commits September 10, 2026 11:23
The 2026-09-09 wording named summarize_and_store in the normative clause; an
independent trial build (fresh session, spec-only) built the general injection
mechanism but applied it to exactly that tool, leaving expand_summary unscoped
and weakening D13's thread isolation. Wording-discipline violation #2 (anchored
to a course specific instead of a concept), caught by the disposable-clone
trial. Now covers every thread-scoped agent-triggered operation, requires the
parameter be absent from the model-facing schema, and preserves genuinely
model-held identifiers (summary ids read from [Summary ID: ...] references).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…olved-decisions.md

Spec §0 step 5 + §7 evidence list; guide §6.0 checklist-echo rule (with the
why: screen output cannot be verified afterwards, and a fixed name is what
makes compliance checkable), the emitted §0 template, the §0 anatomy comment,
the express-lane bullet, and two §14 audit lines. The file is a record, never
an input — a new or resumed build re-runs the gate from the spec and never
reads a previous checklist; each run overwrites it.

Prompted by the disposable-clone trial: across three builds the checklist
landed in three different places (two of them files no spec mentions), so gate
compliance has never been checkable from a delivered tree.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@Carr1005

Copy link
Copy Markdown
Collaborator Author

Two fixes added mid-review (commits eb953a0, 0813752), both prompted by a disposable-clone trial run after this PR opened: the spec was copied alone into an empty folder and built by a fresh session with zero project context. That build came out green (50 passed / 6 skipped) and scored 9/10 on an independent eval — and it exposed two defects in the assets this PR touches.

1. eb953a0 — R11's harness-identity rule was too narrow (my defect from this PR's own R11 commit). The wording named summarize_and_store inside the normative clause. The trial built the general mechanism — injected params stripped from the model-facing schema, harness values overriding model-supplied ones — but applied it to exactly that one named tool, leaving expand_summary unscoped and quietly weakening D13's thread isolation. That is wording-discipline violation #2 (anchored to a course specific rather than a concept). The rule now covers every thread-scoped agent-triggered operation, requires the parameter be absent from the model-facing schema, and states explicitly that identifiers the model legitimately holds (a summary id read from a [Summary ID: ...] reference) stay model-supplied.

2. 0813752 — the gate checklist is now a durable artifact. §0 told the build agent to print the resolved-decision checklist; §7 then cited "the printed resolved-decision checklist from §0" as evidence — demanding durable proof of something that only ever existed on screen. The drift this caused is measurable: across three builds the checklist landed in resolved-decisions.md (a probe build, spontaneously), inside BUILD-REPORT.md (run-09), and partially inside README.md (the trial — 3 of 13 rows) — three places, two of them files no spec mentions, so no eval could ever verify the gate ran. It is now printed and written to a fixed-name file, resolved-decisions.md, in the build folder, with the reasoning in the guide's §6.0 rule. Owner constraint captured in the wording: the file is a record, never an input — a new or resumed build re-runs the gate from the spec and never reads a previous run's checklist, and each gate run overwrites it (no accumulation; build folders are gitignored anyway).

Files to re-read: courses/agent-memory-building-memory-aware-agents/spec.md (§0 step 5, §7 evidence line, R11) and .claude/skills/generate-spec/references/spec-generation-guide.md (§6.0 checklist-echo rule, the emitted §0 template step 5, the §0 anatomy comment, the express-lane bullet, two §14 audit lines).

A template-conformance validation regen is running against the new guide commit; its result and the accepted residual (the behavioral check — does a build actually write the file — rides the next trial rather than a paid probe build) are recorded in docs/research/guide-revisions-batch-matrix.md. Four further trial findings are deferred to the next pass and logged there, not applied here.

🤖 Generated with Claude Code

Carr1005 and others added 2 commits September 10, 2026 11:47
…guide half shown to be needed

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Owner review: the 'mechanically checkable' framing was false (the file is
forgeable — only probes test gate behavior) and the '3/3 builds wrote it
unprompted' evidence was inflated (one clean instance, one was our own build,
one partial). Removed the false framing from the guide's §6.0 rule; the stated
purpose is now what survives scrutiny: a recoverable decision record in a
fixed place instead of whatever file the agent invents.

Prose cost cut ~60%: guide +256 -> +104 words, spec step 5 +66 -> +26. Guide
and spec template text verified byte-identical, so regen-D's validation still
covers the (content-defined) requirement set. Matrix records the correction,
the deliberate economy and its risk, and the batch's size accounting.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant