Repository navigation
feat: prompt sections and per-thread foundations as transcript state, sent in place across providers - #116
Merged
Conversation
The foundation goes out as the system prompt; every tagged section is declared by a system entry. The first request declares them all, a later one only the sections that changed, with any tool change, so resume and fork replay what the model saw and in-place models keep their cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The system entry now lands before the prompt it comes with, so the model reads the setup, then the request. The finished-reply guard and fork-before skip system entries, so a declaration left by a cancelled prompt is inert. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ponses Anthropic (Opus 4.8/5/5.5, Sonnet/Haiku 5.5, Fable/Mythos 5/5.1) and direct OpenAI Responses models now take later system messages in place through the AI SDK adapters, so a prompt section change keeps the cached prefix. On Anthropic a later system message is held for the next assistant message, as the API takes one only after a user message; one ahead of the conversation joins the leading prompt, since a trailing system message's cache entry is never read once a reply follows it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Transcript.diffSections replaces the harness copy of the section diff. - Copilot Claude models take system messages in place, as direct Anthropic. - An Anthropic system message ahead of the conversation is never held, so it stays in the leading prompt where it caches. - An Anthropic wire test covers holding a change past a tool result and the folded fallback; the live test covers Copilot and checks the cache only where caching is explicit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anthropic takes a mid-conversation system message only after a user message, so its wire puts a change after the prompt whatever the stored order, and OpenAI and Codex answer from either order. Declaring after the prompt drops the pending-input pre-check, the second assembly and the system-entry skips in the leaf guard and fork-before. On Anthropic, a system message before the first reply now joins the leading prompt, so the first declaration still caches. The live helper reads the GitHub Copilot login, whose token never expires, and the live test covers Copilot Claude. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gpt-6-luna takes the change in place on the Responses transport; gpt-5.3-codex, without mid-conversation system messages, folds it into the leading prompt. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request declares the initial tools, then every other tool the thread can reach with defer_loading: the deferred catalog, now passed as Context.deferredTools, and every tool the transcript declared. A later system message adds or removes one as a tool reference, so loading a deferred tool keeps the cached prefix; adding one to the top-level list did not. Direct Anthropic only: Copilot rejects tool changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent document gains foundation: set through configure, it replaces the foundation prompt plugins render for that thread only, and null goes back to it. Forks take it as of their cut and it survives a restart, as the thread's model and thinking level do. thread.configure and session.create take it over RPC. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ThreadInfo carries foundation, in the harness and over RPC, so a client reads back what it set. A live test runs a session on a buggy function on Anthropic, OpenAI, Codex and Copilot Claude: main explains it with the default foundation, a reviewer fork and a tester fork each answer in their own role, and main keeps its own after them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The harness now sends prompt sections as transcript system messages instead of one rendered
systemPromptstring, so a section can change between prompts without breaking a cached prefix.plugin/prompt/render.tsrenders{ foundation, sections }. The foundation is untagged; every other section (tools, built-ins,cwd, plugin sections) is stored as its full tagged text.State.Snapshot.systemPromptbecomesprompt.At each turn start,
runner/loop.tsdeclareSystemChangesdiffs the snapshot's sections againstTranscript.getCurrentSections(transcript)and the offered tools against the replayed tool state. When either changed, it publishes oneSystemChangedevent (wasToolsChanged). The first request declares every section; later ones carry only the patch (nullremoves a section).The projector stores the patch in the
systementry next totoolsAdded/toolsRemoved, and the codec replays it, so resume and fork rebuild what the model saw.aikit:
Transcript.getCurrentSections, factored out ofgetCurrentSystemPrompt, so the harness diffs against aikit's own replay rather than a copy of it.aikit: Anthropic (Opus 4.8/5/5.5, Sonnet/Haiku 5.5, Fable/Mythos 5/5.1, direct and through Copilot) and direct OpenAI Responses models now take prompt changes in place through the AI SDK adapters, flagged by modelgen; the baseline catalog is refreshed with the flags.
aikit: Anthropic changes tools in place by reference. The request declares every tool the thread can reach up front, deferred (
defer_loading): the catalog passed asContext.deferredTools, and every tool the transcript declared. Loading one throughtoolsearchkeeps the cached prefix.harness: a thread can have its own prompt foundation in its
agentdocument (configure({ foundation }),nullto clear), reported byThreadInfo.foundation. Forks take it as of the cut, and it survives a restart. RPC:runtime.foundationonthread.configureandsession.create.Design notes
systemPrompt/toolsand keeps everything in transcript system messages. The harness keepsContext.toolsas the declared set, and aikit treats a leading system message as the head only when there are no initial tools. Putting the foundation in a section would leave Codex/Copilot with an empty head and the foundation framed as an update. So the foundation stays inContext.systemPromptand only the tagged sections go into the transcript, the same split the harness already uses for tools (base from config, deltas in the transcript). Trade-off: a foundation change is not persisted and changes the head (no worse than today).systementry goes after the prompt it comes with, once a request is sure to follow. Anthropic takes a mid-conversation system message only after a user message, so its wire would put it there anyway, and OpenAI and Codex answer correctly from either order (checked live both ways).instructions) and removal of a section named like anObjectprototype key (constructor).Tests
test/prompt.changes.test.ts(E2E, real loop, file DB): first declaration, a changed section, removal, an unchanged prompt adding nothing, resume in a new runtime (nothing re-declared), re-addition, and a fork cut before the change (declares onlynull). It checks that each request's in-place rendering is a prefix of the next one. Artifact:__artifacts__/prompt.changes.json.test/prompt.changes.live.test.ts(live: openai-codex and openai gpt-6-luna, anthropic claude-sonnet-5-5, github-copilot claude-sonnet-5.5): the codeword section changes from alpha to bravo between two prompts; the model answers both, the change goes in place with the earlier section still sent, and on Anthropic the second request reads from the cache.aikit/test/transcript.test.ts: the Anthropic wire, in place and folded, and tool references.test/tools.toolsearch.live.test.ts(live, Codex and Anthropic): on Anthropic, the request after a deferred tool loads reads the one before it from the cache.test/prompt.foundation.test.ts(E2E) andtest/prompt.foundation.live.test.ts(live, Anthropic, OpenAI, Codex, Copilot Claude): forked threads each lead with their own foundation and answer in their own role. Artifact:__artifacts__/prompt.changes.live.openai-codex.json.Transcript.getCurrentSystemPrompt(context)) and expect the newsystementry.🤖 Generated with Claude Code