Skip to content

feat: prompt sections and per-thread foundations as transcript state, sent in place across providers - #116

Merged
sanchitrk merged 11 commits into
canaryfrom
claude/lucid-poincare-b6f8bf
Oct 31, 2026
Merged

sanchitrk merged 11 commits into
canaryfrom
claude/lucid-poincare-b6f8bf

Conversation

@sanchitrk

@sanchitrk sanchitrk commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

The harness now sends prompt sections as transcript system messages instead of one rendered systemPrompt string, so a section can change between prompts without breaking a cached prefix.

  • plugin/prompt/render.ts renders { foundation, sections }. The foundation is untagged; every other section (tools, built-ins, cwd, plugin sections) is stored as its full tagged text. State.Snapshot.systemPrompt becomes prompt.

  • At each turn start, runner/loop.ts declareSystemChanges diffs the snapshot's sections against Transcript.getCurrentSections(transcript) and the offered tools against the replayed tool state. When either changed, it publishes one SystemChanged event (was ToolsChanged). The first request declares every section; later ones carry only the patch (null removes a section).

  • The projector stores the patch in the system entry next to toolsAdded/toolsRemoved, and the codec replays it, so resume and fork rebuild what the model saw.

  • aikit: Transcript.getCurrentSections, factored out of getCurrentSystemPrompt, so the harness diffs against aikit's own replay rather than a copy of it.

  • aikit: Anthropic (Opus 4.8/5/5.5, Sonnet/Haiku 5.5, Fable/Mythos 5/5.1, direct and through Copilot) and direct OpenAI Responses models now take prompt changes in place through the AI SDK adapters, flagged by modelgen; the baseline catalog is refreshed with the flags.

  • aikit: Anthropic changes tools in place by reference. The request declares every tool the thread can reach up front, deferred (defer_loading): the catalog passed as Context.deferredTools, and every tool the transcript declared. Loading one through toolsearch keeps the cached prefix.

  • harness: a thread can have its own prompt foundation in its agent document (configure({ foundation }), null to clear), reported by ThreadInfo.foundation. Forks take it as of the cut, and it survives a restart. RPC: runtime.foundation on thread.configure and session.create.

Design notes

  • Where the foundation lives. Pi passes no systemPrompt/tools and keeps everything in transcript system messages. The harness keeps Context.tools as the declared set, and aikit treats a leading system message as the head only when there are no initial tools. Putting the foundation in a section would leave Codex/Copilot with an empty head and the foundation framed as an update. So the foundation stays in Context.systemPrompt and only the tagged sections go into the transcript, the same split the harness already uses for tools (base from config, deltas in the transcript). Trade-off: a foundation change is not persisted and changes the head (no worse than today).
  • Placement. The system entry goes after the prompt it comes with, once a request is sure to follow. Anthropic takes a mid-conversation system message only after a user message, so its wire would put it there anyway, and OpenAI and Codex answer correctly from either order (checked live both ways).
  • Anthropic placement and caching. Anthropic rejects a mid-conversation system message before a user message, and a cache entry ending on a trailing system message is never read once a reply follows it (both verified against the raw API). So a later change is held until the next assistant message or the end, and a declaration before the first reply joins the leading prompt, where it caches; the live test shows the second request reading the first one's sections from the cache.
  • Every declaration framed as an update. Live, gpt-6-luna kept a section it first read whole even after an update of it, but follows the latest of two updates; so the first declaration is framed as an update too.
  • Review (Codex gpt-6.1-sol, high): fixed a blank foundation (now falls back to the default, so Codex's transport never promotes a later section update into instructions) and removal of a section named like an Object prototype key (constructor).
  • Whole-prompt override (the open proposal on the issue) is not built. Nothing here blocks it: an override would replace only the outgoing request's prompt, and the transcript keeps recording sections.

Tests

  • test/prompt.changes.test.ts (E2E, real loop, file DB): first declaration, a changed section, removal, an unchanged prompt adding nothing, resume in a new runtime (nothing re-declared), re-addition, and a fork cut before the change (declares only null). It checks that each request's in-place rendering is a prefix of the next one. Artifact: __artifacts__/prompt.changes.json.
  • test/prompt.changes.live.test.ts (live: openai-codex and openai gpt-6-luna, anthropic claude-sonnet-5-5, github-copilot claude-sonnet-5.5): the codeword section changes from alpha to bravo between two prompts; the model answers both, the change goes in place with the earlier section still sent, and on Anthropic the second request reads from the cache.
  • aikit/test/transcript.test.ts: the Anthropic wire, in place and folded, and tool references.
  • test/tools.toolsearch.live.test.ts (live, Codex and Anthropic): on Anthropic, the request after a deferred tool loads reads the one before it from the cache.
  • test/prompt.foundation.test.ts (E2E) and test/prompt.foundation.live.test.ts (live, Anthropic, OpenAI, Codex, Copilot Claude): forked threads each lead with their own foundation and answer in their own role. Artifact: __artifacts__/prompt.changes.live.openai-codex.json.
  • Existing tests now read the replayed prompt (Transcript.getCurrentSystemPrompt(context)) and expect the new system entry.
  • Pi tests not ported as isolated tests: the section diff and first declaration are covered by the E2E; "preamble untagged and replaced like any section" does not apply (our foundation is not a section); the forced-prompt case belongs to the undecided override.

🤖 Generated with Claude Code

sanchitrk and others added 11 commits October 30, 2026 22:50
The foundation goes out as the system prompt; every tagged section is
declared by a system entry. The first request declares them all, a later
one only the sections that changed, with any tool change, so resume and
fork replay what the model saw and in-place models keep their cache.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The system entry now lands before the prompt it comes with, so the model
reads the setup, then the request. The finished-reply guard and fork-before
skip system entries, so a declaration left by a cancelled prompt is inert.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ponses

Anthropic (Opus 4.8/5/5.5, Sonnet/Haiku 5.5, Fable/Mythos 5/5.1) and direct
OpenAI Responses models now take later system messages in place through the
AI SDK adapters, so a prompt section change keeps the cached prefix.
On Anthropic a later system message is held for the next assistant message,
as the API takes one only after a user message; one ahead of the
conversation joins the leading prompt, since a trailing system message's
cache entry is never read once a reply follows it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Transcript.diffSections replaces the harness copy of the section diff.
- Copilot Claude models take system messages in place, as direct Anthropic.
- An Anthropic system message ahead of the conversation is never held, so it
  stays in the leading prompt where it caches.
- An Anthropic wire test covers holding a change past a tool result and the
  folded fallback; the live test covers Copilot and checks the cache only
  where caching is explicit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anthropic takes a mid-conversation system message only after a user
message, so its wire puts a change after the prompt whatever the stored
order, and OpenAI and Codex answer from either order. Declaring after the
prompt drops the pending-input pre-check, the second assembly and the
system-entry skips in the leaf guard and fork-before.

On Anthropic, a system message before the first reply now joins the
leading prompt, so the first declaration still caches. The live helper
reads the GitHub Copilot login, whose token never expires, and the live
test covers Copilot Claude.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
gpt-6-luna takes the change in place on the Responses transport;
gpt-5.3-codex, without mid-conversation system messages, folds it into the
leading prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The request declares the initial tools, then every other tool the thread
can reach with defer_loading: the deferred catalog, now passed as
Context.deferredTools, and every tool the transcript declared. A later
system message adds or removes one as a tool reference, so loading a
deferred tool keeps the cached prefix; adding one to the top-level list
did not. Direct Anthropic only: Copilot rejects tool changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent document gains foundation: set through configure, it replaces
the foundation prompt plugins render for that thread only, and null goes
back to it. Forks take it as of their cut and it survives a restart, as
the thread's model and thinking level do. thread.configure and
session.create take it over RPC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ThreadInfo carries foundation, in the harness and over RPC, so a client
reads back what it set. A live test runs a session on a buggy function
on Anthropic, OpenAI, Codex and Copilot Claude: main explains it with the
default foundation, a reviewer fork and a tester fork each answer in
their own role, and main keeps its own after them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sanchitrk sanchitrk changed the title feat(harness): declare prompt sections as transcript system messages feat: prompt sections and per-thread foundations as transcript state, sent in place across providers Oct 31, 2026
@sanchitrk
sanchitrk merged commit 9e012a1 into canary Oct 31, 2026
2 checks passed
@sanchitrk
sanchitrk deleted the claude/lucid-poincare-b6f8bf branch October 31, 2026 07:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant