Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 9 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ This file is the canonical source for unreleased changes and published release n
- MCP OAuth: servers sign in with discovery, client registration and PKCE, tokens stored in `~/.codework/mcpauth.json` by server name and URL and refreshed under a lock shared across processes. A server waiting for sign-in reconnects on the next message once its sign-in changes. `auth: { provider }` sends a model provider's credential instead.
- The ephemeral `mcp.server.status` event reports each MCP server state change (`connecting`, `connected`, `disconnected`, `failed`, `authenticate`, `disabled`) to every thread using the server. A tool the MCP SDK rejects is skipped instead of failing its server, and listed as `skipped` on the `connected` status.
- `toolsearch` turns itself on when a thread has deferred tools or an enabled MCP server with `deferred` exposure; a settings entry still turns it off or on. A search waits up to 10 seconds for such servers still connecting.
- A thread can have its own prompt foundation: `configure({ foundation })` records it in the thread's `agent` document, in place of the one prompt plugins render, and `foundation: null` goes back to that. Other threads keep theirs; a fork starts with the foundation of its cut, and the choice survives a restart. Each change leads the thread's next request with the new foundation, a cache miss once. `Session.create` takes `foundation` for the main thread, and `Session.ThreadInfo.foundation` reports a thread's choice.
- The deferred tools of the catalog reach every request as `deferredTools`, so Anthropic declares them up front and loading one through `toolsearch` keeps the cached prefix.
- Prompt sections reach the model as transcript system messages. A request sends the foundation as its system prompt; the first request of a thread appends a `system` entry declaring every tagged section (`<tools>`, `<rules>`, `<cwd>`, plugin sections), and a later request appends one only for the sections that changed, a removed section as `null`, together with any tool change. Resume and fork replay these entries, so the model sees what it saw before, and a model that takes system messages in place (OpenAI Codex, OpenAI and GitHub Copilot Responses, Anthropic and GitHub Copilot Claude) keeps its cached prefix when a section changes.

#### Breaking Changes

Expand All @@ -50,6 +53,7 @@ This file is the canonical source for unreleased changes and published release n
- Migrations add `session_input.cancelled_seq`, and `session.execution` and `session.restarts`.
- Stored context messages are user and assistant messages only; a stored system message fails to decode with "system messages are not stored as entries".
- Tests use `gpt-6-luna` in place of the deprecated `gpt-5.5`.
- `session.thread.tools.changed` (`ToolsChanged`) is now `session.thread.system.changed` (`SystemChanged`), and its `system` entry data has an optional `sections` patch. `State.Snapshot.systemPrompt` is now `prompt: { foundation, sections }`.

### @codeworksh/cli

Expand All @@ -60,6 +64,7 @@ This file is the canonical source for unreleased changes and published release n
- Added the `thread.update` RPC (`{ sessionId, threadId?, messageId, delivery?, text? }`), which moves a queued prompt to the other lane or rewrites its text, keeping its place. `delivery: "steer"` also wakes the thread. `updated` is `false` when the prompt was already taken in.
- `codework serve` resumes the sessions a crash or a shutdown cut off when it starts. Thread info carries `parked: true` for a thread that restarted too often to resume on its own; its next prompt runs it again.

- `thread.configure` and `session.create` take `runtime.foundation`, a thread's own prompt foundation; `null` on `thread.configure` goes back to the rendered one. `ThreadInfo.foundation` reports it.
- `codework run --thread <id|main>` continues one thread of `--session`.
- `codework thread list <session>` lists a session's threads with their model, usage and title. `codework thread fork <session> --at <entry>` or `--before <entry>` starts a new thread and prints its id, then for `--before` the text of the prompt it left out. `codework session relink <session> --thread <id|main>` moves one thread to another checkout. All three take `--json`.
- `ThreadInfo.usage` reports a thread's own cost and tokens. `event.subscribe` and `session.log` take `threadId` to keep only that thread's events.
Expand Down Expand Up @@ -89,8 +94,10 @@ This file is the canonical source for unreleased changes and published release n

- Transcript system messages change the tools mid-conversation: `Message.createSystemMessage({ toolsAdded?, toolsRemoved?, metadata? })`. `Context.tools` is the initial set. OpenAI Codex and GitHub Copilot Responses models that support it declare every tool in place, in `additional_tools` items, and announce removals as text, so the cached prompt prefix holds across tool changes; models with tool search only get later additions as a tool search result; every other model gets the current tools top-level.
- `Transcript` helpers: `getCurrentTools`, `getDeclaredTools`, `getToolStateChanges`, `declarationsEqual`, `toToolDeclaration` and `resolveTranscriptTools`.
- Transcript system messages also change the system prompt: `content` adds instructions from that point on, and `sections` replace named sections, `null` removing one. `Context.systemPrompt` is the initial prompt. OpenAI Codex and GitHub Copilot Responses models with `supportsMidConvoSystemMessages` keep the leading prompt and get each change in place as a developer message (`Updated system prompt section "<name>": …`), so the cached prompt prefix holds; every other model gets the current prompt as its system prompt. Helpers: `getCurrentSystemPrompt`, `renderSystemPrompt`, `renderSystemPromptUpdate` and `resolveTranscriptPrompt`.
- `supportsMidConvoSystemMessages` compat flag, set by `modelgen` for the OpenAI Codex and GitHub Copilot Responses models that accept developer messages mid-conversation. In-place tool declarations (`supportsAdditionalTools`) require it.
- Transcript system messages also change the system prompt: `content` adds instructions from that point on, and `sections` replace named sections, `null` removing one. `Context.systemPrompt` is the initial prompt. Models with `supportsMidConvoSystemMessages` keep the leading prompt and get each change in place as a system or developer message (`Updated system prompt section "<name>": …`), so the cached prompt prefix holds; every other model gets the current prompt as its system prompt. On Anthropic a change is sent before the next assistant message or at the end, since the Messages API takes a system message only after a user message and never between a tool call and its result; one before the first reply joins the top-level system prompt, where it caches. Helpers: `getCurrentSystemPrompt`, `renderSystemPrompt`, `renderSystemPromptUpdate` and `resolveTranscriptPrompt`.
- `supportsMidConvoSystemMessages` compat flag, set by `modelgen` for the models that accept system messages mid-conversation: OpenAI Codex, OpenAI and GitHub Copilot Responses (gpt-5.4 and later), and Anthropic and GitHub Copilot Claude (claude-opus-4.8, -5 and -5.5, claude-sonnet-5.5, claude-haiku-5.5, claude-fable-5 and -5.1, claude-mythos-5 and -5.1). The baseline catalog carries it. In-place tool declarations (`supportsAdditionalTools`) require it.
- Anthropic models take tool changes in place (`supportsAdditionalTools`, set for direct Anthropic only, as Copilot rejects them): the request declares its initial tools, then every other tool the thread can reach deferred (`defer_loading`), the catalog passed as the new `Context.deferredTools` and every tool the transcript declared, and a later system message adds or removes one by reference. Loading a deferred tool then keeps the cached prompt prefix.
- `Transcript.getCurrentSections`: the sections a transcript declares after replaying its system messages. `Transcript.diffSections(previous, current)`: the patch between two section sets, `null` removing one.
- `filterGitHubCopilotModels` keeps the catalog models a Copilot login can reach.
- GitHub Copilot GPT models turn reasoning off with `reasoning.effort: "none"` where OpenAI accepts it, such as gpt-5.3-codex, gpt-5.6-luna and gpt-6-luna.

Expand Down
13 changes: 13 additions & 0 deletions packages/aikit/src/cli/modelgen.ts
Original file line number Diff line number Diff line change
Expand Up @@ -353,6 +353,9 @@ function applyOpenAiMetadata(model: Model.Info, source: ModelsDevModel | undefin
}
// models.dev reports the input sub-limit as gpt-5-pro's output.
if (id === "gpt-5-pro") model.maxTokens = OPENAI_MAX_TOKENS;
if (model.api?.method === Model.APIMethodEnum.responses && OPENAI_MID_CONVO_SYSTEM_MESSAGE_MODEL_IDS.has(id)) {
mergeCompat(model, { supportsMidConvoSystemMessages: true });
}
}
model.cost.serviceTierMultipliers = serviceTierMultipliers(model, source);

Expand Down Expand Up @@ -391,6 +394,9 @@ function isAnthropicMessages(model: Model.Info): boolean {

// The Messages API rejects `temperature` on these.
const ANTHROPIC_NO_TEMPERATURE = /opus-4[-.][78]|opus-5|(sonnet|haiku)-5[-.]5/;
// These take system messages inside the conversation, verified live against the Messages API.
const ANTHROPIC_MID_CONVO_SYSTEM_MESSAGES =
/^claude-(?:opus-(?:4[.-]8|5(?:[.-]5)?)|(?:sonnet|haiku)-5[.-]5|(?:fable|mythos)-5(?:[.-]1)?)(?:-\d{8})?$/;

// Checked live against the Messages API: effort levels each adaptive family
// accepts, the 5.5 and Fable models that reject `thinking: {type: "disabled"}`, and
Expand Down Expand Up @@ -422,6 +428,10 @@ function applyAnthropicMetadata(model: Model.Info): void {
mergeThinkingLevelMap(model, { off: null });
}
if (ANTHROPIC_NO_TEMPERATURE.test(id)) mergeCompat(model, { supportsTemperature: false });
// Copilot passes these system messages through but rejects tool changes in them.
if (model.provider.id === "anthropic" && ANTHROPIC_MID_CONVO_SYSTEM_MESSAGES.test(id)) {
mergeCompat(model, { supportsMidConvoSystemMessages: true, supportsAdditionalTools: true });
}
}

//
Expand Down Expand Up @@ -701,6 +711,9 @@ export function githubCopilotBuiltInModels(provider: ModelsDevProvider | undefin
if (method === Model.APIMethodEnum.responses && OPENAI_MID_CONVO_SYSTEM_MESSAGE_MODEL_IDS.has(id)) {
mergeCompat(info, { supportsMidConvoSystemMessages: true });
}
if (method === Model.APIMethodEnum.messages && ANTHROPIC_MID_CONVO_SYSTEM_MESSAGES.test(id)) {
mergeCompat(info, { supportsMidConvoSystemMessages: true });
}
if (method === Model.APIMethodEnum.responses && GITHUB_COPILOT_ADDITIONAL_TOOLS_MODEL_IDS.has(id)) {
mergeCompat(info, { supportsAdditionalTools: true });
}
Expand Down
9 changes: 6 additions & 3 deletions packages/aikit/src/llm/stream.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
import { APICallError } from "@ai-sdk/provider";
import { RetryError, streamText, type TextStreamPart, type ToolSet } from "ai";
import * as Message from "../message/message.ts";
import * as Transcript from "../message/transcript.ts";
import * as Model from "../model/model.ts";
import { AssistantMessageEventStream } from "../utils/eventstream.ts";
import { compact } from "../utils/helpers.ts";
Expand All @@ -18,6 +17,7 @@ import {
convertMessages,
isOwnResponses,
convertTools,
requestTools,
createAssistantMessage,
encodeOpenAIReasoningSignature,
grammarToolInputProperties,
Expand Down Expand Up @@ -416,7 +416,8 @@ export const stream: Protocol.StreamFunction<Model.KnownProviderEnum, typeof Opt
const plan = Thinking.resolvePlan(model, context, runtimeOptions);
const languageModel = await resolveAISDKLanguageModel(model, runtimeOptions, plan.budget);
// Every tool callable now, so calls to tools declared in place still parse.
const tools = convertTools(Transcript.getCurrentTools(context), model, runtimeOptions.method);
const declared = requestTools(context, model, runtimeOptions.method, runtimeOptions.activeTools);
const tools = convertTools(declared.tools, model, runtimeOptions.method, declared.deferred);
const messages = convertMessages(context, model, runtimeOptions.method, runtimeOptions.activeTools);
const providerOptions = resolveProviderOptions(model, context, runtimeOptions, plan) as Parameters<
typeof streamText<ToolSet>
Expand All @@ -433,7 +434,9 @@ export const stream: Protocol.StreamFunction<Model.KnownProviderEnum, typeof Opt
let params: Parameters<typeof streamText<ToolSet>>[0] = compact({
model: languageModel,
// Our own Responses transport receives the system prompt among the messages.
system: isOwnResponses(model, runtimeOptions.method) ? undefined : leadingSystemPrompt(context),
system: isOwnResponses(model, runtimeOptions.method)
? undefined
: leadingSystemPrompt(context, model, runtimeOptions.method),
messages,
// Prompt and tool changes sent in place travel as system messages.
allowSystemInMessages: messages.some((message) => message.role === "system") || undefined,
Expand Down
121 changes: 111 additions & 10 deletions packages/aikit/src/llm/transform.ts
Original file line number Diff line number Diff line change
Expand Up @@ -485,16 +485,85 @@ export function transcriptToolSupport(
}

/**
* Whether prompt changes go in place as developer messages. Only our own Responses transport
* sends them; the AI SDK adapters take the current prompt as the leading system prompt.
* Whether prompt changes go in place as system messages. Our own Responses transport and the
* AI SDK's OpenAI Responses and Anthropic Messages adapters keep them where they stand; Chat
* Completions takes the current prompt as the leading system prompt.
*/
export function supportsMidConvoSystemMessages(model: Model.Info, method: Model.APIMethodEnum | undefined): boolean {
return isOwnResponses(model, method) && model.compat?.supportsMidConvoSystemMessages === true;
if (model.compat?.supportsMidConvoSystemMessages !== true) return false;
return isOwnResponses(model, method) || !isChatCompletions(model, resolveMethod(model, method));
}

/** The prompt the AI SDK adapters take as `system`: the current one, every change folded in. */
export function leadingSystemPrompt(context: Message.Context): string | undefined {
return Transcript.getCurrentSystemPrompt(context) || undefined;
/**
* Anthropic with tool changes in place: every tool the thread can reach is declared up front, the
* ones beyond its initial set deferred, and later changes name them by reference. Adding a tool
* to the top-level list costs the cached prefix; referencing a deferred one does not.
*/
function anthropicToolReferences(model: Model.Info, method: Model.APIMethodEnum, restricted: boolean): boolean {
return (
!restricted &&
isAnthropicWire(model, method) &&
supportsMidConvoSystemMessages(model, method) &&
model.compat?.supportsAdditionalTools === true
);
}

/** Messages before the first reply: on Anthropic their system messages join the leading prompt and tools. */
function beforeFirstReply(messages: readonly Message.Message[]): Message.Message[] {
const reply = messages.findIndex((message) => message.role === "assistant");
return reply === -1 ? [...messages] : messages.slice(0, reply);
}

/**
* The tools the request declares top-level, and which of them are deferred. Usually every tool
* callable now. With Anthropic tool references, the initial set, then the deferred catalog and
* every other tool the transcript declared, deferred, in an order that holds across requests.
*/
export function requestTools(
context: Message.Context,
model: Model.Info,
methodOverride?: Model.APIMethodEnum,
activeTools?: readonly string[],
): { tools: Message.Tool[]; deferred: ReadonlySet<string> } {
if (!anthropicToolReferences(model, resolveMethod(model, methodOverride), activeTools !== undefined)) {
return { tools: Transcript.getCurrentTools(context), deferred: new Set() };
}
const initial = Transcript.getCurrentTools({ ...context, messages: beforeFirstReply(context.messages) });
const latest = new Map(Transcript.getDeclaredTools(context).map((tool) => [tool.name, tool]));
const tools = new Map(initial.map((tool) => [tool.name, tool]));
const deferred = new Set<string>();
for (const tool of [...(context.deferredTools ?? []), ...latest.values()]) {
if (tools.has(tool.name)) continue;
tools.set(tool.name, latest.get(tool.name) ?? tool);
deferred.add(tool.name);
}
return { tools: [...tools.values()], deferred };
}

/** A later system message's tool changes as Anthropic tool references; a redefinition needs no removal. */
function toolReferences(message: Message.SystemMessage) {
const added = new Set((message.toolsAdded ?? []).map((tool) => tool.name));
return [
...(message.toolsRemoved ?? [])
.filter((tool) => !added.has(tool.name))
.map((tool) => ({ type: "tool_removal" as const, toolName: tool.name })),
...[...added].map((name) => ({ type: "tool_addition" as const, toolName: name })),
];
}

/**
* The prompt the AI SDK adapters take as `system`: the leading one when changes go in place,
* else the current one, every change folded in.
*/
export function leadingSystemPrompt(
context: Message.Context,
model: Model.Info,
method: Model.APIMethodEnum | undefined,
): string | undefined {
return (
Transcript.resolveTranscriptPrompt(context, supportsMidConvoSystemMessages(model, method)).systemPrompt ||
undefined
);
}

/** A tool as the language model receives it, for declaring it inside the prompt. */
Expand Down Expand Up @@ -557,11 +626,32 @@ export function convertMessages(
messages.push({ role: "system", content: sanitizeSurrogates(prompt.systemPrompt) });
}

// Every system message goes as an update, the first declaration too: a section a model read
// whole does not yield to a later update of it, one it read as an update does. On Anthropic,
// one before the first reply joins the leading prompt, where it caches: a trailing system
// message's cache entry is never read once a reply follows it. Anthropic takes a later one
// only after a user message, and never between a tool call and its result: each is held for
// the next assistant message, or the end.
const anthropic = isAnthropicWire(model, method);
const references = anthropicToolReferences(model, method, activeTools !== undefined);
const leading: ModelMessage[] = [];
const held: ModelMessage[] = [];
let replied = false;
let systemIndex = 0;
for (const msg of transformedMessages) {
if (msg.role === "system") {
const index = systemIndex++;
const update = index < prompt.foldedSystemMessages ? "" : Transcript.renderSystemPromptUpdate(msg);
// The tools of one before the first reply belong to the initial set.
const toolChanges = references && replied ? toolReferences(msg) : [];
if (toolChanges.length > 0) {
held.push({
role: "system",
content: sanitizeSurrogates(update),
providerOptions: { [Model.optionsKey(model)]: { toolChanges } },
});
continue;
}
const folded = index < transcript.foldedSystemMessages;
const change = systemChangeMessage(
update,
Expand All @@ -571,17 +661,22 @@ export function convertMessages(
model,
method,
);
if (change) messages.push(change);
if (change) (!anthropic ? messages : replied ? held : leading).push(change);
continue;
}
if (msg.role === "user") {
messages.push(...userContent(msg.parts));
continue;
}
messages.push(...assistantMessages(msg, model, method));
const converted = assistantMessages(msg, model, method);
if (converted.length > 0) {
replied = true;
messages.push(...held.splice(0));
}
messages.push(...converted);
}

return messages;
return [...leading, ...messages, ...held];
}

/**
Expand Down Expand Up @@ -619,17 +714,23 @@ export function convertTools(
tools?: Message.Tool[],
model?: Model.Info,
method?: Model.APIMethodEnum,
deferred: ReadonlySet<string> = new Set(),
): ToolSet | undefined {
if (!tools || tools.length === 0) return;

const result: ToolSet = {};
for (const tool of tools) {
const { inputSchema, strict, providerOptions } = toolWire(tool, model, method);
const key = model && deferred.has(tool.name) ? Model.optionsKey(model) : undefined;
const options =
key === undefined
? providerOptions
: { ...providerOptions, [key]: { ...providerOptions?.[key], deferLoading: true } };
result[tool.name] = {
description: tool.description,
inputSchema: jsonSchema(inputSchema as any),
...(strict ? { strict } : {}),
...(providerOptions ? { providerOptions } : {}),
...(options ? { providerOptions: options } : {}),
};
}
return result;
Expand Down
Loading
Loading