Repository navigation
workbuddy 0.1.32: serve Claude models on Messages, where reasoning effort (incl. max) is honored - #346
workbuddy 0.1.32: serve Claude models on Messages, where reasoning effort (incl. max) is honored#346waynesguan wants to merge 3 commits into
Conversation
yetone
left a comment
There was a problem hiding this comment.
Thanks for this, @waynesguan. The measurements are useful, and the routing through magpie is right: pluginFetch takes the model's own api.url once the loader's baseURL is empty, so /v1/messages lands on <endpoint>/v1/messages and chat models still reach /v2. I reviewed 9792707 line by line and ran it. Before it can merge, three things need fixing and one test is missing.
What I ran
HOME=$(mktemp -d) bun testinpackages/workbuddy: 130 pass, 0 fail.bun scripts/check.mjs workbuddy: ✓.- With real accounts on both builds, the PR's plugin called directly (its loader's
fetch, sandbox HOME):- China build (
copilot.tencent.com):/v1/messagesexists.glm-5.3anddeepseek-v4.1-flashanswer 500 there, as you found. Our account ispersonal_free_user, so every Claude model answers 11158 and I couldn't get a Claude reply. One thing does support your point that Messages has no channel check. On chat, a Claude model with Claude Code's "official CLI" line, its billing line or DeepSeek Harness's opening gets 11128, so the channel check runs before entitlement there. On/v1/messagesthe same bodies get past it to 11158. - International build (
www.workbuddy.ai):/v1/messagesanswers404 {"error_msg":"404 Route Not Found"}, for Claude andglm-5.3alike, and so do/v2/messagesand/v1/messages?beta=true.
- China build (
- Sandbox magpie (main a93190ae1) with this plugin installed: a Claude-Code-shaped Anthropic request (billing line, "official CLI" line, "Main branch (you will usually use this for PRs)", DeepSeek Harness's opening,
output_config.effort: max) toworkbuddy/glm-5.3andworkbuddy-ai/glm-5.3gets 200 "pong" on both builds. So the 0.1.29–0.1.31 rewording still works on the chat route.
1. The international build doesn't serve Messages (index.mjs:125, native.test.mjs:25)
nativeOf(site, model) ignores site, and the test says the international build "serves them the same way". It doesn't: its /v1/messages is a 404 route. Today its cli list has no Claude model (claude-opus-5.5 on its chat answers "service info not found"), so nothing breaks yet. The day it lists one, though, every request for it goes to a 404, where on 0.1.31 it would have gone to chat. Please scope the rule to the site that was measured, e.g. a messages: true on SITES.workbuddy only, have nativeOf check it, and turn the test around (workbuddy-ai → false). effortsOf (index.mjs:1716) repeats the /^claude-/ test on its own. It should ask nativeOf(site, id), so max is added only where Messages actually carries it.
2. On Messages, each step of one send is counted as a new request again (#48) (index.mjs:1888, with turnKey/answeredBefore at :396/:409)
The PR says the conversation/turn ids are shared by both routes. The headers are set, but turnKey only knows chat shapes: a tool result is a role: "tool" message, and an assistant's calls are tool_calls. In an Anthropic body the tool result is a user message of tool_result blocks after an assistant message of tool_use blocks. So answeredBefore is false, and turnKey either returns "" (a fresh random id) or keys on Claude Code's <system-reminder> text. I ran attend on a three-step Claude Code tool loop (prompt → Read → Edit) under one session, and got three different X-Conversation-Request-IDs. The chat version of the same loop keeps one. That is the bug 5d1beb3 fixed: WorkBuddy counts a user send as one request by this id. answeredBefore should also treat a user message whose content is tool_result blocks (after an assistant message with tool_use) as answering, and its text parts should be left out of the turn's signature, as a tool's images are. Please add a test with a Messages-shaped tool loop.
3. configModels' api field is never read (index.mjs:1598, native.test.mjs:105-107)
OpenCode reads a config model's package and URL from provider: { npm, api }, not api: { url, npm }. magpie's host does the same (internal/plugin/host.js info(): npm: m.provider?.npm ?? cfg.npm …, url: m.provider?.api ?? cfg.api …). So a Claude model in the fallback list, used when /v3/config can't be read, would still go as chat to /v2. The test only checks the object the plugin builds, not what a host makes of it. The fix is provider: { npm: NPM_MESSAGES, api: apiMessages(site) }. Today's static list has no Claude model, so this is latent.
4. Nothing tests the loader's fetch on a Messages URL
native.test.mjs checks isMessages, nativeOf, modelOf and withSystem. No test sends a request through the loader to …/v1/messages. Please add one (a stubbed fetch, as unauthorized.test.mjs does) that checks:
- the body arrives byte-for-byte as sent (no default system message, no
max_tokensrespelling, no tool renaming); - a 401 is renewed once and sent again on this route too;
- the conversation and turn headers are set (with the fix in 2).
Smaller points, not blockers:
README.md:103says magpie estimates the count "as it does for the other plugins in itsestimatedMovedlist".workbuddyisn't in that list. The estimate comes fromunsupportedCounttreating the 404 as "no counter", so please say that.- A Messages error comes back as
{"code":11102,"error":"11158:…"}, soexplained()readscodeas 11102. 11140 (restricted) and 14018 (credits used up) are then recognised only if their words appear in the text. Did you see either one on/v1/messages? If so, a fixture of the real body in a test would settle it.
Once 1–4 are in, I'll run it again and merge. Thanks again: getting max through to the server is worth having.
…is honored WorkBuddy serves the Claude models on Anthropic's Messages API at <endpoint>/v1/messages, beside the chat completions the rest use. Asked on 2026-10-24, each model of /v3/config's cli agent: claude-opus-5.5, claude-sonnet-5.5, claude-opus-5 and claude-opus-4.8-1m answered in Anthropic's stream; glm-5.3 and deepseek-v4.1-flash answered 500, gpt-5.5 and kimi-k3-1 400. The route is what makes reasoning effort mean anything. A chat carries reasoning_effort alone, and WorkBuddy's chat endpoint answers at about the same length whatever it says: the same question four times each gave a median of 3884 output tokens at low, 5144 at max, and 5715 at a value that is not a level at all. Messages carries output_config.effort, and max is honored: a median of 30850 against 5748 at low, same model and question. So a Claude model lists max in its levels even where /v3/config stops at xhigh, and a client that picks max reaches the server as max. A Messages request also goes as the agent sent it: no default system message, no max_completion_tokens/developer rewrites, no tool reshaping, and nothing moved for a refused system prompt. Claude Code's own prompt is taken as it is — billing line, "official CLI for Claude" line and all — where a chat answers 11128 to it. Its tool loop, thinking blocks with a signature to send back, and prompt cache all work there: a Claude Code 2.1.307 session fixed a file and solved a puzzle through the route, with every level sent as max and thinking blocks signed. The loader gives no baseURL any more: a model's own api.url places it, on /v2 for a chat and /v1 for Messages. A provider-wide base put /messages under /v2. /v1/messages/count_tokens is not served (404); magpie estimates a count for these models, as it does for the other plugins it estimates for.
…n id per send, unsigned thinking left out
- nativeOf asks site.messages, set on WorkBuddy (China) alone: WorkBuddy AI's
/v1/messages is a 404 route, so a Claude model it lists goes to chat as on
0.1.31. effortsOf adds max only where nativeOf says Messages carries it.
- answeredBefore knows Anthropic's shape: a user message of tool_result
blocks answers the tool_use before it (whatever text sits beside the
results), and so does a message right after it; Claude Code's
{role: "system"} messages between steps are passed over. One Claude Code
send keeps one X-Conversation-Request-ID again (magpie-community#48).
- A Messages body goes without thinking blocks that have no signature.
A session that began on chat carries them; once this route's signed
blocks sit beside them WorkBuddy refuses every request, 11159 "bound to a
different conversation" (drop_block doesn't help). Seen in two live
Claude Code sessions moved mid-way; their captured bodies answer 200
with the unsigned blocks left out.
- configModels places a Claude model with provider: { npm, api }, as
OpenCode and magpie's host read a config model.
- explained reads the code inside the Messages envelopes
({"error": {"internal_code": …}}, {"code": 11102, "error": "11158:…"}),
so 11140 and 14018 are told there too.
- Tests for each, the loader's fetch on /v1/messages included (byte for
byte, 401 renewed once, turn ids through the loader). README: WorkBuddy AI
stays on chat, the unsigned-thinking rule, count_tokens' estimate comes
from unsupportedCount.
9792707 to
9bab215
Compare
|
Thanks for the careful review. All four points are fixed in 9bab215 (rebased on main), along with both smaller ones. Your item 2 led me to a related bug: a session that moves onto this route mid-way can fail on every request. That's fixed here too. Details below. 1. WorkBuddy AI stays on chat
2. One send gets one turn id on Messages (#48)
I checked this against real traffic, not only fixtures. I captured the bodies magpie passed to the plugin's
3. Config models use
|
|
Thanks @waynesguan, this covers all four points and both smaller ones. I reviewed 9bab215 line by line and ran it. What I ran
One thing to settle before merge: the empty text block
|
|
Thanks again, @waynesguan. I reviewed 9bab215 a second time, line by line, and ran it. Everything from the first round holds. One point is still open, and it blocks the merge. Two smaller ones can wait. What I ran at 9bab215
1. Blocker: the empty text block is still unconfirmed (
2. Smaller: 3. Question, not a blocker: signed thinking across accounts and restarts Once 1 is answered, and 2 fixed if it's easy, I'll run it again and merge. |
…on the Messages route, a signature we didn't make retried without thinking
- The empty text block withoutUnsigned leaves is confirmed against
WorkBuddy, not only reasoned about: with a real account, the unsigned
block plus a signed turn after it is 11159 (3 runs), the same with
[{type:"text",text:""}] in its place is 200 (3 runs, twice over).
WorkBuddy takes what Anthropic's own API refuses.
- What binds a signature is now measured and written down: not the
conversation id, not the account, not the model, not where in the
history. A block keeping an unrelated turn's signature, another model's
(sonnet-5.5's on opus-5.5), or another account's answers 200; a truncated
or altered one is 11159; a message that is thinking alone is refused
whether its thinking is signed or not.
- A signature WorkBuddy didn't make (a session moved here from another
provider, or an altered one) is 11159 too, and drop_block doesn't help.
Such a request now goes once more without any thinking block, only when
the refusal says so and only when there was one to drop.
- isMessages counts /v1/messages/count_tokens as a Messages request, so it
no longer takes the chat route's rewrites.
|
All three settled in 39f71a7. 1. Blocker: the empty text block — WorkBuddy answers 200Replayed against WorkBuddy's
So WorkBuddy takes the empty text block, where Anthropic's own API refuses it. The comment above 2.
|
| change to a turn that answers 200 | answer |
|---|---|
new X-Conversation-ID, no id, new request ids |
200 |
| first user message reworded, an extra user turn before it | 200 |
| system prompt changed, effort changed | 200 |
| the assistant's own text before the block changed | 200 |
| an assistant turn inserted before it | 200 |
| a block keeping another turn's signature | 200 |
| a block keeping sonnet-5.5's signature, sent to claude-opus-5.5 | 200 |
| signature truncated / one char changed / random base64 | 11159 |
| an assistant message that is thinking alone | 11159 |
So a magpie restart (new SALT, new conversation id) is fine, and so is a second account. The README now says this. The one thing that does break it: a message that is thinking alone, which is why withoutUnsigned empties it rather than leaving it.
One more failure I found, and fixed
That table exposed a gap of mine. A session moved onto this route from another provider keeps that provider's signed thinking blocks, and they are 11159 too — a real signature, just not one WorkBuddy made. block_binding.prefix_mismatch_behavior: "drop_block" doesn't help. That would have refused every request of such a session, which is the same failure this change exists to end.
So a Messages request now goes once more without any thinking block when the refusal names a bad signature, and only when there was one to drop. A session on this route never sends twice (no such refusal comes back), and a refusal that isn't about a signature isn't retried. Three tests: the retry drops the thinking and only that; a body with no thinking isn't sent twice; an entitlement refusal passes straight through.
Tests: 149 pass, 0 fail, check.mjs workbuddy ✓. Each new guard is pinned by mutation: no retry → 1 red, count_tokens back to chat → 1 red, no empty-text fallback → 1 red.
What
WorkBuddy's China build serves its Claude models on Anthropic's Messages API (
@ai-sdk/anthropicat<endpoint>/v1/messages), beside the/v2/chat/completionsthe plugin uses today. The plugin now sends them there. Every other model stays on chat, unchanged. WorkBuddy AI keeps all of its models on chat: it has no Messages route.This is the general route for any Anthropic-shaped client — Claude Code, the Claude Agent SDK, anything that sends
/v1/messages— against WorkBuddy's Claude models, with no translation layer in between.Why
Reasoning effort has no effect on a Claude model over chat. A chat carries
reasoning_effortalone, and the chat endpoint answers at about the same length whatever it says. Same prompt,claude-opus-5.5, 4 runs per level, median output tokens:lowmax/v2/chat/completions(reasoning_effort)/v1/messages(output_config.effort)The plugin also drops a Claude model's
maxtoday, because/v3/configlists only up toxhigh, so magpie sendsmaxasxhigh(the routing log showseffort: max→ tryeffort: xhigh).Two more things follow from the route:
FLAGGED/systemAsUserworkarounds are needed.Changes (
packages/workbuddy, 0.1.31 → 0.1.32)nativeOf(site, model): a Claude model on a site withmessages: true(the China build alone) goes to Messages; everything else stays on chat. Before choosing it, each/v3/configclimodel was asked on/v1/messages: the four Claude models answered in Anthropic's stream;glm-5.3anddeepseek-v4.1-flashreturned 500,gpt-5.5andkimi-k3-1400.apiis{ npm: "@ai-sdk/anthropic", url: "<endpoint>/v1" }, andconfigModelswritesprovider: { npm, api }, where OpenCode and magpie's host read it. magpie core needs no change: it routes plugin models bynpmalready.baseURL; each model'sapi.urldecides. A provider-wide/v2base had put/messagesunder/v2, a 404.max_completion_tokens/developer, tool reshaping, tool renaming, moving a refused system prompt, the empty reply after tool results). Sign-in, 401 re-issue, enterprise headers and the conversation/turn ids are shared by both routes.count_tokensis a Messages request too, so it no longer takes the chat rewrites.maxadded to its levels where/v3/configstops atxhigh.thinkingblocks (a session moved here from chat or another provider) are left out. With a signed block beside one, every request of the session is 11159 "bound to a different conversation";drop_blockdoesn't help. A message emptied that way keeps[{type:"text",text:""}], which WorkBuddy accepts (Anthropic's own API would not).X-Conversation-Request-IDon this route too (workbuddy: one user send, one X-Conversation-Request-ID #48): ausermessage oftool_resultblocks answers thetool_usebefore it, and Claude Code's{role:"system"}messages between steps are passed over.Verified
HOME=$(mktemp -d) bun test: 149 pass, 0 fail.bun scripts/check.mjs: ✓. Every new guard is pinned by mutation (revert the fix, a test goes red).--effort max:@ai-sdk/anthropic,low … xhigh, max.effort: max→ tryeffort: max, 200. Before, the same session showed tryeffort: xhigh.max, history left with signed and unsigned blocks mixed, task still verified.anthropic-version,anthropic-beta,cache_controlwithttl: 1h,context_management,metadata, image input,max_tokens: 64000. Prompt cache is read back (cache_readgrows turn over turn).Not covered
/v1/messages/count_tokensreturns 404. magpie treats a 404 there as "no counter" (unsupportedCount) and answers with its own estimate; a count through magpie returns 200 with one.maxthinks thanxhighvaries by model and run; the table is one prompt, 4 runs per cell. What the change guarantees is thatmaxreaches the server asmax.workbuddy-aiis excluded by construction, not by a request.