[CMGS]

Problem

Two prompt-cache gaps, both found while running coding agents through the gateway to OpenRouter.

1. /v1/messages lost Claude Code's mid-conversation system messages.

• Current Claude Code sends role: "system" entries inside messages, such as its environment block and per-turn reminders. Some carry a per-message output_config.
• The request's message-level cache breakpoint usually sits on the trailing one.
• ClaudeEngine folded these turns into system_parts, then threw them away whenever the top-level system was a block array, which it always is for Claude Code. So the model never saw them, and only the system prompt cached.
• Block-level cache_control on user, tool_result and tool entries was already forwarded unchanged. The loss was specific to system turns.

2. OpenRouter's Responses API caches nothing for anthropic/* models unless the request carries a top-level cache_control. Codex sends none.

Change

• Messages surface. On anthropic-messages and aws-anthropic, system turns stay in place, verbatim. Extra per-message fields (output_config, clear_at) ride along through a new InMessage.extra → ChatMsg.extra. On aws-converse, which has no such turn, they are appended to the system blocks instead of being dropped. They are also appended there when a fallback or variant serves a pre-5 Claude model in place of the one the client named: Claude Code sends these turns only for 5.x models, and 4.x models reject both the turn and its per-turn effort with a 400 ("role 'system' is not supported on this model", "output_config.effort requires a model that supports per-turn effort", or "Extra inputs are not permitted" on Bedrock). The swaps are fallback_models, variants and a tenant's fallback_model; each sets fallback_from. A swap to another 5.x model or to a non-Claude Anthropic-wire vendor keeps them in place, so a variant alias of the same model keeps its message-level caching. The chat surface is unchanged.
• prompt_cache precedence. The gateway's breakpoints are placed after the body is fully built. Client breakpoints win:
  • The latest user turn (skipping trailing system turns) is marked unless the client marked a message block (tool_result content included) or sent a top-level cache_control. System or tool breakpoints alone don't suppress it, so a system-only client keeps conversation caching.
  • The system prompt is marked unless the client marked it, or a 1-hour breakpoint follows it. A 5-minute entry may not come before a 1-hour one.
  • The user turn takes a free slot first. Neither is added past Anthropic's four breakpoints.
• Responses on OpenRouter. When prompt_cache is on, the account's kind is openrouter and the model is anthropic/* (including ~anthropic/* aliases), the body gets cache_control: {type: ephemeral} unless the client set one. This uses the existing per-model knob; there is no new config.
• Docs. docs/configuration.md and docs/examples.md are updated.

Hot-path cost

• Messages: no new allocation in the body transform. System turns are moved, not cloned, and the in-place path skips the system_text() build.
• serde(flatten) on InMessage: buffers only unknown per-message keys. Known fields, including content, are deserialized directly.
• Breakpoint scan: a read-only walk over every block, run only for prompt_cache models (no allocation).
• Responses: three cheap checks, plus one map insert when the marker is injected.

Tests

New tests fail on main and pass here:

• native_system_turns_stay_in_place_on_the_anthropic_wires
• native_system_turns_fold_into_the_system_prompt_on_converse
• prompt_cache_fills_only_the_breakpoints_the_client_left_out covers 10 cases: system-only clients (5m and 1h) get the turn breakpoint; message, tool_result or top-level breakpoints suppress it; a 1h message or top-level breakpoint suppresses the gateway's system breakpoint; four taken means none added; with one slot left it goes to the turn. Every case also checks that no 5m breakpoint precedes a 1h one. main fails the 1h case (it adds a 5m system breakpoint ahead of the client's 1h one), and the first commit of this PR fails the system-only case.
• prompt_cache_marks_the_last_user_turn_ahead_of_a_trailing_system_turn
• openrouter_anthropic_responses_carry_a_top_level_cache_control
• views messages_system_turns_reach_the_anthropic_upstream_in_place, which runs the full /v1/messages path with a Claude Code-shaped body

Gates: cargo fmt --check, cargo clippy --workspace --all-targets -- -D warnings and cargo test --workspace are clean. The GW_TEST_PG_URL / GW_TEST_REDIS_URL suites were not run; these changes don't touch the store code they cover.

What Claude Code loses on main

Captured from Claude Code 2.1.310, three -p turns, tools off, against a local recorder. Every request carries these messages entries, and main drops all of them:

• Environment message. A role: "system" turn after the first user turn: working directory, git repo flag, platform, shell, OS version, the untrusted-download rule, model name and ID, knowledge cutoff, the <total_tokens> budget and today's date. It is 976 characters and carries output_config: {effort: "medium"}.
• Per-turn reminders. One role: "system" turn after every later user turn (<total_tokens>… tokens left</total_tokens>, 49 characters). Earlier copies stay in history, and the newest one carries the request's only message-level cache_control.
• The message breakpoint. Because it rides on the dropped reminder, main caches only the system prompt.

Token cost, measured on the live probe below: the environment message plus one reminder came to 405 prompt tokens (19,246 with them vs 18,841 without). That figure includes about 30 tokens of probe-only rule text. Each further turn adds roughly one more reminder. With tools, Claude Code may send more system turns; this capture had tools off.

Live probe (OpenRouter, anthropic/claude-opus-5.5, direct)

call	result
Messages with two mid-conversation system turns (one with output_config, the trailing one with cache_control)	HTTP 200. Both turn-only instructions were honored: the reply came back in uppercase and ended with the marker word. Cache write 19,244 tokens.
Same request repeated	HTTP 200. cache_read_input_tokens 19,244, i.e. the whole prefix through the trailing system turn, not just the system prompt. Cost $0.0987 → $0.0062.
Same request without the system turns	HTTP 200. Plain reply with neither instruction applied. 18,841 prompt tokens (the 405-token delta above).
Responses with top-level cache_control, twice	HTTP 200 both times. Cached 18,813, then 18,823. The first call also read the prefix the Messages calls had written (same text), and wrote 10 tokens of its own, which the second call read.

The Responses pair shows that top-level cache_control is accepted, reads cache and writes cache. It does not isolate the flag against a no-flag baseline on a fresh prefix. That baseline (0 cached without the flag) comes from an earlier manual probe.

Anthropic-wire accounts in use

The two deployed gateways have one anthropic-messages account between them, on OpenRouter. Neither has an aws-anthropic account or a third-party Anthropic-compatible vendor (e.g. MiniMax). The opt-in guard (keep folding for non-Anthropic providers unless the account opts in) is not added: no such account exists. Anthropic-compatible vendors configured elsewhere will now receive the client's system turns as sent.

Live check on Bedrock

The gateway was run locally from this branch, authenticating with a Bedrock API key (bearer token), region us-east-1, model Opus 5.5.

call	route	result
(a) mid-conversation system turns (one with output_config, the trailing one with cache_control)	aws-anthropic, us.anthropic.claude-opus-5-5, in place	HTTP 200, both turn-only instructions honored. Cache write 18,980, repeat read 18,980.
(b) same request	aws-converse, global.anthropic.claude-opus-5-5, folded	HTTP 200, both honored. Cache write 18,951, repeat read 18,951.
(c) string top-level system plus a cache-marked system turn	aws-converse	Before this fix, the top-level instruction was lost (reply without it). After it, the instruction is kept (BANANA … PINEAPPLE).
(d) 1h user-block breakpoint, then a 5m-marked system turn	aws-converse	Before this fix: HTTP 424, Bedrock reports "a ttl='1h' cache_control block must not come after a ttl='5m' cache_control block". After it: HTTP 200.
(e) direct us.anthropic.claude-sonnet-4-6 with in-place system turns (control)	aws-anthropic	HTTP 424, "messages.1.output_config: Extra inputs are not permitted".
(f) variant to us.anthropic.claude-sonnet-4-6	aws-anthropic, folded	HTTP 200, both instructions honored.
(g) variant to global.anthropic.claude-opus-5-5	aws-anthropic, in place	HTTP 200, both instructions honored.

The fix keeps the top-level text ahead of the folded turns, and drops a folded turn's shorter-TTL breakpoint when a longer-TTL breakpoint follows in messages. The later breakpoint still covers that prefix. Unit tests a_cache_marked_system_turn_keeps_the_top_level_system_text_on_converse and a_folded_system_turn_never_puts_a_5m_breakpoint_before_a_1h_one_on_converse fail without it.

Swapped models: which turns fold (544236b)

Pre-5 detection. The pre-5 check came from the thinking dialect, which parses only Anthropic's hyphenated ids. OpenRouter's anthropic/claude-sonnet-4.6, -opus-4.7, -haiku-4.5, -opus-4 and -sonnet-4 were all treated as 5.x. A variant from claude-opus-5-5 to OpenRouter anthropic/claude-sonnet-4.6 therefore returned 424 on de41d92:

Mid-conversation reasoning effort (configuration_update) is not supported

claude_before_5 now reads the major version that follows claude.

Per-turn effort. Sonnet 5 takes a mid-conversation system turn but rejects its output_config, even with the beta. A swapped request now keeps 5.x turns in place without output_config.

Measured live on the Anthropic API.

model	plain system turn	turn with output_config + per-turn-control-2026-07-01
opus-5.5, opus-5, fable-5.1	200	200
sonnet-5	200	400 "requires a model that supports per-turn effort"
4.x (opus-4.6/4.7, sonnet-4.5/4.6, haiku-4.5)	400 "role 'system' is not supported"	400

On OpenRouter, anthropic/claude-sonnet-4.6 accepts a plain turn and rejects output_config.

What Claude Code 2.1.310 sends (captured locally):

• opus-5.5, fable-5.1 and Bedrock opus-5.5 ids: output_config on the system turns, plus the per-turn beta.
• sonnet-5, opus-5 and OpenRouter anthropic/claude-opus-5.5: plain turns.
• 4.x: no system turns.

Tests.

• native_system_turns_fold_into_the_system_prompt_on_converse_or_a_swapped_model adds anthropic/claude-sonnet-4.6 and anthropic/claude-opus-4.
• native_system_turns_stay_in_place_on_the_anthropic_wires adds a swapped claude-sonnet-5, and now expects swapped turns, the ~anthropic/claude-opus-latest alias included, without output_config.
• Both fail on de41d92.

E2E (e17dc86 vs main a7d6c36)

Full live matrix, every key we hold: main 182/241, PR 182/241, the same cases on both sides.

The 59 failures are vendor-side:

• The Bedrock tokens in our ledger have expired. That run used them; the Bedrock rerun below uses a working token.
• SiliconFlow and Kling have no balance left.
• gpt-4.1-nano is deprecated.

Rerun of the bedrock, agents, anthropic and openrouter groups with a working Bedrock token (us-east-1): main 74/74, this branch 74/74, identical.

Claude Code 2.1.310 through the gateway. Two turns with the Write and Read tools on each route. The proxy logged every upstream request and its usage.

route	main	this branch
direct claude-opus-5-5	turns dropped; each call writes 40–400 tokens of cache	4 system turns in place (first with output_config); 0 cache writes after the first call
variant → claude-sonnet-5	dropped	in place without output_config, 200, 0 writes
OpenRouter anthropic/claude-opus-5.5 (direct and as a variant)	dropped	in place, 0 writes
OpenRouter variant → ~anthropic/claude-opus-latest	dropped	in place, 0 writes
variant → claude-sonnet-4-6 / OpenRouter anthropic/claude-sonnet-4.6	dropped	folded into system (3 → 7 blocks), 200
Bedrock us.anthropic.claude-opus-5-5 (aws-anthropic)	dropped	in place with output_config, 0 writes
Bedrock global.anthropic.claude-opus-5-5 (aws-converse)	dropped	folded (5 → 10 blocks), 200
Bedrock variant → us.anthropic.claude-sonnet-4-6	dropped	folded, 200

All 20 runs completed both turns.

Bedrock betas (bf1f6e1)

On main and on e17dc86, Bedrock answered Claude Code's first attempt on every request with 400. Claude Code then retried, and the retry succeeded:

• aws-anthropic rejected the advanced-tool-use-2025-11-20 beta. Claude Code's retry also turned off defer_loading, which loaded every tool schema: about 27.8k prompt tokens per request instead of 16.6k.
• aws-converse rejected context_management. Converse received no betas at all, and Claude Code's retry dropped context management.

What Bedrock accepts, probed one beta at a time on InvokeModel with Opus 5.5 in us-east-1:

• Rejected: advisor-tool-2026-03-01, advanced-tool-use-2025-11-20 and prompt-caching-scope-2026-01-05.
• Accepted: every other beta Claude Code sends, and fine-grained-tool-streaming.
• Without the beta, defer_loading is still honored.
• Converse accepts context_management once anthropic_beta: ["context-management-2025-06-27"] rides in additionalModelRequestFields.

Fix.

• Those three betas are dropped for Bedrock.
• The rest of the client's list goes to aws-anthropic as before. It now also goes to Claude models on aws-converse, where it lands in additionalModelRequestFields.
• Non-Claude Converse models are unchanged.
• The test bedrock_drops_the_betas_it_rejects_and_converse_carries_the_rest_for_claude fails on e17dc86.
• docs/api.md, docs/examples.md and docs/providers.md state the rule.

Claude Code 2.1.310 through the gateway, e17dc86 vs bf1f6e1, with the proxy recording upstream:

route	e17dc86	bf1f6e1
Bedrock us.anthropic.claude-opus-5-5	2 × 400, 6 calls	0 errors, 4 calls, 7 betas, context_management kept
Bedrock global.anthropic.claude-opus-5-5 (Converse)	4 × 400; the retry ran without context_management	0 errors, 4 calls, context_management and 7 betas delivered
Bedrock variant → us.anthropic.claude-sonnet-4-6	2 × 400	0 errors
Anthropic direct, OpenRouter direct	0 errors	0 errors, the same

The bedrock, agents, anthropic and openrouter matrix groups went 74/74 on both builds, identical.

cargo test --workspace with real Postgres 17 and Redis 7 gives 773 passed, 0 failed, 4 ignored. fmt and clippy are clean.

Gates. cargo fmt --check and cargo clippy --workspace --all-targets -D warnings are clean. cargo test --workspace with real Postgres 17 and Redis 7 gives 772 passed, 0 failed, 4 ignored; it created 4 test databases and wrote 7 Redis keys.

Effort-only system turns on a swap (1153e8b)

An effort-only system turn ({"role": "system", "content": [], "output_config": {...}}) loses its only field when the request is swapped. Anthropic refuses the empty turn that is left: system content must contain at least one block. It does so on opus-5.5 as well as sonnet-5.

A variant from claude-opus-5-5 to claude-sonnet-5 carrying such a turn:

• bf1f6e1: 424.
• This commit: 200, and the remaining turn's instruction is honored.

Such a turn is now skipped. A client's own empty turn still passes through as sent. native_system_turns_stay_in_place_on_the_anthropic_wires fails on bf1f6e1.

Gates: fmt and clippy are clean. cargo test --workspace with real Postgres 17 and Redis 7: 773 passed, 0 failed, 4 ignored.

Size

Against main a7d6c36:

• production +125
• comments ±0
• tests +411

29b8cda is a test-only cleanup, with no assertion changed and the test count unchanged at 773:

• One row per case in the four case tables.
• One CONVERSE_OK fixture in place of five copies.
• One system_texts helper.
• Net −143 test lines.

Known limitations

None open from review. The two Converse folding issues are fixed above, and aws-anthropic in-place system turns are verified live.