Skip to content

fix(aikit): leave room for OpenRouter's input overcount when clamping max tokens - #137

Merged
sanchitrk merged 1 commit into
mainfrom
claude/openrouter-context-length-61f5be
Nov 6, 2026
Merged

sanchitrk merged 1 commit into
mainfrom
claude/openrouter-context-length-61f5be

Conversation

@sanchitrk

Copy link
Copy Markdown
Contributor

Problem

OpenRouter rejected a request at ~10% of a 1M window:

openrouter/stealth/crouton-alpha: this endpoint's maximum context length is 1048576 tokens. However, you requested about 1048871 tokens (80210 of text input, 24943 of tool input, 943718 in the output).

The model's catalog output limit spans the whole window, so clampMaxTokensToContext sized max_tokens as window − estimated input − 4096. aikit estimated the input at 100,762 tokens. OpenRouter counts with its own tokenizer before routing and got 105,153, about 4.4k more than the 4096-token margin, so the request went 295 tokens over the window.

Fix

The margin now scales with the input, matching Pi's fix for the same OpenRouter behaviour:

  • Leave 15% of the estimated input unused, at least 4096 tokens. OpenRouter has been seen counting prose up to 12% above the provider.
  • If that would leave less than 32,768 tokens of output (a nearly full large window, or a small window such as a 32k local model), keep only the fixed 4096 margin and cap output at 32,768.

For the failing request, the ceiling becomes 932,699, and 105,153 + 932,699 fits in the window. Small windows behave as before.

Workaround until this ships: cap the model in settings.jsonc with "stealth/crouton-alpha": { "maxTokens": 8192 }.

Tests

packages/aikit/test/maxtokens.test.ts adds Pi's clamp cases, including the failing request above. vp check and vp run typecheck pass.

🤖 Generated with Claude Code

… max tokens

OpenRouter checks input + max_tokens against the context window before
routing, using its own tokenizer, which can count several percent more
input than our estimate. With only 4096 tokens spare, models whose output
limit spans the window were rejected with a 400 ("This endpoint's maximum
context length is ...") at around 10% of the window.

The output ceiling now leaves 15% of the estimated input unused (at least
4096). When that would leave less than 32768 tokens of output, only the
fixed 4096 margin is kept and output is capped at 32768.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@sanchitrk
sanchitrk merged commit 007e6cc into main Nov 6, 2026
1 check passed
@sanchitrk
sanchitrk deleted the claude/openrouter-context-length-61f5be branch November 6, 2026 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant