Repository navigation
fix: support vLLM 0.31 inference history hooks - #1136
Merged
Merged
Conversation
bradhilton
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Agent: Jarvis
Qwen3.6 B300
/v1/responsesreturns HTTP 500 because vLLM 0.31 removed_make_request. Feature-detect the native_render_resolved_response_inputsresult while keeping the vLLM 0.24 path unchanged; preserve completed responses when optional re-rendering returns a native error. Patch the relocatedconstruct_input_messagesalias so previous-response reasoning/tool history survives.The hook audit also found that vLLM 0.31 passes DeepSeek V3.2 a cached full-chat
last_user_idx, bypassing ART's historical-prefix truncation. Discard that cached index only for preserved prefixes so the native renderer recomputes it.Validation: 293 history, append-only, token-prefix, chat-template and SGLang tests pass; changed-file type checking and Ruff pass. Tests cover both Responses shapes, streaming, previous-response lookup, native render errors and tool history. Downloaded/audited PyPI vLLM 0.24.0/0.31.0 sources; direct native DeepSeek encoding reproduces the regression before this fix and passes on both versions afterward. No GPU or production inference smoke was run.
Caladan adoption requires its five source/runtime pin files to be updated after this PR merges, followed by a rebuilt B300 runtime and Responses canary. This PR does not deploy anything.
🤖 Generated with Claude Code