[pcbeingused333]

Gemini's OpenAI-compatible endpoint (https://generativelanguage.googleapis.com/v1beta/openai/, configured through extra-openai-models.yaml) streams parallel tool calls without tool_calls[].index. Each call arrives whole in its own chunk, and the stream ends with finish_reason: "stop":

data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"function":{"arguments":"{\"city\":\"Paris\"}","name":"get_weather"},"id":"call_4234484","type":"function"}]},"index":0}], ...}
data: {"choices":[{"delta":{"role":"assistant","tool_calls":[{"function":{"arguments":"{\"city\":\"Tokyo\"}","name":"get_weather"},"id":"call_4234487","type":"function"}]},"index":0}], ...}
data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":"stop","index":0}], ...}

Chat.execute and AsyncChat.execute key the accumulated calls by tool_call.index, so both calls go under None. The second call's arguments get appended to the first, and json.loads('{"city":"Paris"}{"city":"Tokyo"}') raises JSONDecodeError: Extra data when the stream ends. With this endpoint, any prompt where the model makes more than one tool call in a turn fails this way.

The fix adds a small _tool_call_key() helper that both paths use. When index is present nothing changes. When it is missing, a delta starts a new call only if it has a new id (different from the latest call's id), or a name with no id. Anything else continues the latest call, so providers that repeat the same id on every argument fragment still get one call.

Tests (in tests/test_openai_messages.py):

• two unindexed parallel calls, sync and async: these fail on main with the JSONDecodeError and pass with this change
• one unindexed call whose id is repeated on each argument fragment, to make sure it isn't split into two calls

The full test suite, black --check, ruff check and mypy llm pass. A stream captured from Gemini was also replayed through Chat and AsyncChat against a local server: main raises, and this branch returns the two calls (Paris, Tokyo) with their original ids.

This change was made with Claude Code.

🤖 Generated with Claude Code

[Muguett-DBY]

Would it be worth guarding against a key collision if a compatible endpoint mixes indexed and unindexed calls? For example, if the accumulator already has a call at index 1 and a new unindexed call arrives with a different id, len(tool_calls) is also 1. _tool_call_key returns that occupied key, so the caller appends the new arguments to the existing call.

I reproduced the collision by executing the exact helper from 4445d4a with those inputs and checked the sync/async accumulation paths. I don't know whether Gemini emits mixed streams; this is a defensive case rather than a claim about its behavior. A distinct key for the unindexed call and a mixed-stream regression test could make the fallback safer. I haven't run the full branch test suite.

[pcbeingused333]

Good catch, @Muguett-DBY, thanks for reproducing it. Added 036bad7: when a call arrives without an index, the key now skips any slot already taken by an indexed call, so a mixed stream (indexed calls plus an unindexed one) no longer merges the unindexed call into an existing one. There's a regression test for the mixed case (test_unindexed_call_does_not_reuse_taken_index), which fails without the change and passes with it.

[Muguett-DBY]

Thanks for fixing that! The new test covers an indexed call followed by an unindexed one. I checked the opposite order too: if an unindexed call with id call_1 arrives first, it takes key 0; a new indexed call with id call_2 and index 0 then returns the same key. The accumulator joins their arguments and loses the second call.

I reproduced this with the exact 036bad7 helper and the same accumulation logic; I don't know whether a real provider sends mixed streams in this order. Would separate key namespaces for explicit indices and unindexed calls, while retaining continuation handling, be worth considering?

[pcbeingused333]

Thanks. Done in ca1d22a: indexed and unindexed calls now accumulate under separate keys, so neither order can merge two calls; added a test for the unindexed-then-indexed order. Like you, I don't know of a provider that mixes both in one stream; this is defensive.