Skip to content

Test that each turn's tool calls name that turn's tool definition - #1828

Open
connabyss wants to merge 1 commit into
simonw:mainfrom
connabyss:test-tool-call-ids-per-turn
Open

connabyss wants to merge 1 commit into
simonw:mainfrom
connabyss:test-tool-call-ids-per-turn

Conversation

@connabyss

Copy link
Copy Markdown

0775cfa ("Resolve tool ids through each turn's turn_tools rows") fixed tool_calls[].tool_id / tool_results[].tool_id in log_row_extras, so a turn reports the tool definition it was actually given when two definitions share a name.

The test added with it, test_same_name_different_definitions_resolve_per_turn, makes no tool calls. It checks extras["tools"], which came from the per-turn turn_tools query before that commit too. So it still passes if the old global name-to-id map is put back.

This adds test_tool_calls_name_the_definition_their_turn_was_given to the same class. It has two turns, each calling lookup with a different definition, and asserts that each call's tool_id matches its own turn's tool.

  • With the old global map (select id, name from tools) swapped back in: the new test fails (turn 1's call points at turn 2's definition), and the existing test still passes.
  • On main: all 177 tests in tests/test_logs_store.py pass. black --check . and ruff check . are clean.

Test only, no code changes.

Found and reviewed with AI tools; I ran the test against the old and current code before opening this.

test_same_name_different_definitions_resolve_per_turn checks
extras["tools"], but that list came from the per-turn turn_tools
query before 0775cfa too, and the test makes no tool calls. So it
still passes with the old global name-to-id map. This test has each
turn call "lookup" and checks the call's tool_id against the turn's
own definition. It fails with the global map and passes on main.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0175Q8iD3ANaDC2DqYJfmj7x

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant