Repository navigation
Bound the Python chain API by default - #1777
XXXXXQ-0206 wants to merge 3 commits into
Conversation
Conversation.chain_limit defaulted to None and Model.conversation() passed it explicitly, so the default of 10 that ChainResponse documents never applied and model.chain() looped forever on a model that keeps asking for tools.
chrikrah
left a comment
There was a problem hiding this comment.
@XXXXXQ-0206, the code does what #1773 describes, and whether it should land is Simon's call, not mine. The four new tests pass on your head 68b5227. With models.py from your parent 2465dfc the first test fails and the file hangs until timeout kills it.
# head 68b52275, exported with git archive, llm imported from that tree; Python 3.12.3, pytest 9.1.1
$ python -m pytest tests/test_chain_limit.py -q -p no:cacheprovider
4 passed in 0.35s
# same tree, llm/models.py from 2465dfc
$ timeout 15 python -m pytest tests/test_chain_limit.py -q -p no:cacheprovider
F
rc=124
$ python probe.py # .chain_limit of the ChainResponse each call builds, echo model; one run per tree, side by side
2465dfc 68b5227
model.chain() None 10
conversation().chain() None 10
conversation().chain(chain_limit=None) None 10
conversation(chain_limit=None).chain() None None
conversation(chain_limit=0).chain() 0 0
$ git merge-tree --write-tree origin/main 68b5227 >/dev/null; echo $? # main aebaca2
0
# not run: the full suite; the network-marked tests
The case for a bound: before 2df619e, Conversation.chain() passed no limit, so the ChainResponse default of 10 applied. The case against: docs/python-api.md:138 calls model.chain() a loop that "keeps going until the model stops requesting tools", and this PR leaves that line alone. The cost lands on Python callers. A loop past 10 tool calls, or a deliberate .chain(chain_limit=None), now raises ValueError: Chain limit of 10 exceeded. with no deprecation step, so this belongs in the changelog.
@simonw, was the unbounded Python chain after 2df619e intended? If not, python-api.md:138 needs the same change, plus a line naming 0 as the opt-out.
The Python API docs still described model.chain() as looping until the model stops requesting tools, which no longer matches the code. Note the 10 call bound, name chain_limit=0 as the opt-out and cover it with a test, and add the behaviour change to the changelog since it can raise ValueError where it previously did not.
|
Thanks @chrikrah — picked up all three points in
I also added On the full suite: I cannot get it green locally, but the failures are pre-existing in this environment rather than from this PR.
|
chrikrah
left a comment
There was a problem hiding this comment.
@XXXXXQ-0206 the docs, the changelog line and the chain_limit=0 test cover my three points at f8bb8a6. One thing will fail the Test workflow once it runs, and I missed it at 68b5227 too, so my approval holds only once black passes.
blocking: test.yml runs black --check ., and it wants to reformat tests/test_chain_limit.py.
$ python -m pytest tests/test_chain_limit.py -q -p no:cacheprovider # f8bb8a6, llm imported from the export, Python 3.12
5 passed in 0.15s
$ black --check . # black 26.10.1
would reformat tests/test_chain_limit.py
1 file would be reformatted, 56 files would be left unchanged.
$ black --check . # base 2465dfc
56 files would be left unchanged.
Can you push black tests/test_chain_limit.py? With that in, this is ready for Simon.
ChainResponsedefaultschain_limitto 10, but no public Python entry point applied it:Conversation.chain_limitwasNoneandModel.conversation()passedchain_limit=None, somodel.chain()andmodel.conversation().chain()ran with no limit. The CLI is bounded separately because--chain-limitdefaults to 5.The default now lives in one place,
models.DEFAULT_CHAIN_LIMIT, used by both_BaseConversation.chain_limitandChainResponse, and the twoconversation()factories default to it instead ofNone, so the existingself.chain_limitfallbacks have a value to fall back to. An explicitchain_limitstill wins.tests/test_chain_limit.pycovers the default, both unbounded entry points, and that an explicit limit still wins. The two chain tests fail on the current code — they never finish — and pass with the change;test_chain_callbacks.pyandtest_pause_resume.pyare 20 passed before and after.Not run: the full suite needs the
llm-echoplugin and network-marked tests and did not finish here; I verified with the files above.Fixes #1773