Repository navigation
Pin HelmsDeep v0.5.0; use Shepherd infores names in the sweep examples - #65
Merged
Merged
Conversation
v0.5.0 stops scoring an ARA's refusal of a query class it doesn't support (e.g. BTE on Pathfinder, a 500 saying "does not support", or a 501) as a load failure in the sync ARA runs, so the mixed profile against BTE no longer fails every checkpoint on it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S2yuRAujys2XXK6YPjSay3
The sweep passes each target's NAME to --target, and --trapi_version auto looks the TRAPI version up in SmartAPI by infores:<NAME>. The examples used aragorn/arax/bte, but the Shepherd-hosted ARAs are registered as shepherd-aragorn/shepherd-arax/shepherd-bte, so a sweep copied from them read the wrong entries (or none, falling back to 2.0.0 with only a warning) and sent Aragorn and BTE TRAPI 2.0 while SmartAPI lists them at 1.6.0. Updates the README, the sweep docstring and --help, the CronJob and Helm examples, and the chart's error message, and says in the README and the --target / NAME help that the name must be the SmartAPI-registered infores (it also labels the results and the Radiator agent). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01S2yuRAujys2XXK6YPjSay3
maximusunc
marked this pull request as ready for review
November 1, 2026 14:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
requirements-runners.txtfromv0.4.0tov0.5.0.aragorn/arax/btetoshepherd-aragorn/shepherd-arax/shepherd-bte, and documents why the name matters.Why
HelmsDeep v0.5.0
v0.5.0 stops counting an ARA's refusal of a query type it doesn't support as a load failure. BTE answers every Pathfinder query with a 500 saying "BTE does not support Pathfinder type queries.", so the mixed (
aras_mixed) run against BTE failed every checkpoint on Pathfinder errors.On the sync path, HelmsDeep now treats such a refusal as unsupported instead of failed. That covers a 501, or a "does not support" / "not supported" / "not implemented" message in the response's status, description or error-level logs. An unsupported answer:
summary.jsonThe ARS path is unaffected. A related Shepherd change, BioPack-team/shepherd#200, makes BTE return 501 for Pathfinder. HelmsDeep handles either form, so this pin doesn't depend on it.
Sweep target names
The sweep passes each target's
NAMEto--target. With the default--trapi_version auto, the version is looked up in SmartAPI asinfores:<NAME>. The Shepherd-hosted ARAs are registered asinfores:shepherd-aragorn,infores:shepherd-araxandinfores:shepherd-bte. A sweep using the bare names read the wrong entries, or none: when nothing is found it falls back to 2.0.0 with only a log warning. As a result, Aragorn and BTE were sent TRAPI 2.0 while SmartAPI lists them at 1.6.0.Updated:
README.mdtest_harness/sweep.pydocstring and--helpdeploy/cronjob.example.yamldeploy/helm/test-harness/values.example.yamlsweep.targetserror messageThe README and the
--target/NAMEhelp now say the name must be the infores the service is registered under in SmartAPI. They also say what else the name controls: each target's output directory and filename prefix, and the agent the Information Radiator reports under.Testing
pip install, because thev0.5.0tag didn't exist yet. It now exists on HelmsDeepmain(merge of Don't score an ARA's "not supported" refusal as a load failure HelmsDeep#23).pytestpasses (205 tests) andblack --check .is clean.🤖 Generated with Claude Code
https://claude.ai/code/session_01S2yuRAujys2XXK6YPjSay3