Repository navigation
fix(backtests): strip the model's action before matching it - #662
Merged
FlyM1ss merged 3 commits intoOct 30, 2026
Merged
Conversation
Contributor
|
@subidhkhanal is attempting to deploy a commit to the allan-feng's projects Team on Vercel. A member of the Team first needs to authorize it. |
Open-Finance-Lab#658 was two readings of one field: the strict check stripped the action, the trade loop did not, so "buy " passed one and matched nothing in the other. This puts every reader of a model's action on one helper and makes an action nobody recognises visible instead of a silent hold. - validator: normalize_action (ASCII trim + ASCII lowercase, None -> "", non-strings via str) and TRADING_ACTIONS, beside normalize_symbol. - portfolio_manager: the strict check and the trade loop both use it. A non-strict loop skips an unrecognised action with a log line instead of treating it as a hold, and a batch with no recognised action holds without crediting llm_decisions (H6). A null or non-text reasoning no longer raises (None[:60] aborted billed runs and sent lenient ones to the rule-based fallback); neither does a null or numeric action. - pipeline_runner: a blank side falls through to the action alias; unrecognised sides and risk actions are handed on as written for the gate to refuse (as Open-Finance-Lab#673 did for size), instead of being coerced to hold. - decision_tape: _order_triples reads action and symbol as the gate does, so gate_rewrote stops flagging a padded order the gate accepted. - AI Hedge Fund adapter, Robinhood live service, Trading Algo backtest: same helper for the action (and the symbol, in the Trading Algo). - backtest_custom_algo: import timedelta from datetime. cd402c8 removed it from backtest_hourly_agent, so every Trading Algo run raised ImportError on its first decision. - SECURITY.md: action vocabulary row; the replay note no longer promises byte-for-byte reproduction across gate changes. - tests: fixture keys fixed (sma20/sma50); strict refusal, lenient skip, H6 credit, non-text fields, pipeline sides, the helper itself, the Robinhood path, and a Trading Algo subprocess run. Two tests that pinned the old crashes now pin the new behaviour; the harness's blanket-except test injects its fault instead of relying on a malformed field. Behaviour change: a pipeline side such as "Buy." used to hold silently; a strict (billed) run now refuses it and aborts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
FlyM1ss
added a commit
to subidhkhanal/AgenticTrading
that referenced
this pull request
Oct 30, 2026
Conflicts with Open-Finance-Lab#662 (normalize_action), all combined: - SECURITY.md: main's count-cap Action count row, this branch's split BUY/SELL per-order size rows. - decision_tape._order_triples: main's normalize_symbol, this branch's sell sizes and unsorted triples (gate_rewrote pairs them as a multiset). - portfolio_manager imports: both model_number and normalize_action. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 30, 2026
FlyM1ss
added a commit
to subidhkhanal/AgenticTrading
that referenced
this pull request
Oct 30, 2026
…confidence-range Conflicts resolved: - portfolio_manager.py: kept main's `_coerce_position_size` and dropped this branch's identical `_requested_buy_shares`; kept both `recognised_any` (Open-Finance-Lab#658) and `skipped_for_confidence`. Open-Finance-Lab#667's sell branch formatted `{confidence:.0%}`, which raises on the unreadable confidence this gate lets a sell trade on; it now uses `conf_label`. - engine.py, leaderboard/service.py and their metadata tests: record both `llm_gate_version` and `llm_sell_sizing`. - validator.py: Open-Finance-Lab#667's sell-sizing block, then the public `confidence_in_range`. - backtest_custom_algo.py, ai_hedge_fund/adapter.py: Open-Finance-Lab#662's normalize_symbol/normalize_action alongside read_confidence and confidence_in_range. - SECURITY.md: the versioned-gate paragraph now names `llm_sell_sizing` and the reply readings; the Trading Algo row lists all three shared readers. LLM_GATE_VERSION's history notes that v2, the first recorded version, also covers Open-Finance-Lab#658 and Open-Finance-Lab#661. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
computedocean
pushed a commit
to computedocean/AgenticTrading
that referenced
this pull request
Oct 31, 2026
The risk-branch comment in pipeline_output_to_decision said any positive size_pct sells the whole position because the gate does not size a sell. That was written against the pre-Open-Finance-Lab#667 gate and reached main through Open-Finance-Lab#662's follow-up; since Open-Finance-Lab#667 the gate sizes it as a share of the position (0.25 sells a quarter, 1 or more sells all). Comment only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #658
The problem
If the model writes its action with an extra space, like
"buy ", the trade is silently skipped, but the step still counts as a model decision.The safety check removes the space, so
"buy "passes. The code that places the trade doesn't, so"buy "doesn't match"buy"and nothing happens. The pipeline path had the same problem:"buy "was turned into"hold"."buy""buy "The fix
Remove the extra spaces before comparing, the same way the safety check already does. That's a
.strip()in three places:portfolio_manager.py: the loop that places tradespipeline_runner.py: pipelineordersandrisk_actionsTests
"buy "and" BUY\n"now place one buy, in strict and normal mode."buy "stays"buy", and" stop_loss\n"becomes"sell".mainand pass here.Note
robinhood_live_service.pyhas the same pattern, but that real-money path was switched off in #656, so I left it alone.Maintainer follow-up (f34a27e, 0639336)
Merged
mainin (conflicts withnormalize_symbol/read_confidence), then fixed what the review found:validator.normalize_action(ASCII trim + lowercase) is used by the strict check, the trade loop, the pipeline converter, the decision tape, the AI Hedge Fund adapter, the Trading Algo, and the Robinhood path (so the note above is now handled).hold; it hands it on and the gate decides, as fix(backtests): read every LLM action's confidence by one 0-1 rule #673 did for size.nullor non-textreasoning, and anullor numericaction, no longer throw the step into the fallback (a billed run used to abort onNone[:60]).backtest_custom_algo.pyimportedtimedeltafrombacktest_hourly_agent, whichcd402c8aremoved, so every run raisedImportErroron its first decision. Fixed, with a subprocess test.Behaviour change: a pipeline side like
"Buy."used to hold silently; a strict (billed) run now refuses it and aborts.Full backend suite green apart from the known local-only
test_report_pdf(reportlab) andtest_robinhood_mcp_transport(mcp version) failures.🤖 Generated with Claude Code