Add opt-in Schwab + AnySearch data vendors, vendor provenance, and cross-source basis disclosure - #1207
Open
kevinkda wants to merge 7 commits into
Open
Add opt-in Schwab + AnySearch data vendors, vendor provenance, and cross-source basis disclosure#1207kevinkda wants to merge 7 commits into
kevinkda wants to merge 7 commits into
Conversation
Add a Schwab-backed OHLCV and technical-indicator vendor for US
equities/ETFs, opt-in behind the ``schwab`` extra. It consumes a
token.json produced by any schwab-py login (no OAuth login of its own)
via a synchronous schwab-py client_from_token_file client, restricts to
plain ^[A-Z]{1,5}$ symbols, classifies on HTTP status without
raise_for_status, disk-caches a 5-year daily window, and falls back to
yfinance on NotConfigured/NoMarketData. schwab-py is imported lazily so
a clean install without the extra never fails at import time.
The vendor follows the existing route_to_vendor / VENDOR_METHODS three
-layer architecture (same shape as the alpha_vantage vendor). Defaults
are unchanged (yfinance); enable by setting data_vendors for both
core_stock_apis and technical_indicators to "schwab,yfinance".
Also apply the low-cost fixes surfaced by the double-blind review:
- indicator path now runs the same stale-frame guard
(_assert_ohlcv_not_stale) as the OHLCV path, after look-ahead trimming
and before cleaning, so year-old frames never feed indicators;
- get_price_history passes an explicit frequency=Frequency.DAILY (wire
value 1) instead of relying on a server default;
- the same-day cache refresh restores the "historical request reuses the
cache forever, unaffected by TTL" short-circuit (only current-day
requests honour the TTL), matching the yfinance cache;
- minor: copy the trimmed frame before rounding to avoid
SettingWithCopyWarning, validate the title-case OHLCV column set on a
cache hit, and drop the previously-unused today_date parameter.
Version is intentionally not bumped (release cadence is the maintainers'
call); the changelog entry sits under [Unreleased].
Follow-ups deferred (not blocking): tighten the parity-test naming
(G2), the bare-BTC whitelist edge (G3), the tz day-boundary semantics
TODO (needs a live Schwab credential), and a wire-value assertion
against the real schwab-py enums (needs schwab-py in the test env).
Add a TRADINGAGENTS_DATA_VENDOR environment variable plus an interactive CLI prompt to choose the market-data source. The selection applies to both core_stock_apis and technical_indicators categories together, so vendors such as Schwab or Alpha Vantage can be picked at the CLI/env layer. The behavior is opt-in and defaults to yfinance, leaving the existing flow unchanged when the variable is unset.
route_to_vendor now records which vendor actually served each method and which primaries were skipped, logs the resolution at info level, and the CLI prints an "Effective configuration" panel at start plus a "Data source" panel at the end. This exposes silent degrades (e.g. an unconfigured Schwab primary skipped in favor of yfinance) that previously left users unsure which data source and Bedrock region/auth mode were actually in use (TauricResearch#988/TauricResearch#977/TauricResearch#1103). Also gitignore local convenience scripts (run.sh, run_aapl.py).
Realized returns for reflection/alpha are measured on yfinance split/dividend-adjusted closes (the correct outcome basis), but the agent may have priced its decision on an unadjusted vendor (e.g. Schwab raw prices). On a ticker with a split/large dividend inside the holding window, the two bases diverge, so an adjustment artifact could be written to the memory log and re-injected into future prompts as if it were a real lesson. Append a deterministic basis caveat to the stored reflection when the core price vendor is not yfinance, and disclose the yfinance-adjusted basis in the verified snapshot so a raw-vs-adjusted level gap is not mistaken for a data error. Keeps yfinance as the returns basis (no behavioral change for the default config).
The Schwab vendor sends an explicit 5-year daily window via PeriodType.YEAR / Period.FIVE_YEARS / FrequencyType.DAILY / Frequency.DAILY. A schwab-py upgrade that renamed a member or changed a wire value would silently send the wrong frequency and return the wrong candles. Add a contract test pinning those enums against the installed schwab-py (importorskip when absent) and a CI job that installs the [schwab] extra and runs the vendor + contract tests, so the drift fails loudly instead of in production.
AnySearch (api.anysearch.com /v1/search) provides real-time web/news search. Add it as an opt-in supplement for get_global_news only. AnySearch results carry no reliable publish date (verified against the live response), so the vendor is strictly look-ahead-safe: it serves only live windows and raises NoMarketDataError for historical (backtest) dates, letting the router fall back to yfinance which date-filters. Uses the core requests dependency (no new extra); ANYSEARCH_API_KEY is optional (anonymous works with lower limits). Not wired into any default config — activated only when selected as a vendor.
- CHANGELOG: add vendor provenance/CLI visibility, cross-source reflection basis disclosure, AnySearch live-only global-news vendor, and the schwab-py enum contract test + CI job. - README: refresh the price-basis note (divergence now disclosed, not silent), add a data-source visibility note, and document the AnySearch live-only vendor. - .env.example: document ANYSEARCH_API_KEY (optional, live-only). - Update test_memory_log to stub _price_basis_caveat and add a test that the cross-source basis caveat is appended for a non-yfinance price vendor. - gitignore local batch_analyze.py alongside run.sh/run_aapl.py.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
An opt-in, zero-behavior-change set of data-vendor and observability improvements. Defaults are unchanged (yfinance for prices/indicators, existing news vendors), so nothing is affected unless a user explicitly opts in.
This PR bundles five related pieces, all following the existing
route_to_vendor/VENDOR_METHODSthree-layer architecture:TRADINGAGENTS_DATA_VENDORenv var and an interactive CLI prompt.1. Schwab market-data vendor (opt-in)
Opt-in vendor for
core_stock_apis(OHLCV) andtechnical_indicators. Lazily imported, disabled by default. Enable via:^[A-Z]{1,5}$; anything else falls back to yfinance.client_from_token_file(refresh-only, never a browser login; safe headless).SCHWAB_APP_KEY/SCHWAB_APP_SECRET+ atoken.json(SCHWAB_TOKEN_PATH/SCHWAB_NATIVE_TOKEN_PATH, or default state locations). The token can come from any schwab-py login; TradingAgents implements no login flow of its own.schwabextra pinned toschwab-py>=1.5.1,<1.6; the import is deferred so a clean install without the extra never fails.2. AnySearch global-news vendor (opt-in, live-only)
New
anysearchvendor forget_global_newsbacked byapi.anysearch.com. AnySearch results carry no reliable publish date (verified against the live/v1/searchresponse), so the vendor is strictly look-ahead-safe: it serves only live windows and raises for historical/backtest dates, letting the router fall back to yfinance (which date-filters). Never leaks future news into a backtest. Uses the corerequestsdependency (no new extra);ANYSEARCH_API_KEYis optional (anonymous works with lower limits). Not wired into any default.3. Vendor provenance + CLI visibility
route_to_vendornow records which vendor actually served each method and which primaries it skipped, and logs the resolution at info level. The CLI prints:This exposes silent degrades (e.g. an unconfigured Schwab primary skipped in favor of yfinance) that previously left it unclear which source and Bedrock region/auth were in use.
4. Cross-source price-basis disclosure
Realized returns for reflection/alpha stay on yfinance split/dividend-adjusted prices (the correct outcome basis), but when the agent priced its decision on an unadjusted vendor (e.g. Schwab raw prices), the stored reflection now carries an explicit basis caveat and the verified snapshot discloses its yfinance-adjusted basis. This prevents a raw-vs-adjusted artifact on split/dividend tickers from silently poisoning the memory log or being mistaken for a data error. Default (yfinance) behavior is unchanged.
5. Data-vendor selection (env / CLI)
TRADINGAGENTS_DATA_VENDORenv var + an interactive CLI "Data Vendor" step (skipped with a✓ ... from environmentnotice when the var is set). Applied to bothcore_stock_apisandtechnical_indicators. Fully opt-in; invalid vendor names fail loudly at startup.Testing
tests/test_schwab_vendor.py(~35 tests + subtests, fully mocked — no real credentials).tests/test_schwab_enum_contract.py— pins the schwab-pyPriceHistoryenum wire-values the vendor sends (PeriodType.YEAR,Period.FIVE_YEARS,FrequencyType.DAILY,Frequency.DAILY);importorskipwhen schwab-py is absent.tests/test_anysearch_vendor.py— live-only guard, error classification, dedup, header/auth (mocked HTTP).tests/test_price_basis_caveat.py— basis caveat + snapshot disclosure.tests/test_vendor_routing.py— new provenance assertions (first-choice, fallback-with-reason, info log).tests/test_data_vendor_env.py, updatedtests/test_cli_env_skip.py, updatedtests/test_memory_log.py.[schwab]extra and runs the Schwab vendor + enum-contract tests, so schwab-py enum drift fails loudly instead of in production.pytest -q→ 648 passed, 1 skipped;ruff check .→ all checks passed. (Python 3.10–3.13 in CI.)Notes
## [Unreleased](Keep a Changelog format).# TODOand a known limitation. The frame is tz-naive and collapsed to a calendar day, so it only matters at cross-day boundaries.Made with Cursor