Skip to content

Add opt-in Schwab + AnySearch data vendors, vendor provenance, and cross-source basis disclosure - #1207

Open
kevinkda wants to merge 7 commits into
TauricResearch:mainfrom
KevinKDA-Resources:main
Open

Add opt-in Schwab + AnySearch data vendors, vendor provenance, and cross-source basis disclosure#1207
kevinkda wants to merge 7 commits into
TauricResearch:mainfrom
KevinKDA-Resources:main

Conversation

@kevinkda

@kevinkda kevinkda commented Aug 6, 2026

Copy link
Copy Markdown

Summary

An opt-in, zero-behavior-change set of data-vendor and observability improvements. Defaults are unchanged (yfinance for prices/indicators, existing news vendors), so nothing is affected unless a user explicitly opts in.

This PR bundles five related pieces, all following the existing route_to_vendor / VENDOR_METHODS three-layer architecture:

  1. Schwab market-data vendor (OHLCV + technical indicators, US equities/ETFs).
  2. AnySearch global-news vendor (live-only supplement, look-ahead-safe).
  3. Vendor provenance + CLI visibility (which vendor actually served; startup config summary).
  4. Cross-source price-basis disclosure in reflection and the verified snapshot.
  5. Data-vendor selection via TRADINGAGENTS_DATA_VENDOR env var and an interactive CLI prompt.

1. Schwab market-data vendor (opt-in)

Opt-in vendor for core_stock_apis (OHLCV) and technical_indicators. Lazily imported, disabled by default. Enable via:

config["data_vendors"]["core_stock_apis"] = "schwab,yfinance"
config["data_vendors"]["technical_indicators"] = "schwab,yfinance"
  • US equities / ETFs only. Symbols whitelisted to ^[A-Z]{1,5}$; anything else falls back to yfinance.
  • Read-only. No orders, no account access — OHLCV + indicators only.
  • Auth: schwab-py's client_from_token_file (refresh-only, never a browser login; safe headless). SCHWAB_APP_KEY / SCHWAB_APP_SECRET + a token.json (SCHWAB_TOKEN_PATH / SCHWAB_NATIVE_TOKEN_PATH, or default state locations). The token can come from any schwab-py login; TradingAgents implements no login flow of its own.
  • Dependency: new optional schwab extra pinned to schwab-py>=1.5.1,<1.6; the import is deferred so a clean install without the extra never fails.

2. AnySearch global-news vendor (opt-in, live-only)

New anysearch vendor for get_global_news backed by api.anysearch.com. AnySearch results carry no reliable publish date (verified against the live /v1/search response), so the vendor is strictly look-ahead-safe: it serves only live windows and raises for historical/backtest dates, letting the router fall back to yfinance (which date-filters). Never leaks future news into a backtest. Uses the core requests dependency (no new extra); ANYSEARCH_API_KEY is optional (anonymous works with lower limits). Not wired into any default.

3. Vendor provenance + CLI visibility

route_to_vendor now records which vendor actually served each method and which primaries it skipped, and logs the resolution at info level. The CLI prints:

  • an "Effective configuration" panel at startup (LLM provider/models/backend, per-category data vendor, and — for Bedrock — the resolved region + auth mode: bearer token vs AWS profile), and
  • a "Data source" panel after the run showing the serving vendor and any fallback reason.

This exposes silent degrades (e.g. an unconfigured Schwab primary skipped in favor of yfinance) that previously left it unclear which source and Bedrock region/auth were in use.

4. Cross-source price-basis disclosure

Realized returns for reflection/alpha stay on yfinance split/dividend-adjusted prices (the correct outcome basis), but when the agent priced its decision on an unadjusted vendor (e.g. Schwab raw prices), the stored reflection now carries an explicit basis caveat and the verified snapshot discloses its yfinance-adjusted basis. This prevents a raw-vs-adjusted artifact on split/dividend tickers from silently poisoning the memory log or being mistaken for a data error. Default (yfinance) behavior is unchanged.

5. Data-vendor selection (env / CLI)

TRADINGAGENTS_DATA_VENDOR env var + an interactive CLI "Data Vendor" step (skipped with a ✓ ... from environment notice when the var is set). Applied to both core_stock_apis and technical_indicators. Fully opt-in; invalid vendor names fail loudly at startup.


Testing

  • tests/test_schwab_vendor.py (~35 tests + subtests, fully mocked — no real credentials).
  • tests/test_schwab_enum_contract.py — pins the schwab-py PriceHistory enum wire-values the vendor sends (PeriodType.YEAR, Period.FIVE_YEARS, FrequencyType.DAILY, Frequency.DAILY); importorskip when schwab-py is absent.
  • tests/test_anysearch_vendor.py — live-only guard, error classification, dedup, header/auth (mocked HTTP).
  • tests/test_price_basis_caveat.py — basis caveat + snapshot disclosure.
  • tests/test_vendor_routing.py — new provenance assertions (first-choice, fallback-with-reason, info log).
  • tests/test_data_vendor_env.py, updated tests/test_cli_env_skip.py, updated tests/test_memory_log.py.
  • New CI job installs the [schwab] extra and runs the Schwab vendor + enum-contract tests, so schwab-py enum drift fails loudly instead of in production.
  • Locally: full pytest -q648 passed, 1 skipped; ruff check . → all checks passed. (Python 3.10–3.13 in CI.)

Notes

  • Version intentionally not bumped — release cadence is the maintainers' call. Changelog entries are under ## [Unreleased] (Keep a Changelog format).
  • Candle timezone semantics not live-verified for Schwab (epoch-ms day-boundary ET vs UTC); flagged as a # TODO and a known limitation. The frame is tz-naive and collapsed to a calendar day, so it only matters at cross-day boundaries.

Made with Cursor

Add a Schwab-backed OHLCV and technical-indicator vendor for US
equities/ETFs, opt-in behind the ``schwab`` extra. It consumes a
token.json produced by any schwab-py login (no OAuth login of its own)
via a synchronous schwab-py client_from_token_file client, restricts to
plain ^[A-Z]{1,5}$ symbols, classifies on HTTP status without
raise_for_status, disk-caches a 5-year daily window, and falls back to
yfinance on NotConfigured/NoMarketData. schwab-py is imported lazily so
a clean install without the extra never fails at import time.

The vendor follows the existing route_to_vendor / VENDOR_METHODS three
-layer architecture (same shape as the alpha_vantage vendor). Defaults
are unchanged (yfinance); enable by setting data_vendors for both
core_stock_apis and technical_indicators to "schwab,yfinance".

Also apply the low-cost fixes surfaced by the double-blind review:
- indicator path now runs the same stale-frame guard
  (_assert_ohlcv_not_stale) as the OHLCV path, after look-ahead trimming
  and before cleaning, so year-old frames never feed indicators;
- get_price_history passes an explicit frequency=Frequency.DAILY (wire
  value 1) instead of relying on a server default;
- the same-day cache refresh restores the "historical request reuses the
  cache forever, unaffected by TTL" short-circuit (only current-day
  requests honour the TTL), matching the yfinance cache;
- minor: copy the trimmed frame before rounding to avoid
  SettingWithCopyWarning, validate the title-case OHLCV column set on a
  cache hit, and drop the previously-unused today_date parameter.

Version is intentionally not bumped (release cadence is the maintainers'
call); the changelog entry sits under [Unreleased].

Follow-ups deferred (not blocking): tighten the parity-test naming
(G2), the bare-BTC whitelist edge (G3), the tz day-boundary semantics
TODO (needs a live Schwab credential), and a wire-value assertion
against the real schwab-py enums (needs schwab-py in the test env).
Add a TRADINGAGENTS_DATA_VENDOR environment variable plus an interactive
CLI prompt to choose the market-data source. The selection applies to
both core_stock_apis and technical_indicators categories together, so
vendors such as Schwab or Alpha Vantage can be picked at the CLI/env
layer. The behavior is opt-in and defaults to yfinance, leaving the
existing flow unchanged when the variable is unset.
route_to_vendor now records which vendor actually served each method and
which primaries were skipped, logs the resolution at info level, and the
CLI prints an "Effective configuration" panel at start plus a "Data source"
panel at the end. This exposes silent degrades (e.g. an unconfigured Schwab
primary skipped in favor of yfinance) that previously left users unsure which
data source and Bedrock region/auth mode were actually in use (TauricResearch#988/TauricResearch#977/TauricResearch#1103).

Also gitignore local convenience scripts (run.sh, run_aapl.py).
Realized returns for reflection/alpha are measured on yfinance
split/dividend-adjusted closes (the correct outcome basis), but the agent may
have priced its decision on an unadjusted vendor (e.g. Schwab raw prices).
On a ticker with a split/large dividend inside the holding window, the two
bases diverge, so an adjustment artifact could be written to the memory log
and re-injected into future prompts as if it were a real lesson.

Append a deterministic basis caveat to the stored reflection when the core
price vendor is not yfinance, and disclose the yfinance-adjusted basis in the
verified snapshot so a raw-vs-adjusted level gap is not mistaken for a data
error. Keeps yfinance as the returns basis (no behavioral change for the
default config).
The Schwab vendor sends an explicit 5-year daily window via PeriodType.YEAR /
Period.FIVE_YEARS / FrequencyType.DAILY / Frequency.DAILY. A schwab-py upgrade
that renamed a member or changed a wire value would silently send the wrong
frequency and return the wrong candles. Add a contract test pinning those
enums against the installed schwab-py (importorskip when absent) and a CI job
that installs the [schwab] extra and runs the vendor + contract tests, so the
drift fails loudly instead of in production.
AnySearch (api.anysearch.com /v1/search) provides real-time web/news search.
Add it as an opt-in supplement for get_global_news only. AnySearch results
carry no reliable publish date (verified against the live response), so the
vendor is strictly look-ahead-safe: it serves only live windows and raises
NoMarketDataError for historical (backtest) dates, letting the router fall
back to yfinance which date-filters. Uses the core requests dependency (no new
extra); ANYSEARCH_API_KEY is optional (anonymous works with lower limits).
Not wired into any default config — activated only when selected as a vendor.
- CHANGELOG: add vendor provenance/CLI visibility, cross-source reflection
  basis disclosure, AnySearch live-only global-news vendor, and the schwab-py
  enum contract test + CI job.
- README: refresh the price-basis note (divergence now disclosed, not silent),
  add a data-source visibility note, and document the AnySearch live-only vendor.
- .env.example: document ANYSEARCH_API_KEY (optional, live-only).
- Update test_memory_log to stub _price_basis_caveat and add a test that the
  cross-source basis caveat is appended for a non-yfinance price vendor.
- gitignore local batch_analyze.py alongside run.sh/run_aapl.py.
@kevinkda kevinkda changed the title Add opt-in Schwab market-data vendor (OHLCV + technical indicators) Add opt-in Schwab + AnySearch data vendors, vendor provenance, and cross-source basis disclosure Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant