Repository navigation
fix(backtests): require sign-in and bill ATL Credits for hosted runtimes - #705
Merged
Merged
Conversation
POST /backtest/run only 401'd guests on the pipeline runtime. A guest (X-Session-Id alone) could clone the AI Hedge Fund agent and launch a run billed to the operator's OPENROUTER_API_KEY with no Credits reservation; the per-owner slot and rate-limit key were a session id the guest mints itself. Refuse a guest on any non-pipeline runtime, before the window bound and readiness checks, so a guest cannot learn whether a runtime is armed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Upstream calls its model inside its own interpreter, so ATL meters whole decisions: the route signs a platform-credits handoff (BYOK is a 422, a balance that cannot hold one decision a 402), and the child reserves a per-decision ceiling and settles at the token usage the bridge reports via langchain-core's get_usage_metadata_callback. Usage above the ceiling is capped, never recorded as overage. Unreported usage bills the ceiling on a completed or timed-out decision and is released on a crash before the usage scope opened. The existing finalize_run releases holds a killed child leaves, and the timeout card reads the spend like a pipeline run's. The Docker build now fails if the venv's langchain-core lacks the meter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A failed decision with no usage report was released, so an OOM SIGKILL after model calls (or an output write that raised outside the handler) made the decision free. The bridge now writes a started marker before upstream can call the model and its final output inside the try; the adapter bills the ceiling for any failure past the marker without a readable report, and for a clean exit whose output it cannot read. Only a bridge that left no file at all, so made no call, is released, and billing treats any outcome other than "failed" as billable. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Bill the ceiling on every escape from a decision, SystemExit from a cancel's SIGTERM included, instead of leaving the hold for finalize_run to release for free. - Meter calls, not just tokens: the bridge's own inheritable callback hook reports calls and unmetered_calls, so calls whose usage went missing bill the ceiling (ERROR, relayed live) and only zero calls is free. - Admit AI Hedge Fund runs through the shared Credits admission at their window ceiling, so they enter the in-flight ledger with pipeline runs. - Share the reserve-error mapping with LLMExecutionService. - Run the ownership check before the guest 401, so it cannot reveal that a stranger's agent is hosted. - Record the Credits lane on hosted runs and in the modal's launch config, and stop a guest's hosted launch client-side. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Six user-facing pages still said the hosted runtime needed no sign-in and spent no Credits. Add an "AI Hedge Fund runs" section to Credits & Billing: per-decision holds, full-hold charge when usage is unknown or a decision is cut short, and the whole-window launch check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
computedocean
pushed a commit
to computedocean/AgenticTrading
that referenced
this pull request
Oct 31, 2026
Open-Finance-Lab#704 and Open-Finance-Lab#705 each fit under the ceiling against their own base; merged together they put main at 40,004. Drop one redundant word (39,994). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Guests could start AI Hedge Fund backtests billed to the operator's
OPENROUTER_API_KEYwith no Credits reservation. This PR closes that and charges every AI Hedge Fund run to the caller's ATL Credits.Sign-in
POST /backtest/runrefuses a guest on any non-pipeline runtime with a 401. The check runs before the window bound and the readiness checks, so a guest learns nothing about how the runtime is configured.!= PIPELINE, so TradingAgents (TradingAgents Hosted Runtime Integration #676) is covered too.Credits (AI Hedge Fund)
billing_mode=byokwith a 422, because the model is platform-managed._admit_platform_creditsas a pipeline run, priced at its ceiling: one decision per trading session, each at the per-decision ceiling. The slot ledger therefore counts AI Hedge Fund and pipeline Credits runs against each other, and they can't each pass alone and then starve together. A short balance gets a 402, a restricted account a 403.(analysts + 1) × tickerscalls at 8k input / 2k output tokens. Reservations go throughreserve_platform_credits, which is now shared withLLMExecutionService.bridge.pymeters every model call with an inheritable langchain-core callback hook and reports tokens pluscallsandunmetered_calls. Counting calls is what separates "no model call" from "calls whose usage went missing".ERROR: ai_hedge_fund.usage_unmetered;ERROR: ai_hedge_fund.usage_unreported;startedmarker but no readable usage (an OOM SIGKILL, a crash mid-write);SystemExitthat a cancel's SIGTERM raises insidesubprocess.run.ERROR: ai_hedge_fund.lines to the service log live, likeERROR: llm.lines.INSUFFICIENT_CREDITS, same as a pipeline run.finalize_runreleases only a hold left by a SIGKILLed child. The bridge shares the child's process group, so it dies with the child.billing_mode/provider_id, and the modal's launch config saysplatform_credits, so the results panel shows "ATL Credits" instead of "—".app.jscache-buster is bumped tov=164.Verification
batchand aContextThreadPoolExecutorwere all counted on worker threads, a call withoutusage_metadatawas counted as unmetered, and calls outside the scope were not counted.main.Not in this PR: metering each model call at an ATL-side OpenAI-compatible endpoint backed by
LLMExecutionService. That would remove the ceiling heuristic and keepOPENROUTER_API_KEYout of the upstream process. It's a redesign and belongs in its own PR.Prod:
MAX_AI_HEDGE_FUND_TRADING_DAYS=0is still set on Render, so AI Hedge Fund answers 503. Set it back to 10 after this deploys.Docs: six
docs/source/lab/pages (accounts,credits_billing,getting_started,key_features,marketplace,operating_modes) now say AI Hedge Fund needs sign-in and bills ATL Credits.credits_billinggains an "AI Hedge Fund runs" section.🤖 Generated with Claude Code