feat(budget): scope budgets to a provider by SantiagoDePolonia · Pull Request #1212 · ENTERPILOT/GoModel

SantiagoDePolonia · 2026-10-20T01:36:52Z

Adds a provider budget scope, mirroring provider rate limits. Refs #1202.

budgets:
  providers:
    - name: openai-prod
      limits:
        - period: daily
          amount: 100

Also SET_PROVIDER_BUDGET_<NAME>, "scope": "provider" in the admin API, and a Provider option in the dashboard.

Caps all spend routed to that provider instance, across every user path and label. It does not narrow a user-path budget (the per-customer case in Feature: Scope a budget to a provider #1202 still needs a filter field).
Matched case-insensitively against provider_name, falling back to provider for older rows. No schema change.
An exhausted provider budget defers to failover when targets exist; failover skips targets whose provider budget is spent. User-path and label breaches still reject and are reported first.
Not applied to batch submissions or MCP calls (provider unknown at admission).
Model scope left out: usage rows record the provider-returned model (e.g. dated snapshots), so a model budget would silently under-count.

Summary by CodeRabbit

New Features
- Added provider-specific budgets with hourly, daily, weekly, and monthly limits.
- Provider budgets can be configured in YAML or through environment variables, and managed from the dashboard and admin API.
- Provider names are normalized and validated consistently across configuration and management tools.
- Failover skips providers that have reached their budgets and uses other eligible providers when available.
Documentation
- Added configuration examples and guidance for provider budgets and environment-variable setup.
- Added dashboard translations for provider budget labels and validation messages.

mintlify · 2026-10-20T01:36:56Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated
gomodel	🟢 Ready	View Preview	Oct 20, 2026, 1:37 AM

💡 Tip: Enable Automations to automatically generate PRs for you.

greptile-apps · 2026-10-20T01:36:58Z

TREX is testing this pull request...

Plan what to test
Set up the environment and test the changed flows
Post test results

coderabbitai · 2026-10-20T01:37:13Z

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Advanced
Run ID: 6f1b667f-c0cd-461f-81ea-e01476d0bfdb

📥 Commits

Reviewing files that changed from the base of the PR and between afdfb49 and 2bbd01a.

📒 Files selected for processing: 2

internal/gateway/failover.go
internal/gateway/failover_test.go

Included review availability: This review used your included allowance. Your plan provides up to 4 included reviews per hour; 2 remain after this review.

📝 Walkthrough

This change adds provider-instance budgets, provider spend matching, routed-provider admission checks, failover skipping for exhausted providers, and admin and dashboard support.

Changes: Provider Budget Flow

Layer / File(s)	Summary
Define and configure provider budgets `internal/budget/types.go`, `config/budget.go`, `config/config_test.go`, `config/*.yaml`, `.env.template`, `docs/advanced/configuration.mdx`	Adds the `provider` scope. YAML and `SET_PROVIDER_BUDGET_<NAME>` values load normalized provider limits and validate provider names.
Match provider spend and evaluate limits `internal/budget/factory.go`, `internal/budget/service.go`, `internal/budget/store_.go`, `internal/budget/_test.go`	Creates provider budgets and matches spend by case-insensitive provider name, with provider-type fallback for missing names.
Pass provider identity through request admission `internal/server/*`, `internal/core/context.go`	Passes routed provider information into budget checks. Provider-budget failures defer when failover targets exist; user-path and label failures remain rejections.
Check provider budgets during failover `internal/gateway/*`, `internal/server/translated_inference_service.go`	Wires an optional budget gate into inference orchestration. Response and streaming failover skip exhausted providers.
Expose provider budgets in admin and dashboard `internal/admin/`, `web/dashboard/`, `docs/features/budgets.mdx`	Adds provider-scope API handling, dashboard validation and payload support, localized labels, and provider-budget documentation.

Priority: ➖ Normal

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant Request
  participant BudgetAdmission
  participant BudgetService
  participant SpendStore
  participant Failover
  Request->>BudgetAdmission: Submit request with routed provider
  BudgetAdmission->>BudgetService: Check provider and consumer budgets
  BudgetService->>SpendStore: Read provider spend
  SpendStore-->>BudgetService: Return spend
  BudgetService-->>BudgetAdmission: Return budget result
  BudgetAdmission->>Failover: Defer provider breach when failover exists
  Failover->>BudgetService: Check next provider
  Failover-->>Request: Skip exhausted provider or attempt target

Merge Risk: ⚪ Minimal · up to 2bbd0

This change makes failover skip providers whose budgets are exhausted, and leaves failover unchanged when budgets are disabled. No concrete merge-blocking risk was identified in the reviewed failover code and tests.

Pre-merge checks: 4 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Docstring Coverage		Docstring coverage is 52.11% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 71 functions across 29 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

Passed checks: 4 passed

Check name	Status	Explanation
Title check		The title clearly and concisely describes the main change: adding provider-scoped budgets.
Description check		The description explains what changed, why it changed, configuration and API additions, matching behavior, failover behavior, exclusions, and known limitations. It is mostly complete, although it does…
Linked Issues check		Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check		Check skipped because no linked issues were found for this pull request.

Full details: Description check

Explanation

The description explains what changed, why it changed, configuration and API additions, matching behavior, failover behavior, exclusions, and known limitations. It is mostly complete, although it does not use the template's explicit ## Description heading.

Fix all pre-merge checks with AI

✨ Finishing Touches: 💡 1

📝 Generate docstrings: 💡

Commit to this branch
Create a new PR

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{A rabbit checks the provider gate,
While budgets guard each model’s fate.
Exhausted paths hop out of line,
And fallbacks seek a limit fine.
Config blooms with names made clear,
The dashboard smiles from ear to ear.}

_{Comment @coderabbitai help to get the list of available commands.}

greptile-apps · 2026-10-20T01:40:44Z

Confidence Score: 4/5

[Medium impact] The change affects budget checks and routing across routed inference paths.

Fix the failover checks before merging so workflows that disable budgets can still use healthy backup providers.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Request with resolved provider] --> B[Check consumer and provider budgets]
  B --> C{Exhausted budget?}
  C -->|Consumer budget| D[Reject request]
  C -->|None| E[Call primary provider]
  C -->|Provider budget and failover targets| F[Skip primary provider]
  E -->|Retryable failure| G[Check failover targets]
  F --> G
  G --> H{Provider budget available?}
  H -->|Yes| I[Call target]
  H -->|No| J[Skip target]

_{Reviews (1) · Last reviewed commit: "feat(budget): scope budgets to a provide..." · Reviewed by Greptile}

greptile-apps · 2026-10-20T01:40:48Z

+		if o.budgetGate != nil && !o.budgetGate.ProviderWithinBudget(ctx, providerName) {
+			slog.Info("skipping over-budget failover target",
+				"request_id", requestID,
+				"to", qualified,
+				"provider", providerName,
+			)
+			continue


Disabled budgets still block backups

When budgets are enabled globally but disabled for a workflow, admission skips budget checks. Both failover sweeps still call ProviderWithinBudget unconditionally. If the primary fails and the only healthy target has an exhausted provider budget, the request returns the primary error instead of using that target.

Honor workflow.BudgetEnabled() in both failover sweeps.

Fixed in 2bbd01a: both failover sweeps now go through providerOverBudget, which skips the gate when workflow.BudgetEnabled() is false, matching admission. Covered by TestTryFailoverIgnoresBudgetsWhenWorkflowDisablesThem.

codecov-commenter · 2026-10-20T01:40:54Z

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 91.42857% with 12 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/budget/factory.go	33.33%	4 Missing ⚠️
config/budget.go	85.71%	3 Missing ⚠️
internal/budget/service.go	96.29%	1 Missing ⚠️
internal/budget/store_mongodb.go	92.85%	1 Missing ⚠️
internal/budget/store_sql.go	80.00%	1 Missing ⚠️
internal/budget/types.go	94.44%	1 Missing ⚠️
internal/server/translated_inference_service.go	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

… budgets

feat(budget): scope budgets to a provider

afdfb49

mintlify Bot deployed to staging - docs October 20, 2026 01:37

View deployment

github-actions Bot added the size:L 100-499 effective changed lines (test files excluded in mixed PRs). label Oct 20, 2026

greptile-apps Bot reviewed Oct 20, 2026

View reviewed changes

fix(budget): skip the failover budget gate when the workflow disables…

2bbd01a

… budgets

SantiagoDePolonia merged commit 4e9b5e8 into main Oct 20, 2026
24 checks passed

SantiagoDePolonia mentioned this pull request Oct 20, 2026

Feature: Budget re-check on failover #1201

Open

Navigation Menu

Repository navigation

feat(budget): scope budgets to a provider - #1212

feat(budget): scope budgets to a provider#1212
SantiagoDePolonia merged 2 commits into
mainfrom
feat/provider-scope-budget

SantiagoDePolonia commented Oct 20, 2026 •

edited by coderabbitai Bot

Loading

Uh oh!

mintlify Bot commented Oct 20, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Oct 20, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 20, 2026 •

edited

Loading

Sequence Diagram(s)

Uh oh!

greptile-apps Bot commented Oct 20, 2026 •

edited

Loading

Diagram

Uh oh!

greptile-apps Bot Oct 20, 2026

Uh oh!

SantiagoDePolonia Oct 20, 2026

Uh oh!

codecov-commenter commented Oct 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

SantiagoDePolonia commented Oct 20, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary by CodeRabbit

Uh oh!

mintlify Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TREX is testing this pull request...

Uh oh!

coderabbitai Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sequence Diagram(s)

Uh oh!

greptile-apps Bot commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 4/5

Diagram

Uh oh!

greptile-apps Bot Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

SantiagoDePolonia Oct 20, 2026

Choose a reason for hiding this comment

Uh oh!

codecov-commenter commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SantiagoDePolonia commented Oct 20, 2026 •

edited by coderabbitai Bot

Loading

mintlify Bot commented Oct 20, 2026 •

edited

Loading

greptile-apps Bot commented Oct 20, 2026 •

edited

Loading

coderabbitai Bot commented Oct 20, 2026 •

edited

Loading

greptile-apps Bot commented Oct 20, 2026 •

edited

Loading

codecov-commenter commented Oct 20, 2026 •

edited

Loading