Skip to content

docs: add The Grid custom LLM guide - #1194

Open
CastilloLuis wants to merge 1 commit into
VapiAI:mainfrom
the-gridai:thegridai/thegrid-custom-llm
Open

docs: add The Grid custom LLM guide#1194
CastilloLuis wants to merge 1 commit into
VapiAI:mainfrom
the-gridai:thegridai/thegrid-custom-llm

Conversation

@CastilloLuis

Copy link
Copy Markdown

What

Adds a docs page for The Grid as a custom-llm provider, following the existing guidance that "Vapi supports using any OpenAI-compatible endpoint as the LLM. This includes services like OpenRouter, AnyScale, Together AI".

Two files: the new page and one nav entry under Custom LLMs.

Why the maxTokens note

Most of the page is the standard credential + assistant JSON. The one thing I thought was worth documenting is a footgun I hit while testing:

Vapi's maxTokens defaults to 250, and on The Grid that budget is shared with reasoning tokens — which are counted and billed but never spoken. On one short prompt:

Instrument completion tokens of which reasoning finish_reason
text-standard 202 114 stop
agent-max 250 29 length (truncated)

So at the default, agent-max stops mid-sentence. The page recommends raising maxTokens and keeping the brevity instruction in the system prompt instead.

Verification

Everything in the page was tested against the live API using the OpenAI Node SDK with baseURL set the way Vapi sets it:

  • streaming works (stream: true, 246 chunks)
  • tool calling returns well-formed tool_calls
  • max_tokens is honored
  • GET /v1/models returns a spec-shaped {object: "list", data: [...]} with 17 instruments

I'm from The Grid, so treat the product framing as vendor-authored and edit it however you'd like — happy to cut the table or the tier list if it's more detail than these pages usually carry.

The Grid is an OpenAI-compatible inference marketplace. Document it as a
custom-llm provider alongside the existing OpenRouter/Together guidance,
and note that Vapi's default maxTokens of 250 is shared with reasoning
tokens on several instruments.
@lightsage-app

lightsage-app Bot commented Sep 5, 2026

Copy link
Copy Markdown

Lightsage docs evals

Waiting for the staging docs URL before running evals.

Lightsage will start the selected PR evals automatically when GitHub reports a successful docs deployment for this PR. This usually happens within 15 minutes.

Commit: e5bf8b7
Status: waiting for staging docs URL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants