Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions agents/build/custom-llm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -58,16 +58,16 @@
"parameters": {
"type": "object",
"properties": {
"order_number": { "type": "string", "description": "The order number, e.g. A12345." }

Check warning on line 61 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L61

Did you really mean 'order_number'?
},
"required": ["order_number"]
}
}
}
],
"session_id": "sess_01j9x4k2m8v3q7n5p6r8t9w0y1",

Check warning on line 68 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L68

Did you really mean 'session_id'?
"user_id": "user-42",

Check warning on line 69 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L69

Did you really mean 'user_id'?
"fishaudio_extra_body": { "chat_id": "chat-9" }

Check warning on line 70 in agents/build/custom-llm.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/build/custom-llm.mdx#L70

Did you really mean 'fishaudio_extra_body'?
}
```

Expand All @@ -91,7 +91,7 @@

Reply to that request with the spoken answer as ordinary `delta.content` chunks ending in `finish_reason: "stop"`.

Requests authenticate with `Authorization: Bearer <your API key>`. The endpoint must use `https` on a publicly reachable host, and must support function calling if the agent has tools configured.
Requests authenticate with `Authorization: Bearer <your API key>`. If your endpoint needs other headers, add them with [`headers`](#point-the-agent-at-your-endpoint). The endpoint must use `https` on a publicly reachable host, and must support function calling if the agent has tools configured.

## Point the agent at your endpoint

Expand All @@ -110,7 +110,7 @@
- Whether it accepted the tools.
- The time to first token.

The saved endpoint appears under **Custom** in the selector. You can edit or remove it there. If you pick a platform model again, the endpoint and the stored key are removed.
The saved endpoint appears under **Custom** in the selector. You can edit or remove it there. If you pick a platform model again, the endpoint, the stored key and any headers are removed.

Through the API, set the `llm.custom` section on the agent's [configuration](/agents/build/configuration#configure-through-the-api):

Expand All @@ -123,7 +123,10 @@
"custom": {
"base_url": "https://llm.example.com/v1",
"model": "persona-70b",
"api_key": "sk-your-endpoint-key"
"api_key": "sk-your-endpoint-key",
"headers": {
"your-header-key": "your-header-value"
}
}
}
}'
Expand All @@ -134,8 +137,11 @@
| `base_url` | `https` only, publicly reachable host, no query string or fragment. A trailing `/chat/completions` is stripped. |
| `model` | Sent to your endpoint verbatim as the request's `model` field. |
| `api_key` | Sent to your endpoint as the `Authorization` bearer token. The field is write-only, so config and version reads return it as `null`. Omit it (or send `null`) on later patches to keep the stored key. |
| `headers` | Optional. Extra headers sent on every request to your endpoint. Names are case-insensitive, and values are printable ASCII. You can't set headers the platform sends itself, such as `Authorization`, `Content-Type` or `Host`. Write-only like `api_key`: reads return `null`. Sending it replaces all stored headers, so include every header you want to keep. Omit it (or send `null`) to keep them; send `{}` to clear them. |

The console doesn't show or edit headers, so set them through the API. Saving the endpoint in the console keeps them, and **Run test** sends them while the base URL matches the saved one.

Once [published](/agents/deploy/versions-publishing), every session of the agent generates on your endpoint, whether it starts from the web, a phone call, or a [preview call](/agents/test/preview-calls) in the Builder. Rotating the key works the same way as any other config change. Patch a new `api_key` into the draft, then publish; sessions started after the publish use the new key. Send `{ "llm": { "custom": null } }` to switch back to platform models.
Once [published](/agents/deploy/versions-publishing), every session of the agent generates on your endpoint, whether it starts from the web, a phone call, or a [preview call](/agents/test/preview-calls) in the Builder. Rotating the key or headers works the same way as any other config change. Patch the new values into the draft, then publish; sessions started after the publish use them. Send `{ "llm": { "custom": null } }` to switch back to platform models.

## Failure behavior

Expand Down
4 changes: 2 additions & 2 deletions agents/deploy/versions-publishing.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@
</Step>
<Step title="Publish">
Click **Publish**. Any pending edits are saved first, then the draft is
snapshotted as the next version. A confirmation shows the new version

Check warning on line 64 in agents/deploy/versions-publishing.mdx

View check run for this annotation

Mintlify / Mintlify Validation (hanabiaiinc) - vale-spellcheck

agents/deploy/versions-publishing.mdx#L64

Did you really mean 'snapshotted'?
number, and the agent shows as **Live** in your agents list.
</Step>
</Steps>
Expand Down Expand Up @@ -120,8 +120,8 @@
Version snapshots never echo credential secrets. Write-only fields (such as
webhook secrets and tool authorization headers) return a `has_secret: true`
marker instead of the value, and the custom LLM
[`api_key`](/agents/build/custom-llm) reads back as `null`, in every version
including historical ones.
[`api_key` and `headers`](/agents/build/custom-llm) read back as `null`, in
every version including historical ones.
</Warning>

## Restore a previous version
Expand Down
Loading