Skip to content

Set up an MCP server from a one-sentence request - #1612

Merged
DoodleBears merged 4 commits into
mainfrom
claude/mcp-setup-guidance
Oct 13, 2026
Merged

DoodleBears merged 4 commits into
mainfrom
claude/mcp-setup-guidance

Conversation

@DoodleBears

@DoodleBears DoodleBears commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Requested by Vain

Why the change

Asked in one sentence to set up an MCP server, a Bot did not use the secret card and the connect tool. It looked through Profile configuration with Shell, proposed editing DSH configuration, asked clarifying questions, or asked the Human to send the key in the chat.

Closes #1611

Special things to note

  • Merge risk: prompt only. One paragraph added to the Orchestrator role prompt, about 120 words on every Orchestrator turn. Revert by reverting this PR.
  • The paragraph says: for a request to set up, connect or configure an MCP server or a service that needs a credential, do it in the conversation with the secret tools; do not inspect or edit Profile or DSH configuration; never ask for a secret to be pasted into a Channel; call ask_secret at once, then secret_mcp_connect with the server's usual launcher; ask first only when the server is unclear; then list the tools. It also points to secret_http and secret_cli.
  • Found while measuring: writing a literal placeholder in double braces in the role prompt makes every Session fail with unknown prompt variable, because DSH treats it as a template variable. The paragraph avoids it and the prompt test now fails on any double brace other than the registered variables.
  • The Bot chooses the package from its own knowledge. In eight runs it chose the official reference package seven times and a third-party package once (@cablate/mcp-google-map for Google Maps). The start approval card shows the exact package, which is where the Human checks it.
  • Verification:
    • Automated: dsh-bot-agent-adapter.test.ts asserts the guidance is present and that the role prompt holds no stray template braces; 29 tests in that file pass. pnpm lint, pnpm format:check, pnpm typecheck pass. Other suites were not run locally; CI runs them.
    • Runtime, real model, isolated Profile on DSH 0.2.0-rc.2. Each run is a newly created Bot that receives one message, "我想配置一下 的 MCP。", for Brave Search, GitHub, Google Maps and Slack. A script submits synthetic values to any secret card and allows any start once.
      • Before (main): 0 of 4 runs showed a secret card; 0 of 4 called secret_mcp_connect.
      • After: 8 of 8 runs showed a secret card with fields named for the service, and 8 of 8 called secret_mcp_connect with an npx -y <package> launcher. The servers started and the Bots listed their tools; with synthetic keys the services then rejected real calls, as expected.
    • Not verified: other models than the one configured in the test Profile; requests in a Group; services whose server is started with uvx; a request for a service the model does not know.
    • Human QA: not done.

Change outline

"我想配置一下 X 的 MCP。"
before: Shell through Profile config / questions / "send me the key here"      0/4 reach the card
after:  ask_secret card -> secret_mcp_connect approval -> tools listed          8/8

No screen was changed; these show the flow a newly created Bot now follows after the single message "我想配置一下 Brave Search 的 MCP。", with a synthetic key.

  1. The secret card appears straight after the message.

Secret card after one message

  1. After the card is submitted, the start approval shows the package and arguments the Bot chose.

Start approval card

  1. After approval the server is connected and its tools are listed; the synthetic key is rejected by the real service, which the Bot reports.

Server connected, tools listed

These were taken on this branch, so the start card still shows the earlier wording of the remember button.

🤖 Generated with Claude Code

DoodleBears and others added 3 commits October 14, 2026 03:52
Agent-Task: claude-code/local/9f51f103-7177-4bd0-8fd7-81d1ce39b98f

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Agent-Task: claude-code/local/9f51f103-7177-4bd0-8fd7-81d1ce39b98f

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Issue: #1611
Agent-Task: claude-code/local/9f51f103-7177-4bd0-8fd7-81d1ce39b98f

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DoodleBears DoodleBears self-assigned this Oct 13, 2026
@coderabbitai

coderabbitai Bot commented Oct 13, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Issue: #1611
Agent-Task: claude-code/local/9f51f103-7177-4bd0-8fd7-81d1ce39b98f

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DoodleBears
DoodleBears marked this pull request as ready for review October 13, 2026 20:16
@DoodleBears
DoodleBears merged commit fd80c2e into main Oct 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A one-sentence request to set up an MCP server does not reach the secret card

1 participant