Introduce semantic hints for side-effects of tools

@victorhuangwq

Forking from #45 (comment). MCP already has a concept of annotations to provide deterministic hints of the side-effects a tool execution can have. These are helpful for Agents to decide whether a tool should be used in a specific context. And for users to express preferences of the type of actions they want to allow an Agent to take. As an example, readOnlyHint could be used if the user wants an Agent to be in "conversational mode" only.

Thanks @victorhuangwq for bringing this up!

MCP annotation spec for reference

Annotations for WebMCP are much more exciting than they are for traditional MCP IMO. The ability to give annotations enforceable semantics in a web standard is something I wish we had in MCP. A third party (UA) enforcing middleware on tool calls is something unique to WebMCP and is something we should definitely explore.

For example, the browser could allow concurrent execution of tools marked as readOnly while serializing those marked destructive.

The current spec of MCP is largely focused on the server side (tool provider) with almost no opinions enforced on the client side (tool caller). Annotations are 100% up to the client to handle (or not handle).

One thing I would clarify here, it’s not just about adding semantic hints. What would make hints useful is having a standardized set of hints with well-defined, agreed-upon meanings, so they’re precise and not open-ended.

Hi! Bringing this over from #119 as suggested.
I wanted to raise that WebMCP currently only defines one tool annotation (readOnlyHint), while MCP defines four. Three are missing:

destructiveHint — Without it, the browser can't distinguish a safe tool from a catastrophic one (e.g. editDesign() vs deleteAccount()), and has no basis to automatically prompt the user for confirmation.
idempotentHint — Without it, if a tab freezes or times out mid tool-call, the agent can't safely decide whether to retry — risking either a missed action or a duplicate execution (e.g. placing two orders).
openWorldHint — Without it, the browser can't tell whether a tool sends user data to external services, and can't warn the user or treat external results with appropriate suspicion.

Since WebMCP is explicitly designed to align with MCP, I think it makes sense to consider bringing these three annotations into the spec

DestructiveHint does sound reasonable - this allows agent developers to understand that this is a non-reversible change to its environment, so can appropriately notify users.

IdempotentHint, based on the mcp annotations spec:

idempotentHint?: boolean
If true, calling the tool repeatedly with the same arguments will have no additional effect on the its environment.

not sure if that matches up with what you are describing? idempotent tools probably are referring to tools that you can perhaps call in parallel etc. I would think. Could you clarify your use case?

I'm a little unsure about openworldhints, because even in the MCP tool annotation definition, it's a little bit vague:

openWorldHint?: boolean
If true, this tool may interact with an “open world” of external entities. If false, the tool’s domain of interaction is closed. For example, the world of a web search tool is open, whereas that of a memory tool is not.

idempotentHint?: boolean
If true, calling the tool repeatedly with the same arguments will have no additional effect on the its environment.

So, if a buy-item tool with the same argument (order number) won't order the item twice if already ordered, then the LLM is free to try again in case something went wrong, like the user closed the tab while ordering and the tool response was never received.

That does sound useful to me. Though the name is probably not good for non native English speakers. Another suggestion could be "retrySafetyHint"

I ran into another issue with some clients removing duplicate calls to tools with same argument, which is an issue for something like https://webmcp-maze.bandarra.me/ where you have to 'move' tool with a direction. So, if I say "move east, 3 times", the LLM might call the tool with the same argument 3 times and the second and third time might get filtered out. This is kind of the opposite of "idempotentHint" (retrySafetyHint), so we might need something like that as well (repeatSafetyHint/repeatableHint)

Relevant real-world implementation: onUI is a free, open-source browser extension that exposes 8 MCP tools for browser annotation operations (get, search, update, delete, clear annotations, list pages, get reports). It could serve as a practical reference for how semantic hints and side-effect annotations apply to browser UI tools.

Coming back to this one with some experience from using the hints to decide when to ask the user. A lot has been added to the spec since the thread started, so I'd first suggest re-scoping it.

consequentialHint now covers much of what destructiveHint was for (significant, real-world or non-reversible), and untrustedContentHint covers the output side. What I think the thread still asks for is what a hint means when it is absent, a definition of openWorldHint that people can agree on, and retry safety.

On absent hints: MCP gives them pessimistic defaults (not read-only, may be destructive, not idempotent, open-world; see Tool annotations as a risk vocabulary). In the WebMCP IDL every hint defaults to false, so once the dictionary is converted a tool that leaves a hint out can't be told apart from one that set it to false. For readOnlyHint that is the cautious reading, but for consequentialHint and untrustedContentHint it is the permissive one. It matters in practice: we shipped a bug where a tool that didn't declare readOnlyHint was treated as having declared false, and our approval logic depends on the difference between absent, false and true. A note in the spec on what silence means would help (#355 has a draft).

For openWorldHint, victorhuangwq found the MCP wording vague, and I agree. One definition that is concrete enough to act on: the tool may send data to, or read data from, a party other than the page's own origin. Before the call it decides whether the arguments are checked for secrets and personal data because they might leave the origin. After the call it signals that the result may carry content from a third party, so it's treated as less trusted.

For retry safety, I suggested earlier renaming idempotentHint to something like retrySafetyHint. Hosts give up on quiet calls. Two MCP clients we tested closed a call after 60 s and 180 s, and one of them then retried it. A tool that places an order must be able to say that a repeat with the same arguments is safe, or the host has to guess (more on timeouts and progress in #196).

One caution about enforcement. MiguelsPizza suggested that a browser could enforce hints, for example running read-only tools concurrently and serialising destructive ones. Hints are written by the page, so they can be wrong or hostile. Behaviour that only adds caution is safe to base on a hint: serialising, asking the user, scanning arguments. Behaviour that removes caution (running in parallel because a tool says it's read-only) turns a false hint into a race, so I wouldn't base enforcement on a hint in that direction. That asymmetry is worth one sentence in the spec.

A correction to my comment above. I wrote that the WebMCP hints are listed without defaults. That is wrong: the IDL gives each of them a default of false (boolean consequentialHint = false; and so on), and the annotations struct starts with the same values.

The point I was making still stands, in a sharper form.

Because of the dictionary defaults, an absent hint and an explicit false are the same thing from the moment the dictionary is converted. An agent cannot tell "the page said this tool is not consequential" from "the page said nothing".
For readOnlyHint a default of false is the cautious reading (the tool might write). For consequentialHint and untrustedContentHint it is the permissive one: a tool that declares nothing is reported as not consequential and its output as not untrusted. MCP's defaults go the other way for the hints that raise scrutiny (a tool is assumed to be possibly destructive and open-world until it says otherwise).

I'm preparing a small PR with wording for this and for openWorldHint, and will link it here.

The PR is up as a draft: #355. The first commit adds a note that a hint a tool leaves out is not an assurance, and the second adds openWorldHint (defaulting to true, as in MCP). They can be taken separately.

@kenchris

Thanks for the PR @kenchris! I like the idea of adding the openWorldHint. However, I'm wondering if adding an explanation for the defaults on hints is a good idea, or if it would be better to push back on the current defaults and instead move to defaults that are consistently the most dangerous option (ex: consequentialHint default to true.), or take away the defaults and make it so hints can be empty. Personally, I am leaning towards removing defaults because it gives the most information possible to the agents that will pick up the hints later.

Let me know what you think. It would probably be easiest to land your addition of openWorldHint in its own PR while the other threads are discussed.

@jpagnucco

I'd take the first option and remove the defaults. If every hint defaults to false, a host/agent can't tell "the page said this tool is not consequential" from "the page said nothing". With no default, absent means unknown and the host decides what to do about it.

We shipped a bug exactly there: a tool that hadn't declared readOnlyHint was treated as having declared false. Our approval logic depends on absent, false and true being three different things, and a tool that declares itself consequential always asks the user.

I wouldn't make consequentialHint default to true. Every tool would look dangerous by default, so hosts would learn to ignore the flag, which would be bad. A default also isn't something the page claimed, and a hint is only worth acting on when it is.

Removing the defaults also settles the openWorldHint default I proposed in #355 (true, like MCP). There would be none, and absent would mean unknown there, too.

Landing openWorldHint on its own is fine with me. I'll cut #355 down to that commit. Your question @jpagnucco about whether it's needed isstill open there.

@kenchris

Coming back to openWorldHint -

@kenchris you mentioned that consequential hint + untrusted hint does not cover "a read-only search tool that sends the query to a third party API" in PR #355 as a motivation for needing openWorldHint.

Can you give an example tool call that might benefit from this just to anchor this discussion? I'm curious if this is just a case of ensuring tools are well-formed, i.e. submitData(secretToken) vs. search(secretToken)

Is it possible to collect model vendor feedback on openWorldHint to help us understand use cases?

Also if anyone from privacy wants to chime in on this, that'd be great (maybe @npdoty ?)

Some data on openWorldHint, since vendors came up.

OpenAI asks for readOnlyHint, destructiveHint and openWorldHint as explicit booleans on every tool in app submissions, so they do read it. Their definition is true for public or open-ended things, including web search and arbitrary destinations, and false for a bounded private account or workspace even when it's externally hosted. They also say that connectivity alone doesn't decide it (guidelines). That's close to what #355 describes, other parties than the origin.

GitHub's MCP server sets it on its issue and PR write tools too, so this isn't only an idea on paper.

OpenAI asking for explicit values is also one more point for dropping the defaults. They don't want to guess what a missing hint means.

@kenchris

@kenchris thanks for the detailed examples. There are multiple problems being discussed on this issue, so I'm jotting them down here. We can take them as separate resolutions.

Annotations are optional, which creates ambiguity about what to do when the author doesn't provide them. The options are:

a) Make them required.
b) Keep them optional and forward the undefined state to agents. Agents can then decide the behaviour for the undefined state.
c) Keep them optional and pick a default in the spec. That default could lean towards either the conservative or the permissive behaviour for each hint. MCP takes the conservative default, e.g. destructiveHint and openWorldHint default to true.
d) Keep them optional and default everything to false.

Currently we're doing d), which is the worst of all since the defaults aren't consistent either way. I lean towards a). Based on your research, OpenAI guidelines are leaning in that direction too.
Adding openWorldHint and clarifying its overlap with consequentialHint. My understanding of consequentialHint is that it's meant to trigger user consent. The site is telling the agent: "this tool should have a human in the loop". @jpagnucco can confirm.

For openWorldHint, I'm still not following what an agent would do differently based on it. The agent already has to decide which information is OK to share with an origin, based on whether sharing it aligns with user intent, did the information came from the origin itself, was the information previously shared with this origin etc. Once some information has been shared with an origin, you have to assume that the origin may share it with any third party. So what would an agent do differently based on this hint?

Thanks for splitting it up.

On 1, I'd go with b). With a default, a missing hint and an explicit one look the same to the agent, and the page never said that. We ran into this in our agent harness, where a tool with no hint and a tool with an explicit one ended up handled the same way. If a missing hint reaches the agent as missing, the host can decide what that means for itself.

I wouldn't make them required. The next hint that gets added would break every tool that's already out there, and new members of a dictionary have to be optional on the web. OpenAI can ask for them in a submission process. A browser has no such step for the sites that register tools.

On 2, I get that data sent to an origin may travel on from there, but that's an assumption about what a site could do. The hint is the site saying that this tool does. Say the user trusts a site fully, so its tool calls get approved automatically. If one of its tools says openWorldHint: true, the host can stop auto-approving that call and ask, because the user trusted the site that declared the tool, and the other parties it sends data to were never part of that. It works the other way too. When a trusted site says false, the host can skip the argument checks it would otherwise run on that call.

A true is safe to act on, since a page that lies about it only makes the host stricter. A false only matters for sites the user already trusts. Sites that write WebMCP tools for their own pages also have no reason to give wrong information. They add the tools so agents work well with the site, and wrong hints make that worse. There could be legal trouble too. Telling users, or their agents, that data stays inside when it goes to a third party could run into rules like GDPR in Europe. It's also a way for a site to look after its users. A site that forwards data to a third party can say so, and the host asks the user first, instead of the data going out under the trust the user gave the site. The same site can have tools that stay inside and tools that forward to a third party, and the general assumption can't tell those apart.

Coming back to the MCP post on annotations I linked earlier, Tool Annotations as Risk Vocabulary. It covers more of this discussion than the defaults.

On optional vs required, it says that making annotations optional "kept the barrier to entry low for server authors". It also admits that coverage ends up uneven, so there is a cost, but it's the trade I'd make for something that keeps getting new members.

On openWorldHint it describes the hint as "about where the tool reaches and what its output might carry back", and tells server authors to set false on closed-domain tools. Clients "must treat them as untrusted unless they come from a trusted server", and annotations from a trusted one "can drive policy". That's the same idea as the trusted site and auto-approval case above.

There's also an interest group in MCP working on these questions, so it may be worth watching where they end up.

Thanks, both & +1 to Khushal

"the host can stop auto-approving that call and ask" --> I'm still struggling to see (from a security perspective) how in this case openWorldHint = true vs. consequentialHint = true makes the agent behave any differently here. I think if we have consequential = true here the agent would see that and ask for HITL?

I made it sound like the outcome is the same, which is on me. It isn't.

consequentialHint: true tells the host the action is significant, and the host will usually want to ask the user first. That's right for deleting an order and wrong for a read-only search. A site that marked its search tool consequential just to be safe would likely get a prompt in front of every search.

For a tool that isn't consequential but has openWorldHint: true, I'd want the host to leave the user alone and look at the arguments instead. It only asks if they carry something that shouldn't leave, like a secret or something read from another origin. So the two hints aren't a more and a less cautious version of the same thing. They cover different cases:

Not consequential, closed: just run it.
Consequential, closed: usually ask the user.
Not consequential, open: check the arguments.
Consequential and open: usually ask the user, and the check can still run.

It also matches what people care about. I think users really want to be asked before an agent spends their money, and many won't care much if their name or email goes to a third party (I would care less than spending my money). Those are different decisions with different stakes, so one hint shouldn't have to carry both.

The find_flights example is the third row. It changes nothing, so consequentialHint has nothing to say, and it still sends the query to someone else. If consequential were stretched to cover that, every read-only tool that talks to a third party would count as consequential, and the hint would stop meaning "this changes something".

One more thing on the cost side. Some argument checks are cheap, like parsing and pattern detection for secrets, emails or card numbers. Others need a model to judge whether the text is something that shouldn't leave, and that costs tokens on a hosted model and time on a local one (I work on local AI ;-)). A host can't run the model check on every call. The hint is what tells it which calls are worth it.

In a tool loop with say 20-50 tool calls (increasingly more common), having to run a local classifier on each call that takes seconds is a real slowdown for local AI, and for cloud AI it is just token intensive. Given that I work on guardrails, I can tell that some enterprises want additional guardrails to avoid leaking addresses, named entities etc. which requires running some, at times sizable, models.

One more reason not to ask the user on every open-world call is prompt fatigue. Anthropic's write-up on Claude Code's auto mode (How we built Claude Code auto mode) says constant prompts lead to approval fatigue, where "people stop paying close attention to what they're approving", and that "in practice users accept 93% of them anyway". Their fix is a model-based classifier that approves on the user's behalf.

So I'd keep asking for the consequential cases, like spending money, and let open world mean "check the arguments". That way a prompt still gets read when it shows up.

Perhaps the W3C's Privacy Principles could be useful in thinking about the relevant details about the implications of different tools/operations. I don't think the Web defines anything like openWorld, but there are more nuanced notions of contexts (which could be smaller than an origin) and an LLM-based tool might be better able to recognize when something is inappropriately expanding the context of information, even when it's within the same origin or corporation.

Prompt fatigue could certainly be a problem; the privacy principles in particular note the problem of "privacy labor" being outsourced onto the user. Perhaps an alternative approach could be providing fewer unsafe interactions, rather than encouraging so many unsafe unpredictable interactions that prompting becomes constant and users are inevitably both harmed and then blamed for the harm.

@kenchris

On 2, @kenchris I like your phrasing of the difference: consequentialHint implies always ask the user while openWorldHint is saying it depends on which information is being sent to the origin. But I still don't think a boolean hint is enough for this problem, the expected agent UX here is very ambiguous. What should the the user prompt say: "I'm giving your email to this site, they may share it with... everyone!?". And once some data has been shared with a site, from the platform's perspective the user trusts the site on where that information goes further. I suspect there's an overlap between this and #165. Because the site might want to provide more precise information to the user about where they are sending this user's information.

On 1, "I wouldn't make them required. The next hint that gets added would break every tool that's already out there". That makes a lot of sense. 1b) then makes a lot of sense. If a new hint is added, agents might also differentiate behaviour on sites which adopt it explicitly vs older ones which haven't adopted it yet.

@khushalsagar

Adding my perspective here, following @khushalsagar’s earlier split of the discussion topics:

For 1, I would prefer b and d. I remember concerns in the earlier discussion about agents handling the third state (undefined) properly, so it would be worth revisiting that: #217 if we are considering the "undefined" state.

For 2, there were a few reasons we landed on consequentialHint in #176:

Looking at how MCP clients were using the hints then, “destructive” didn’t cover all the use cases. Creating a cloud instance, sending an email or sending a calendar invite can all be consequential without deleting or overwriting anything. We were already seeing MCP tools stretch destructiveHint to cover creates and other important actions.
The idea behind the hints was to let authors communicate that significance and leave the agent to decide when confirmation is needed. My understanding is that the current spec doesn’t require the agent to ask the user every time it sees this hint, right?

On introducing openWorldHint, @kenchris, I want to make sure I understand what you’re trying to get a signal of here. Is it whether a tool forwards its inputs to a party beyond the page’s own origin, so agents can apply extra screening? Or whether it interacts with an open-ended set of external entities? Those seem like different things to me.

I’m a bit unsure about adopting the openWorldHint MCP concept directly. MCP covers local, bounded tools as well as tools that interact with the wider world. For websites, that distinction doesn’t necessarily line up. Posting a comment on Reddit, for example, interacts with the outside world even if everything goes through Reddit’s own origin. That's why I would like to clarify what we want agents to understand from this hint before carrying over the name and definition. I feel like going from what is needed direction, and then discussion what we can do about it would be more productive.

You're right about consequentialHint. The spec doesn't tell the agent to ask every time. It says the action is significant, real-world or hard to reverse, and the agent decides what to do with that. "Ask every time" is how I was thinking of implementing it, and I wrote it in an earlier comment as if it were the rule. I've fixed that comment.

For openWorldHint I mean the first thing you listed: the tool sends its arguments to a party other than the site that registered it, so the host knows to look at them before the call goes out. The "open-ended set of entities" reading doesn't carry over to the web well, I agree.

Your Reddit example sits on the other side of that line. Posting a comment stays on Reddit's origin, so openWorldHint would be false. It's public and hard to take back, which is what consequentialHint covers. A flight search that passes the query on to an airline API is the opposite. It changes nothing, so it isn't consequential, but the text leaves the site.

I'm happy to write the definition for the web first. Something like "the tool sends its arguments to a party other than the origin that registered it." I'd keep the name, since MCP clients already know it, but the definition should be the web one.

On 1, I'm not sure how b and d fit together. Do you mean the host sees undefined as missing, and false only applies when something reads the value without caring? And what did you want me to look at in #217? It looks like the PR that added consequentialHint.

On 2, I don't think the hint has to write the prompt. What it does is make the host look at the arguments, and then the prompt can say something concrete, like "this sends your email address to flights.example". Without the hint the host has nothing specific to say, so it asks about everything or about nothing.

You're right that a boolean can't tell the user where the data goes next. That sounds like extra detail on top of it, and if #165 is where that would live, I don't see it conflicting with having the boolean first.

Good point on 1b, by the way. Agents telling sites that adopted a new hint from ones that haven't is a good reason to keep undefined visible.

@johannhof

My read on the discussion is that there's general consensus on 1b): if the author doesn't set an annotation, the agent harness should get a tri-state (unset, true, false). Having the information gives agent harness developers the most flexibility. Since this change is backwards incompatible, we need to make it before shipping the API.

@johannhof @jpagnucco is that reasonable from a security perspective? @bvandersloot-mozilla any concerns on your end?

Navigation Menu

Repository navigation

Introduce semantic hints for side-effects of tools #53

1 remaining item

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Introduce semantic hints for side-effects of tools #53

Description

Activity

MiguelsPizza commented on Nov 11, 2025

victorhuangwq commented on Nov 11, 2025

themaker00001 commented on Feb 28, 2026

victorhuangwq commented on Mar 3, 2026

kenchris commented on Mar 4, 2026

prakersh commented on Mar 16, 2026

kenchris commented on Oct 20, 2026

kenchris commented on Oct 20, 2026

kenchris commented on Oct 20, 2026

sjdallst2 commented on Nov 2, 2026

kenchris commented on Nov 4, 2026

jpagnucco commented on Nov 4, 2026

1 remaining item

kenchris commented on Nov 4, 2026

khushalsagar commented on Nov 4, 2026

kenchris commented on Nov 4, 2026

kenchris commented on Nov 4, 2026

jpagnucco commented on Nov 4, 2026

kenchris commented on Nov 4, 2026

kenchris commented on Nov 4, 2026

kenchris commented on Nov 4, 2026

npdoty commented on Nov 4, 2026

khushalsagar commented on Nov 4, 2026

victorhuangwq commented on Nov 4, 2026

kenchris commented on Nov 5, 2026

kenchris commented on Nov 5, 2026

khushalsagar commented on Nov 5, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions