Skip to content

Commit 7b3f50f

Browse files
Add consequentialHint to ToolAnnotations (#217)
Co-authored-by: bvandersloot-mozilla <[email protected]>
1 parent 55fb7ee commit 7b3f50f

1 file changed

Lines changed: 22 additions & 2 deletions

File tree

‎index.bs‎

Lines changed: 22 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,9 @@ An <dfn>annotations</dfn> is a [=struct=] with the following [=struct/items=]:
211211

212212
: <dfn>untrusted content hint</dfn>
213213
:: a [=boolean=], initially false.
214+
215+
: <dfn>consequential hint</dfn>
216+
:: a [=boolean=], initially false.
214217
</dl>
215218

216219
<h3 id="pending-tool-executions">Pending tool executions</h3>
@@ -758,6 +761,9 @@ The <dfn method for=ModelContext>registerTool(<var>tool</var>, <var>options</var
758761
: [=annotations/untrusted content hint=]
759762
:: |tool|'s {{ModelContextTool/annotations}}'s {{ToolAnnotations/untrustedContentHint}}
760763

764+
: [=annotations/consequential hint=]
765+
:: |tool|'s {{ModelContextTool/annotations}}'s {{ToolAnnotations/consequentialHint}}
766+
761767
: [=tool definition/exposed origins=]
762768
:: |exposed origins|
763769

@@ -874,9 +880,11 @@ The <dfn method for=ModelContext>getTools(<var>options</var>)</dfn> method steps
874880
: {{RegisteredTool/annotations}}
875881
:: if |tool definition|'s [=tool definition/annotations=] is not null, a
876882
{{ToolAnnotations}} dictionary whose {{ToolAnnotations/readOnlyHint}} is |tool
877-
definition|'s [=tool definition/annotations=]'s [=annotations/read-only hint=] and
883+
definition|'s [=tool definition/annotations=]'s [=annotations/read-only hint=],
878884
{{ToolAnnotations/untrustedContentHint}} is |tool definition|'s [=tool
879-
definition/annotations=]'s [=annotations/untrusted content hint=].
885+
definition/annotations=]'s [=annotations/untrusted content hint=], and
886+
{{ToolAnnotations/consequentialHint}} is |tool definition|'s [=tool
887+
definition/annotations=]'s [=annotations/consequential hint=].
880888

881889
1. [=list/Append=] |registeredTool| to |tools|.
882890

@@ -1068,6 +1076,7 @@ dictionary ModelContextTool {
10681076
dictionary ToolAnnotations {
10691077
boolean readOnlyHint = false;
10701078
boolean untrustedContentHint = false;
1079+
boolean consequentialHint = false;
10711080
};
10721081

10731082
dictionary ToolExecuteCallbackOptions {
@@ -1124,6 +1133,9 @@ The {{ToolAnnotations}} dictionary provides optional metadata about a tool:
11241133

11251134
: <code><var ignore>annotations</var>["{{ToolAnnotations/untrustedContentHint}}"]</code>
11261135
:: If true, indicates that the tool's output contains data that is untrusted, from the perspective of the author registering the tool.
1136+
1137+
: <code><var ignore>annotations</var>["{{ToolAnnotations/consequentialHint}}"]</code>
1138+
:: If true, indicates that executing the tool will result in consequential actions that are significant, real-world, or non-reversible, ex: booking a flight, transferring money.
11271139
</dl>
11281140

11291141
<h4 id="tool-execute-callback-options">ToolExecuteCallbackOptions Dictionary</h4>
@@ -1797,6 +1809,14 @@ respective private browsing modes are safely exposed to [=agents=] and that thes
17971809

17981810
**How:** A boolean {{ToolAnnotations/untrustedContentHint}} annotation that acts as a signal to the client that the payload requires heightened security handling, allowing the client to sanitize the payload, use indicators such as spotlighting [[SPOTLIGHTING]] to highlight untrustworthy content to the model, or hide that part of the response entirely.
17991811

1812+
<h4 id="mitigation-consequential-annotation">Consequential Annotation for Tool Executions</h4>
1813+
1814+
**What:** Providing agents with a signal that a tool's execution results in significant, real-world, or non-reversible consequences.
1815+
1816+
**Threats addressed:** [[#misrepresentation-of-intent]]
1817+
1818+
**How:** A boolean {{ToolAnnotations/consequentialHint}} annotation acts as a signal to the client or agent that the tool performs a consequential action, such as booking a flight or transferring money. This way they can selectively enforce mandatory user confirmation prompts before executing high-stakes tools, directly mitigating the risk of accidental or malicious misrepresentation of intent.
1819+
18001820
<h2 id="accessibility">Accessibility considerations</h2>
18011821

18021822

0 commit comments

Comments
 (0)