Fix key mapping in Anthropic CUA Client #1374

miguelg719 · 2025-12-05T21:17:04Z

why

Anthropic agents in CUA mode are unable to issue key presses (not to be confused with type actions)

what changed

The format for the anthropic tool computer_20250124 replies with:

{
  "action":"key",
  "text":"BackSpace"
}

wasn't properly mapped to our internal action abstraction: keypress, which accepts parameter keys. It was issued directly from the anthropic format. Updated AnthropicCUAClient.ts to account for this and map appropriately

test plan

Tested on sample eval

Summary by cubic

Fixes key action mapping in Anthropic CUA so agents can send key presses (e.g., Backspace) correctly instead of failing on the "key" action.

Bug Fixes
- Map Anthropic "key" to internal "keypress" and pass keys from input.text.
- Remove the old "key" path and Playwright key mapping to avoid mismatches.

^{Written for commit b9716b9. Summary will update automatically on new commits.}

changeset-bot · 2025-12-05T21:17:08Z

🦋 Changeset detected

Latest commit: b9716b9

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 2 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

cubic-dev-ai

No issues found across 2 files

greptile-apps · 2025-12-05T21:21:44Z

Greptile Overview

Greptile Summary

Fixed Anthropic CUA agents to properly handle key press actions by mapping the key action type to the internal keypress action format.

Removed old key action handler that used mapKeyToPlaywright utility and standalone processing
Added key action support to the unified keypress handler, mapping input.text to the keys parameter
Removed unused mapKeyToPlaywright import from AnthropicCUAClient

This change enables Anthropic agents in CUA mode to issue key presses (like Backspace, Enter, etc.) which were previously failing.

Confidence Score: 4/5

This PR fixes a bug and is mostly safe to merge with one type casting issue to address
The core fix correctly maps Anthropic's "key" action to the internal "keypress" action. However, there's a type casting issue on line 777 where input.text (a string) is cast to string[], which could cause the handler to iterate over characters instead of treating it as a single key.
Pay close attention to packages/core/lib/v3/agent/AnthropicCUAClient.ts line 777 for the type casting fix

Important Files Changed

File Analysis

Filename	Score	Overview
.changeset/shy-sides-crash.md	5/5	Standard changeset file for tracking the patch release
packages/core/lib/v3/agent/AnthropicCUAClient.ts	4/5	Fixed key action mapping to properly convert Anthropic's "key" action to internal "keypress" action, but has a potential type casting issue

Sequence Diagram

sequenceDiagram
    participant Agent as Anthropic Agent
    participant Client as AnthropicCUAClient
    participant Handler as v3CuaAgentHandler
    participant Browser as Playwright Page

    Agent->>Client: Tool use: {action: "key", text: "BackSpace"}
    Client->>Client: convertToolUseToAction()
    Note over Client: Maps "key" action to "keypress"
    Note over Client: Extracts text parameter
    Client->>Client: Return {type: "keypress", keys: text}
    Client->>Handler: Execute keypress action
    Handler->>Handler: Normalize keys to array
    Handler->>Handler: Map key to Playwright format
    Handler->>Browser: page.keyPress(mapped)
    Browser-->>Handler: Key pressed
    Handler-->>Client: Action complete
    Client-->>Agent: Screenshot with result

greptile-apps

_{1 file reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

packages/core/lib/v3/agent/AnthropicCUAClient.ts

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

@miguelg719

This PR was opened by the [Changesets release](https://github.com/changesets/action) GitHub action. When you're ready to do a release, you can merge this and the packages will be published to npm automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated. # Releases ## @browserbasehq/[email protected] ### Patch Changes - [#1388](#1388) [`605ed6b`](605ed6b) Thanks [@miguelg719](https://github.com/miguelg719)! - Fix multiple click event dispatches on CDP and Anthropic CUA handling (double clicks) - [#1400](#1400) [`34e7e5b`](34e7e5b) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - don't write base64 encoded screenshots to disk when caching agent actions - [#1345](#1345) [`943d2d7`](943d2d7) Thanks [@tkattkat](https://github.com/tkattkat)! - Add support for aborting / stopping an agent run & continuing an agent run using messages from prior runs - [#1334](#1334) [`0e95cd2`](0e95cd2) Thanks [@tkattkat](https://github.com/tkattkat)! - Add support for google vertex provider - [#1410](#1410) [`d4237e4`](d4237e4) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: include extract in stagehand.history() - [#1315](#1315) [`86975e7`](86975e7) Thanks [@tkattkat](https://github.com/tkattkat)! - Add streaming support to agent through stream:true in the agent config - [#1304](#1304) [`d5e119b`](d5e119b) Thanks [@miguelg719](https://github.com/miguelg719)! - Add support for Microsoft's Fara-7B - [#1346](#1346) [`4e051b2`](4e051b2) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: don't attach to targets twice - [#1327](#1327) [`6b5a3c9`](6b5a3c9) Thanks [@miguelg719](https://github.com/miguelg719)! - Informed error parsing from api - [#1335](#1335) [`bb85ad9`](bb85ad9) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - add support for page.addInitScript() - [#1331](#1331) [`88d28cc`](88d28cc) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: page.evaluate() now works with scripts injected via context.addInitScript() - [#1316](#1316) [`45bcef0`](45bcef0) Thanks [@tkattkat](https://github.com/tkattkat)! - Add support for callbacks in stagehand agent - [#1374](#1374) [`6aa9d45`](6aa9d45) Thanks [@miguelg719](https://github.com/miguelg719)! - Fix key action mapping in Anthropic CUA - [#1330](#1330) [`d382084`](d382084) Thanks [@seanmcguire12](https://github.com/seanmcguire12)! - fix: make act, extract, and observe respect user defined timeout param - [#1336](#1336) [`1df08cc`](1df08cc) Thanks [@tkattkat](https://github.com/tkattkat)! - Patch agent on api - [#1358](#1358) [`2b56600`](2b56600) Thanks [@tkattkat](https://github.com/tkattkat)! - Add support for 4.5 opus in cua agent ## @browserbasehq/[email protected] ### Patch Changes - [#1364](#1364) [`ca0630e`](ca0630e) Thanks [@tkattkat](https://github.com/tkattkat)! - Update model handling in agent evals cli - Updated dependencies \[[`605ed6b`](605ed6b), [`34e7e5b`](34e7e5b), [`943d2d7`](943d2d7), [`0e95cd2`](0e95cd2), [`d4237e4`](d4237e4), [`86975e7`](86975e7), [`d5e119b`](d5e119b), [`4e051b2`](4e051b2), [`6b5a3c9`](6b5a3c9), [`bb85ad9`](bb85ad9), [`88d28cc`](88d28cc), [`45bcef0`](45bcef0), [`6aa9d45`](6aa9d45), [`d382084`](d382084), [`1df08cc`](1df08cc), [`2b56600`](2b56600)]: - @browserbasehq/[email protected] ## @browserbasehq/[email protected] ### Patch Changes - Updated dependencies \[[`605ed6b`](605ed6b), [`34e7e5b`](34e7e5b), [`943d2d7`](943d2d7), [`0e95cd2`](0e95cd2), [`d4237e4`](d4237e4), [`86975e7`](86975e7), [`d5e119b`](d5e119b), [`4e051b2`](4e051b2), [`6b5a3c9`](6b5a3c9), [`bb85ad9`](bb85ad9), [`88d28cc`](88d28cc), [`45bcef0`](45bcef0), [`6aa9d45`](6aa9d45), [`d382084`](d382084), [`1df08cc`](1df08cc), [`2b56600`](2b56600)]: - @browserbasehq/[email protected] Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

@miguelg719

Fix key mapping in Anthropic CUA Client

772b8af

remove log

cbedcdd

miguelg719 marked this pull request as ready for review December 5, 2025 21:17

cubic-dev-ai bot reviewed Dec 5, 2025

View reviewed changes

greptile-apps bot reviewed Dec 5, 2025

View reviewed changes

packages/core/lib/v3/agent/AnthropicCUAClient.ts Outdated Show resolved Hide resolved

Update packages/core/lib/v3/agent/AnthropicCUAClient.ts

b9716b9

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

seanmcguire12 approved these changes Dec 5, 2025

View reviewed changes

miguelg719 merged commit 6aa9d45 into main Dec 8, 2025
17 checks passed

This was referenced Dec 8, 2025

Version Packages #1305

Merged

Version Packages CloudEngineHub/stagehand#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix key mapping in Anthropic CUA Client #1374

Fix key mapping in Anthropic CUA Client #1374

miguelg719 commented Dec 5, 2025 •

edited by cubic-dev-ai bot

Loading

Uh oh!

changeset-bot bot commented Dec 5, 2025 •

edited

Loading

Uh oh!

cubic-dev-ai bot left a comment

Uh oh!

greptile-apps bot commented Dec 5, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fix key mapping in Anthropic CUA Client #1374

Fix key mapping in Anthropic CUA Client #1374

Conversation

miguelg719 commented Dec 5, 2025 • edited by cubic-dev-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

why

what changed

test plan

Summary by cubic

Uh oh!

changeset-bot bot commented Dec 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot commented Dec 5, 2025

Greptile Overview

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

miguelg719 commented Dec 5, 2025 •

edited by cubic-dev-ai bot

Loading

changeset-bot bot commented Dec 5, 2025 •

edited

Loading