update agent cache handling #1431

tkattkat · 2025-12-18T01:21:49Z

why

add support for keys cache

what changed

save click , and type actions in act cache for agent

test plan

ran on sign in example with click type and keys , re ran using cache

Summary by cubic

Makes agent actions deterministic and cacheable by recording XPath-based “act” steps for clicks, typing, drag-and-drop, click-and-hold, and form fills, and adds replay support for key events. This improves cache hit rates and makes replays more reliable.

New Features
- Record tools as “act” steps with Action objects (method, selector, args) using XPath captured via returnXpath.
- Add AgentReplayKeysStep and AgentCache logic to replay type/press (with repeat support).
- Introduce ensureXPath utility and use it across tools and v3CuaAgentHandler to normalize selectors.

^{Written for commit f62c537. Summary will update automatically on new commits.}

changeset-bot · 2025-12-18T01:21:54Z

🦋 Changeset detected

Latest commit: 29d57a3

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

cubic-dev-ai

2 issues found across 10 files

Prompt for AI agents (all 2 issues)


Check if these issues are valid — if so, understand the root cause of each and fix them.


<file name="packages/core/lib/v3/agent/tools/type.ts">

<violation number="1" location="packages/core/lib/v3/agent/tools/type.ts:65">
P2: The replay step is silently skipped when xpath is not available. Previously, the step was always recorded. This could cause incomplete replay caches with no logging or warning. Consider adding an else clause to log when xpath cannot be determined, or fall back to recording with coordinates.</violation>
</file>

<file name="packages/core/lib/v3/agent/tools/click.ts">

<violation number="1" location="packages/core/lib/v3/agent/tools/click.ts:61">
P1: Silent failure when xpath is unavailable - click succeeds but no replay step is recorded. The old code always recorded a replay step, but now if `xpath` is null/undefined/empty, caching is silently skipped. Consider adding a fallback to record using coordinates when xpath is unavailable, or at least log a warning.</violation>
</file>

_{Reply to cubic to teach it or ask questions. Re-run a review with @cubic-dev-ai review this PR}

packages/core/lib/v3/agent/tools/type.ts

packages/core/lib/v3/agent/tools/click.ts

greptile-apps · 2025-12-18T01:25:43Z

Greptile Summary

This PR enhances agent cache handling by converting coordinate-based actions (click, type, dragAndDrop, clickAndHold, fillFormVision) to XPath-based deterministic actions for reliable cache replay. The implementation extracts returnXpath from page interactions and normalizes them using a new ensureXPath utility function.

Key changes:

Created packages/core/lib/v3/agent/utils/xpath.ts utility to normalize XPath selectors
Modified agent tools to capture XPaths during execution and record as "act" steps instead of coordinate-based steps
Added AgentReplayKeysStep type and replay handler to support keyboard action caching
Refactored v3CuaAgentHandler to use the extracted ensureXPath utility

Architecture:
The changes shift from coordinate-based replay (brittle, viewport-dependent) to selector-based replay (deterministic, resilient to layout changes). When recording, tools now capture the XPath of interacted elements and store them as Action objects with selectors. During replay, takeDeterministicAction uses these selectors to locate and interact with elements reliably.

Trade-offs:
Actions silently skip cache recording if XPath extraction fails (returns null/empty), which could lead to cache misses without clear debugging signals. For fillFormVision, partial XPath failures result in incomplete action lists being cached.

Confidence Score: 4/5

Safe to merge with minor observability considerations
The implementation is sound and follows good patterns (utility extraction, proper type definitions, graceful fallbacks). The logic correctly handles XPath normalization and recording. Score of 4 (not 5) due to silent failures when XPath extraction fails - while this prevents crashes, it could make debugging cache misses difficult. The fillFormVision partial recording behavior could also cause subtle issues on replay.
Pay close attention to packages/core/lib/v3/agent/tools/fillFormVision.ts - the partial action recording behavior could cause incomplete form fills on cache replay if some XPath extractions fail

Important Files Changed

Filename	Overview
packages/core/lib/v3/agent/utils/xpath.ts	New utility function to normalize XPath selectors with proper `xpath=` prefix, extracted from v3CuaAgentHandler for reusability
packages/core/lib/v3/cache/AgentCache.ts	Added `replayAgentKeysStep` method to handle replaying cached keyboard actions with proper text/key repetition logic
packages/core/lib/v3/agent/tools/click.ts	Modified to capture XPath from click action and record as deterministic "act" step for cache replay; only records if xpath is valid
packages/core/lib/v3/agent/tools/type.ts	Modified to capture XPath from click before typing and record as deterministic "act" step; silently skips recording if xpath is invalid
packages/core/lib/v3/agent/tools/fillFormVision.ts	Modified to capture XPaths for each form field and record as single "act" step with multiple actions; may record partial actions if some xpaths fail

Sequence Diagram

sequenceDiagram
    participant Agent as Agent Tool (click/type/etc)
    participant Page as Page (Understudy)
    participant XPath as ensureXPath Util
    participant Recording as v3.recordAgentReplayStep
    participant Cache as AgentCache
    participant Replay as Cache Replay

    Note over Agent,Replay: Recording Phase (First Execution)
    Agent->>Page: click(x, y, {returnXpath: true})
    Page-->>Agent: xpath string
    Agent->>XPath: ensureXPath(xpath)
    XPath-->>Agent: normalized xpath with "xpath=" prefix (or null)
    alt xpath is valid
        Agent->>Recording: recordAgentReplayStep({type: "act", actions: [{selector: xpath, method: "click", ...}]})
        Recording->>Cache: store step with xpath-based Action
    else xpath is null/invalid
        Note over Agent,Recording: Skip recording (silent failure)
    end

    Note over Agent,Replay: Replay Phase (Cache Hit)
    Cache->>Replay: replayAgentActStep(step)
    Replay->>Page: takeDeterministicAction(action with xpath selector)
    Page->>Page: Locate element by xpath and execute action
    Note over Page: Action replayed deterministically using selector

greptile-apps

_{9 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}

packages/core/lib/v3/agent/tools/click.ts

packages/core/lib/v3/agent/tools/fillFormVision.ts

packages/core/lib/v3/agent/tools/type.ts

pirate

approved but I would recommend recording the typing step even if we cant get a normalized xpath, typing into the page without a specific focus is still better than not typing at all

@tkattkat

update agent cache handling

f62c537

format

29d57a3

cubic-dev-ai bot reviewed Dec 18, 2025

View reviewed changes

packages/core/lib/v3/agent/tools/type.ts Show resolved Hide resolved

packages/core/lib/v3/agent/tools/click.ts Show resolved Hide resolved

greptile-apps bot reviewed Dec 18, 2025

View reviewed changes

packages/core/lib/v3/agent/tools/click.ts Show resolved Hide resolved

packages/core/lib/v3/agent/tools/fillFormVision.ts Show resolved Hide resolved

pirate reviewed Dec 18, 2025

View reviewed changes

packages/core/lib/v3/agent/tools/type.ts Show resolved Hide resolved

pirate approved these changes Dec 18, 2025

View reviewed changes

tkattkat merged commit 05f5580 into main Dec 18, 2025
26 checks passed

This was referenced Dec 18, 2025

Version Packages #1414

Merged

Version Packages CloudEngineHub/stagehand#1

Open

Version Packages kenchikuliu/stagehand#1

Open

Version Packages shaneholloman/stagehand#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

update agent cache handling #1431

update agent cache handling #1431

Uh oh!

tkattkat commented Dec 18, 2025 •

edited by cubic-dev-ai bot

Loading

Uh oh!

changeset-bot bot commented Dec 18, 2025 •

edited

Loading

Uh oh!

cubic-dev-ai bot left a comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot commented Dec 18, 2025

Uh oh!

greptile-apps bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pirate left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

update agent cache handling #1431

update agent cache handling #1431

Uh oh!

Conversation

tkattkat commented Dec 18, 2025 • edited by cubic-dev-ai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

why

what changed

test plan

Summary by cubic

Uh oh!

changeset-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🦋 Changeset detected

Uh oh!

cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

greptile-apps bot commented Dec 18, 2025

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

pirate left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tkattkat commented Dec 18, 2025 •

edited by cubic-dev-ai bot

Loading

changeset-bot bot commented Dec 18, 2025 •

edited

Loading