Skip to content

[Bug]: Browser aria snapshot refs resolve in snapshots but fail in follow-up actions #69289

Description

@FoundryHouseAI

Bug type

Regression (worked before, now fails)

Beta release blocker

No

Summary

In my local OpenClaw runtime, browser page open/snapshot works, but follow-up browser actions against aria snapshot refs fail with locator timeouts.

Steps to reproduce

Minimal repro

  1. Open https://example.com
  2. Take interactive snapshot with snapshotFormat: "aria"
  3. Try clicking the Learn more link using returned ref (ax13)
  4. Action times out waiting for locator

Also reproduced on Google search input with aria refs.

Expected behavior

Using snapshotFormat: "aria" should return refs that can be used in later browser.act calls for click/type.

Actual behavior

Snapshot succeeds and returns refs like ax13 / ax39, but follow-up actions fail, for example:

TimeoutError: locator.click: Timeout 15000ms exceeded
waiting for locator('aria-ref=ax13')
TimeoutError: locator.fill: Timeout 15000ms exceeded
waiting for locator('aria-ref=ax39')

OpenClaw version

OpenClaw: 2026.4.18

Operating system

OS: macOS 12.7.6

Install method

pnpm dev

Model

openai-codex/gpt-5.4

Provider / routing chain

webchat -> local OpenClaw gateway -> OpenAI provider -> openai-codex/gpt-5.4

Additional provider/model setup details

Main agent running in local webchat on macOS. Browser control used the managed \openclawbrowser profile through the local LaunchAgent-managed gateway. Effective model in runtime/status:openai-codex/gpt-5.4`.

Logs, screenshots, and evidence

Evidence summary:

Reproduced on OpenClaw 2026.4.18, macOS 12.7.6, Node 24.14.0.
Browser page open and snapshot succeeded on both https://example.com and https://www.google.com.
With interactive aria snapshots, returned refs like ax13 / ax39 appeared valid in the snapshot but failed in follow-up actions.
Example errors:
TimeoutError: locator.click: Timeout 15000ms exceeded
waiting for locator('aria-ref=ax13')
TimeoutError: locator.fill: Timeout 15000ms exceeded
waiting for locator('aria-ref=ax39')
Reproduced on:
example.com “Learn more” link
Google search box / Google Search button
Confirmed workaround:
snapshotFormat: "ai" with refs like e1 / e7 / e22 works for click/type actions
evaluate/DOM actions also work
Confirmed browser transport is not generally broken:
page open works
snapshot works
AI-ref click/type works
evaluate works
Additional observed context:
managed browser profile showed extra Chrome tabs such as profile picker / tab search, but core repro remained specifically that aria-ref based actions failed while AI-ref actions succeeded.`
If there’s another field for exact logs, paste these lines too:

`TimeoutError: locator.click: Timeout 15000ms exceeded
Call log:

waiting for locator('aria-ref=ax13')`
`TimeoutError: locator.fill: Timeout 15000ms exceeded
Call log:

waiting for locator('aria-ref=ax39')`

Impact and severity

Affected users/systems/channels: Users relying on OpenClaw browser automation in local webchat/runtime setups where browser actions are driven from interactive snapshots. Observed in the local managed browser profile (\openclaw`) through the local gateway.

Severity:
Blocks a documented browser interaction workflow. Browser automation can appear broken when using aria snapshot refs for follow-up click/type actions. Workarounds exist (AI refs, evaluate), so this is not a total browser outage, but it is a serious workflow blocker for normal browser-control use.

Frequency:
Consistently reproducible in this environment during testing on both Example.com and Google when using aria snapshot refs for actions. AI-ref actions and evaluate-based actions succeeded.

Consequence:
Failed browser actions, repeated retries, debugging time, and reduced trust in browser automation reliability. Practical effect is that normal browser-control tasks may fail unless the user knows to switch to AI refs or a JS/evaluate workaround.`

Additional information

Observed workaround is reliable so far:

use snapshotFormat: "ai"
use AI refs like e1 / e7 / e22
re-snapshot after UI changes before the next action
use evaluate as fallback when needed
Additional grounded context:

Google produced duplicate "Google Search" buttons after typing, so a fresh snapshot was needed before the follow-up click.
That ambiguity was manageable with AI refs, but aria refs still failed earlier in the flow.
I did not establish a last known good version or first known bad version during this session, so regression version boundaries are not confirmed.`

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingregressionBehavior that previously worked and now fails

    Type

    No type

    Fields

    Priority

    None yet

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions