improve(browser): batch navigation guard, snapshot delta markers, tool output schema, code-mode recipe#113749
Conversation
openclaw-barnacle
Bot
added
docs
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Codex review: needs real behavior proof before merge. Reviewed July 25, 2026, 11:49 AM ET / 15:49 UTC. ClawSweeper reviewWhat this changesThis PR makes browser batches stop after a main-frame navigation or page close, annotates new elements in compatible consecutive snapshots, declares structured Browser tool outputs, and documents a code-mode browsing loop. Merge readiness⛔ Blocked until stronger real behavior proof is added - 5 items remain Keep this PR open for maintainer review. The batch guard addresses a real stale-document hazard, but its current response can report every executed action as successful while silently skipping the remaining requested actions after navigation; that weakens the established batch failure signal for existing programmatic callers. Priority: P2 Review scores
Verification
How this fits togetherThe bundled Browser plugin turns agent tool calls into browser-control requests and returns snapshots or action results to agents. This PR changes how multi-action batches and repeated snapshots communicate document changes back through that tool boundary. flowchart LR
Agent[Agent browser request] --> Tool[Browser tool]
Tool --> Batch[Batch action executor]
Tool --> Snapshot[Snapshot route]
Batch --> Guard[Navigation and close guard]
Snapshot --> Delta[Per-tab snapshot baseline]
Guard --> Result[Structured tool result]
Delta --> Result
Result --> AgentResponse[Agent recovery or next action]
Decision needed
Why: The PR intentionally changes whether later requested actions run, but current documented batch consumers are guided by per-entry failures and nonzero failure behavior rather than an aborted field. Before merge
Findings
Agent review detailsSecurityNone. PR surfaceSource +592, Tests +483, Docs +54. Total +1129 across 34 files. View PR surface stats
Review metrics
Stored data modelPersistent data-model change detected: Merge-risk optionsMaintainer options:
Copy recommended automerge instructionTechnical reviewBest possible solution: Preserve the guard, but make an aborted batch unambiguously unsuccessful through the established batch result/error contract while retaining structured abort metadata for recovery, then provide a redacted real browser-run artifact that shows the navigation and recovery path. Do we have a high-confidence way to reproduce the issue? Yes, source inspection gives a high-confidence path: run a batch whose first action commits a navigation, then observe that the result can contain only successful executed entries plus an aborted summary while later actions are skipped. The PR includes focused unit coverage for navigation and close detection, but no independent runtime execution was possible in this read-only review. Is this the best way to solve the issue? Unclear. Stopping after document replacement is the safer behavior, but reporting that partial execution only through a new optional field is not the best compatibility path unless maintainers explicitly adopt that as the Browser batch API. Full review comments:
Overall correctness: patch is incorrect AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning high; reviewed against 924aef2e7c4e. LabelsLabel changes:
Label justifications:
EvidenceWhat I checked:
Likely related people:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale
Overall follows the weaker of proof and patch quality. Workflow
|
|
Addressing the two rank-up moves: 1. Nonzero/failure signal for aborted batches — consciously skipped. The executed prefix of an aborted batch genuinely succeeded (the click worked; it just navigated). Folding that into a failure signal would teach callers that a successful, navigation-causing action is an error, which is exactly the ambiguity the 2. Live transcript — attached below. Managed headless Chromium on an isolated dev gateway against a local fixture site: three-click batch where click 1 navigates; the guard aborts the remaining two with metadata; re-snapshot yields fresh document-scoped refs; follow-up click lands on the target page. # Batch navigation guard: abort -> re-snapshot -> safe follow-up
# Managed headless Chromium against a local fixture shop (isolated dev gateway).
$ openclaw browser snapshot --interactive # on /catalog.html
- link "Alpine Anorak" [ref=e1]
- link "Glacier Crampons" [ref=e2]
- link "Summit Stove" [ref=e3]
$ openclaw browser batch --json --actions '[{"kind":"click","ref":"e3"},{"kind":"click","ref":"e1"},{"kind":"click","ref":"e2"}]'
# e3 = "Summit Stove" -> navigates. Pre-guard, the remaining clicks would resolve
# e1/e2 against the NEW document. Now the batch aborts with metadata:
{
"ok": true,
"targetId": "E154EA8A11A534577AF0BE679CC107D5",
"url": "http://127.0.0.1:8931/item-stove.html",
"results": [
{
"ok": true,
"navigated": true,
"url": "http://127.0.0.1:8931/item-stove.html"
}
],
"aborted": {
"reason": "navigation",
"afterAction": 1,
"url": "http://127.0.0.1:8931/item-stove.html",
"skipped": 2
}
}
$ openclaw browser snapshot --interactive # re-snapshot on the new document
- link "Full specifications" [ref=e1]
$ openclaw browser click e1 # safe follow-up with a fresh ref
clicked ref e1 on http://127.0.0.1:8931/item-stove-specs.html
$ openclaw browser snapshot # landed on the specs page
- heading "Summit Stove — Specifications" [level=1] [ref=e2]
- cell "Weight" [ref=e6]
- cell "212 g" [ref=e7]
- cell "Fuel" [ref=e9] |
|
Merged via squash.
|
…l output schema, code-mode recipe (openclaw#113749) * feat(browser): guard batches and annotate snapshot deltas * docs(browser): document reliable automation loops * test(browser): align batch guard mocks * fix(browser): reset snapshot deltas after navigation * fix(browser): detect document navigation without URL equality * fix(browser): require stable identity for snapshot deltas * fix(browser): keep snapshot delta helper private
1 participant
What Problem This Solves
Multi-step browser automation is fragile in three specific ways:
batchof actions has no awareness of page identity. When an early action triggers a navigation (link click, form submit), the remaining actions run against the new document, where role/name refs resolve to the wrong elements — the batch silently does the wrong thing instead of failing.browsertool declares no output schema, so code-mode catalogs index its output as unknown and agents burn an extra round trip discovering the result shape before they can write filtering code.Why This Change Was Made
These are the dominant failure modes in longer agent browsing sessions: acting on a stale document, missing in-place UI changes, and dumping full page trees into context. All three are fixable at the tool layer without new input parameters or config surface:
batchViaPlaywrightnow tracks the main-frame URL across actions. A URL-changing navigation (or page close) aborts the remaining actions and returns anabortedsummary (reason,afterAction,url,skipped) telling the model to re-snapshot. Snapshot refs are document-scoped; continuing after navigation acts on the wrong document. Same-URL SPA mutations are intentionally out of scope.[new], report anewElementscount, and append a short footer. The baseline is a bounded (32-entry LRU) process-local cache owned by the control-server context; navigation or tab close resets the baseline so a new page is never rendered as a wall of[new]markers.aborted/[new].User Impact
Agents driving the browser through multi-page flows fail less and spend fewer tokens: mid-batch navigations become an explicit, recoverable signal instead of silent wrong-element actions; in-place UI changes (autocomplete, dialogs) are called out directly in the snapshot; code-mode sessions can filter page state in the sandbox instead of round-tripping full trees through model context. No input-schema or config changes; existing callers see purely additive result fields.
Evidence
node scripts/run-vitest.mjs extensions/browser: 160 files, 1,980 tests passed (1 skipped), including new coverage for the batch guard (navigation abort, close abort, stopOnError interplay), delta annotation (first-snapshot baseline, same-document markers, navigation reset, truncation interplay), and the output schema.node scripts/check-changed.mjs: formatting, production/test typechecks, lint, boundary guards, import cycles — all green.openai/gpt-5.6-lunaon the embedded runtime, isolated per-run sessions, same harness both sides on the same day: baseline (main) 19/24 → this branch 20/24, median wall time 114s → 112s. Aggregate pass rate is statistically flat at this sample size — the wins are in the mechanisms, not the headline: branch transcripts show[new]delta markers firing in 11 events during live autocomplete/reveal flows (baseline: zero, feature absent), directly surfacing in-place DOM changes the model previously had to diff mentally. The batch navigation guard did not trigger in these runs (the model issued no cross-navigation batches); its behavior is locked in by unit tests (navigation abort, same-URL reload viaframenavigated, closed-page abort, stopOnError interplay).