fix(browser): normalize batch act dispatch for selector and batch support by vincentkoc · Pull Request #45457 · openclaw/openclaw

vincentkoc · 2026-03-13T21:07:36Z

Summary

Problem: the new browser act batch flow still forwarded raw JSON actions, dropped the route-level evaluateEnabled setting, and exposed results at runtime without typing it on the client response.
Why it matters: batched browser actions could diverge from single-action normalization, evaluate/wait --fn inside batches would not receive the configured gate consistently, and typed browserAct() callers could not access batch results.
What changed:
- normalize batch action payloads in src/browser/routes/agent.act.ts before dispatch
- pass evaluateEnabled through to pw.batchViaPlaywright
- add the batch results field to BrowserActResponse
- cover normalized batch dispatch and malformed batch rejection in the browser contract tests
- remove the stray TASK.md file from the cherry-picked feature branch and add an Unreleased changelog entry
What did NOT change (scope boundary): this does not redesign the batch executor itself or the existing direct evaluateEnabled route behavior outside the batch path.

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

User-visible / Behavior Changes

Browser act batches now use the same normalization rules as direct act requests for supported fields.
Batched evaluate / wait --fn requests now receive the route's evaluateEnabled setting.
Typed browser client callers can now read batch results without casting.

Security Impact (required)

New permissions/capabilities? (No)
Secrets/tokens handling changed? (No)
New/changed network calls? (No)
Command/tool execution surface changed? (No)
Data access scope changed? (No)

Repro + Verification

Environment

OS: macOS
Runtime/container: Node.js 22 / pnpm
Model/provider: n/a
Integration/channel (if any): browser control server
Relevant config (redacted): default test harness config

Steps

Run pnpm test -- src/browser/client.test.ts
Run pnpm test -- src/browser/server.agent-contract-form-layout-act-commands.test.ts -t "normalizes batch actions and threads evaluateEnabled into the batch executor"
Run pnpm test -- src/browser/server.agent-contract-form-layout-act-commands.test.ts -t "rejects malformed batch actions before dispatch"

Expected

batch requests are normalized before dispatch
evaluateEnabled is passed into the batch executor
malformed batch actions fail before reaching Playwright mocks
browser client typing includes batch results

Actual

Matches expected for the focused cases above.

Evidence

Failing test/log before + passing after
Trace/log snippets
Screenshot/recording
Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

Verified scenarios:
- focused browser client test passes
- focused browser contract test for normalized batch dispatch passes
- focused browser contract test for malformed batch rejection passes
Edge cases checked:
- batch click fields are coerced through the same string/boolean/number normalization path as direct actions
- typed browser client response now exposes batch results
What you did not verify:
- full browser test suite / full repo test suite
- unrelated direct evaluateEnabled route behavior outside the batch path

Review Conversations

I replied to or resolved every bot review conversation I addressed in this PR.
I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

Backward compatible? (Yes)
Config/env changes? (No)
Migration needed? (No)

Failure Recovery (if this breaks)

How to disable/revert this change quickly: revert 6a191745c8
Files/config to restore: src/browser/routes/agent.act.ts, src/browser/client-actions-core.ts, browser test files, CHANGELOG.md
Known bad symptoms reviewers should watch for: batch actions skipping normalization or typed callers missing results

Risks and Mitigations

Risk: batch normalization drifts from the direct route as new action kinds evolve.
- Mitigation: the helper stays in the route layer and the new contract tests lock the current behavior.

…ayMs Adds three improvements to the browser act tool: 1. CSS selector support: All element-targeting actions (click, type, hover, drag, scrollIntoView, select) now accept an optional 'selector' parameter alongside 'ref'. When selector is provided, Playwright's page.locator() is used directly, skipping the need for a snapshot to obtain refs. This reduces roundtrips for agents that already know the DOM structure. 2. Click delay (delayMs): The click action now accepts an optional 'delayMs' parameter. When set, the element is hovered first, then after the specified delay, clicked. This enables human-like hover-before-click in a single tool call instead of three (hover + wait + click). 3. Batch actions: New 'batch' action kind that accepts an array of actions to execute sequentially in a single tool call. Supports 'stopOnError' (default true) to control whether execution halts on first failure. Results are returned as an array. This eliminates the AI inference roundtrip between each action, dramatically reducing latency and token cost for multi-step flows. Addresses: #44431, #38844

…input validation, recursion limit Fixes all 4 issues raised by Greptile review: 1. Security: batch actions now respect evaluateEnabled flag. executeSingleAction and batchViaPlaywright accept evaluateEnabled param. evaluate and wait-with-fn inside batches are rejected when evaluateEnabled=false, matching the direct route guards. 2. Security: batch input validation. Each action in body.actions is validated as a plain object with a known kind string before dispatch. Applies same normalization as direct action handlers. 3. Perf: SELECTOR_ALLOWED_KINDS moved to module scope as a ReadonlySet<string> constant (was re-created on every request). 4. Security: max batch nesting depth of 5. Nested batch actions track depth and throw if MAX_BATCH_DEPTH exceeded, preventing call stack exhaustion from crafted payloads.

aisle-research-bot · 2026-03-13T21:16:44Z

🔒 Aisle Security Analysis

We found 3 potential security issue(s) in this PR:

#	Severity	Title
1	🟡 Medium	Unbounded `press` delayMs allows long-running keypresses (DoS)
2	🟡 Medium	Unbounded recursion in batch request normalization enables denial-of-service via stack overflow / event-loop blocking
3	🔵 Low	Untrusted selector/ref reflected into API error messages (log/terminal injection risk)

1. 🟡 Unbounded `press` delayMs allows long-running keypresses (DoS)

Property	Value
Severity	Medium
CWE	CWE-400
Location	`src/browser/pw-tools-core.interactions.ts:198-212`

Description

The press action accepts a user-controlled delayMs value that is not bounded and is forwarded into Playwright's keyboard.press() delay option.

Input source (batch): normalizeBatchAction for kind "press" uses toNumber(raw.delayMs) without any max bound.
Input source (single /act): /act kind "press" also reads delayMs = toNumber(body.delayMs) without bounds.
Sink: pressKeyViaPlaywright passes delayMs directly to page.keyboard.press() as delay.

This allows an authenticated caller to send an extremely large delayMs (e.g., hours), causing the request/worker to remain busy waiting on Playwright and potentially tying up browser/session resources, leading to a denial of service.

Vulnerable code:

await page.keyboard.press(key, {
  delay: Math.max(0, Math.floor(opts.delayMs ?? 0)),
});

Recommendation

Enforce an upper bound for press.delayMs (similar to click/wait bounds) before calling Playwright.

Option A (recommended): reuse the existing resolveBoundedDelayMs() helper and introduce a MAX_PRESS_DELAY_MS constant.

const MAX_PRESS_DELAY_MS = 5_000;

export async function pressKeyViaPlaywright(opts: {
  cdpUrl: string;
  targetId?: string;
  key: string;
  delayMs?: number;
}): Promise<void> {
  const key = String(opts.key ?? "").trim();
  if (!key) throw new Error("key is required");

  const delay = resolveBoundedDelayMs(opts.delayMs, "press delayMs", MAX_PRESS_DELAY_MS);

  const page = await getPageForTargetId(opts);
  ensurePageState(page);
  await page.keyboard.press(key, { delay });
}

Also apply the same max bound at the HTTP layer for batch normalization (normalizeBatchAction case "press") and the /act press handler, so invalid requests fail fast with a 400 response.

2. 🟡 Unbounded recursion in batch request normalization enables denial-of-service via stack overflow / event-loop blocking

Property	Value
Severity	Medium
CWE	CWE-674
Location	`src/browser/routes/agent.act.ts:140-146`

Description

The /act route's new batch support performs recursive normalization and counting of nested { kind: "batch", actions: [...] } structures without any nesting-depth limit, before rejecting oversized batches.

Key issues:

normalizeBatchAction() recursively normalizes nested batches with raw.actions.map(normalizeBatchAction).
Only after fully normalizing the nested structure, it calls countBatchActions(actions) which is also recursive.
The /act handler for kind: "batch" maps all body.actions through normalizeBatchAction before any top-level length check.

Although the server has an Express JSON body limit of 1mb (express.json({ limit: "1mb" })), a 1mb JSON payload can still contain very deep nesting (thousands of levels). This can trigger RangeError: Maximum call stack size exceeded or otherwise consume significant CPU synchronously, blocking the Node.js event loop during request handling.

Vulnerable code (recursive normalization + recursive counting):

case "batch": {
  const actions = Array.isArray(raw.actions) ? raw.actions.map(normalizeBatchAction) : [];
  // ...
  if (countBatchActions(actions) > MAX_BATCH_ACTIONS) {
    throw new Error(`batch exceeds maximum of ${MAX_BATCH_ACTIONS} actions`);
  }
  return { kind, actions, ... };
}

Top-level route normalization (no early array-length cap):

actions = Array.isArray(body.actions) ? body.actions.map(normalizeBatchAction) : [];

Impact:

An authenticated client (or any client if this service is exposed via a proxy/port-forward) can submit a deeply nested batch payload causing excessive synchronous recursion, potentially throwing a stack overflow error and/or consuming CPU, reducing availability (DoS).

Recommendation

Add explicit nesting-depth and action-count limits before recursion in the route-layer normalization.

Suggested approach:

Pass a depth parameter through normalization and reject requests beyond a small MAX_BATCH_DEPTH (align with Playwright layer’s MAX_BATCH_DEPTH = 5).
Enforce a cheap top-level array limit before mapping/normalizing.
Avoid recursion for counting/validation (use an explicit stack/queue) to prevent stack overflow.

Example (depth guard + early length cap):

const MAX_BATCH_DEPTH = 5;
const MAX_BATCH_ACTIONS = 100;

function normalizeBatchAction(value: unknown, depth = 0): BrowserActRequest {
  if (depth > MAX_BATCH_DEPTH) {
    throw new Error(`batch depth exceeds maximum of ${MAX_BATCH_DEPTH}`);
  }
  // ... same validation as today ...
  if (kind === "batch") {
    const rawActions = Array.isArray(raw.actions) ? raw.actions : [];
    if (rawActions.length > MAX_BATCH_ACTIONS) {
      throw new Error(`batch exceeds maximum of ${MAX_BATCH_ACTIONS} actions`);
    }
    const actions = rawActions.map((a) => normalizeBatchAction(a, depth + 1));
    return { kind: "batch", actions };
  }
  // ...
}

// In route handler (before mapping):
const rawActions = Array.isArray(body.actions) ? body.actions : [];
if (rawActions.length > MAX_BATCH_ACTIONS) {
  return jsonError(res, 400, `batch exceeds maximum of ${MAX_BATCH_ACTIONS} actions`);
}
const actions = rawActions.map((a) => normalizeBatchAction(a, 0));

Also consider keeping the existing express.json({ limit: "1mb" }) (already present) and optionally lowering it for this endpoint if appropriate.

3. 🔵 Untrusted selector/ref reflected into API error messages (log/terminal injection risk)

Property	Value
Severity	Low
CWE	CWE-117
Location	`src/browser/pw-tools-core.interactions.ts:882-888`

Description

User-controlled selector (and other ref-like strings) are incorporated verbatim into generated error messages via toAIFriendlyError(...) and then returned to callers (including batched responses).

This creates an output-injection surface:

Input: selector comes from the /act request body (or nested batch action objects) and is only trim()’d (requireRefOrSelector).
Propagation: the value is used as label = resolved.ref ?? resolved.selector! and passed into toAIFriendlyError(err, label).
Sink: toAIFriendlyError embeds the label inside quoted strings (e.g. Selector "${selector}" ...), and in batch mode the resulting err.message is returned to the caller as JSON ({ ok:false, error: message }).

If an attacker can influence selectors (directly via API calls, or indirectly if an agent derives selectors from untrusted page content/attributes), they can inject control characters (e.g. \n, \r) or terminal escape sequences into returned error strings. While JSON encoding prevents breaking JSON structure, this can still:

forge/mislead logs in downstream consumers that log these messages line-by-line,
cause terminal escape injection in CLIs that print error strings,
create prompt-injection-like contamination when these errors are fed back into an LLM context.

Vulnerable code (batch sink):

} catch (err) {
  const message = err instanceof Error ? err.message : String(err);
  results.push({ ok: false, error: message });
  if (opts.stopOnError !== false) {
    break;
  }
}

Recommendation

Sanitize/normalize selector/ref strings before embedding them into error messages and before returning them in batch results.

Suggested pattern (single-line + length bound + strip ANSI escapes):

function sanitizeLabel(label: string, maxLen = 200): string {
  return label
    // strip ANSI escape sequences
    .replace(/\u001b\[[0-9;]*m/g, "")
    // collapse control whitespace that can forge logs
    .replace(/[\r\n\t]/g, " ")
    .slice(0, maxLen);
}

Apply it at the boundary:

In toAIFriendlyError(error, selector) use sanitizeLabel(selector) before interpolating.
In batchViaPlaywright, return a normalized message, or better return structured errors { code, message } where message is sanitized.

Example:

} catch (err) {
  const raw = err instanceof Error ? err.message : String(err);
  results.push({ ok: false, error: sanitizeLabel(raw, 500) });
}

Analyzed PR: #45457 at commit 36d3e27

_{Last updated on: 2026-03-13T22:43:50Z}

greptile-apps · 2026-03-13T21:18:55Z

Greptile Summary

This PR normalizes batch action payloads before dispatch in the browser act route, adds CSS-selector targeting alongside ref for all major interaction types (click, type, hover, drag, scrollIntoView, select), introduces delayMs on click, threads evaluateEnabled through to batchViaPlaywright, and exposes batch results on the typed BrowserActResponse. The contract and unit tests are well-structured and cover the primary happy and error paths.

Key issues found:

Nested batch failures are silently swallowed (pw-tools-core.interactions.ts, lines 831–841): batchViaPlaywright never throws — it always resolves with { results: [...] }. The outer executeSingleAction therefore never enters the catch block for a nested batch, always recording { ok: true } regardless of inner failures. This breaks the semantics of stopOnError: true in a parent batch when nested batches contain failing actions.
normalizeBatchAction has no recursion depth guard (routes/agent.act.ts, line 352): Unlike batchViaPlaywright which caps nesting at MAX_BATCH_DEPTH = 5, the validation step recurses without limit. A deeply-nested payload causes a RangeError (caught and returned as 400), but the error message is cryptic rather than the user-friendly depth-exceeded message from the execution layer.
delayMs: 0 inconsistency (pw-tools-core.interactions.ts, line 84): Normalization includes delayMs: 0 (via !== undefined check), but execution skips the hover-then-delay step because if (opts.delayMs) is falsy for 0. Minor, but worth aligning with an explicit > 0 guard.

Confidence Score: 3/5

Mostly safe to merge, but the nested-batch failure propagation bug means stopOnError semantics are broken for nested batches, which could mask automation failures in production.
The selector targeting, delayMs, evaluateEnabled threading, and normalization logic are all well-implemented with appropriate existing-session guards. The main concern that drops the score is the logic bug where a nested batch's inner failures are invisible to the parent batch — callers relying on stopOnError to halt a sequence on error will not get that behavior if any step is itself a nested batch. The normalization depth gap and delayMs:0 issue are lower-severity style points.
src/browser/pw-tools-core.interactions.ts — specifically the executeSingleAction batch case (lines 831–841) and batchViaPlaywright error-recording loop (lines 860–870).

Comments Outside Diff (3)

src/browser/pw-tools-core.interactions.ts, line 831-841 (link)

Nested batch failures are silently swallowed by the parent batch

batchViaPlaywright never throws — it always returns { results: [...] }, even when inner actions fail. Because of this, executeSingleAction for the "batch" case never triggers the outer batch's catch block, and the outer loop unconditionally pushes { ok: true } for the nested batch at line 863.

This means:
- A nested batch with failing actions is invisible to the parent batch.
- The parent's stopOnError: true (the default) will not stop execution after a nested batch error — outer actions following the nested batch will still run.
- The final response only surfaces the outer results array; the inner failures are only accessible if the client drills into the nested batch's own results entry, which is not returned by the outer batch at all.
For example: outer batch [inner_batch_with_failing_click, outer_action_2] with stopOnError: true — outer_action_2 will still execute even if inner_batch_with_failing_click failed internally.

One approach is to check the inner results after the call and throw if any failed while stopOnError !== false:
```
case "batch": {
  const batchResult = await batchViaPlaywright({
    cdpUrl,
    targetId: effectiveTargetId,
    actions: action.actions,
    stopOnError: action.stopOnError,
    evaluateEnabled,
    depth: depth + 1,
  });
  const anyFailed = batchResult.results.some((r) => !r.ok);
  if (anyFailed) {
    const firstError = batchResult.results.find((r) => !r.ok)?.error ?? "nested batch failed";
    throw new Error(firstError);
  }
  break;
}
```
This would let the outer catch handle propagation naturally, respecting stopOnError.
src/browser/routes/agent.act.ts, line 351-364 (link)

normalizeBatchAction has no recursion depth guard during validation

batchViaPlaywright uses MAX_BATCH_DEPTH = 5 to cap execution depth, but normalizeBatchAction calls itself recursively with no equivalent limit. A pathologically nested payload (e.g., 50+ batch-in-batch levels) will recurse deeply during the validation pass. Node.js will eventually throw a RangeError: Maximum call stack size exceeded, which is caught by the try/catch in the route handler and returned as a 400 — but the error message is cryptic compared to the friendly "Batch nesting depth exceeds maximum of 5" seen at execution time.

Consider threading a depth parameter through normalizeBatchAction and throwing early when it exceeds MAX_BATCH_DEPTH:
```
function normalizeBatchAction(value: unknown, depth = 0): BrowserActRequest {
  if (depth > MAX_BATCH_DEPTH) {
    throw new Error(`Batch nesting depth exceeds maximum of ${MAX_BATCH_DEPTH}`);
  }
  // ...
  case "batch": {
    const actions = Array.isArray(raw.actions)
      ? raw.actions.map((a) => normalizeBatchAction(a, depth + 1))
      : [];
    // ...
  }
}
```
src/browser/pw-tools-core.interactions.ts, line 84-87 (link)

delayMs: 0 passes normalization but is silently ignored at execution

normalizeBatchAction in agent.act.ts includes delayMs when it is !== undefined (so delayMs: 0 makes it into the normalized action), but this truthiness check if (opts.delayMs) evaluates to false for 0. The hover-and-delay step is therefore skipped even when the caller explicitly passed delayMs: 0.

For consistency, prefer an explicit undefined check:

Prompt To Fix All With AI

This is a comment left during a code review.
Path: src/browser/pw-tools-core.interactions.ts
Line: 831-841

Comment:
**Nested batch failures are silently swallowed by the parent batch**

`batchViaPlaywright` never throws — it always returns `{ results: [...] }`, even when inner actions fail. Because of this, `executeSingleAction` for the `"batch"` case never triggers the outer batch's `catch` block, and the outer loop unconditionally pushes `{ ok: true }` for the nested batch at line 863.

This means:
- A nested batch with failing actions is invisible to the parent batch.
- The parent's `stopOnError: true` (the default) will **not** stop execution after a nested batch error — outer actions following the nested batch will still run.
- The final response only surfaces the outer `results` array; the inner failures are only accessible if the client drills into the nested batch's own `results` entry, which is not returned by the outer batch at all.

For example: outer batch `[inner_batch_with_failing_click, outer_action_2]` with `stopOnError: true` — `outer_action_2` will still execute even if `inner_batch_with_failing_click` failed internally.

One approach is to check the inner results after the call and throw if any failed while `stopOnError !== false`:

```ts
case "batch": {
  const batchResult = await batchViaPlaywright({
    cdpUrl,
    targetId: effectiveTargetId,
    actions: action.actions,
    stopOnError: action.stopOnError,
    evaluateEnabled,
    depth: depth + 1,
  });
  const anyFailed = batchResult.results.some((r) => !r.ok);
  if (anyFailed) {
    const firstError = batchResult.results.find((r) => !r.ok)?.error ?? "nested batch failed";
    throw new Error(firstError);
  }
  break;
}
```
This would let the outer `catch` handle propagation naturally, respecting `stopOnError`.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/browser/routes/agent.act.ts
Line: 351-364

Comment:
**`normalizeBatchAction` has no recursion depth guard during validation**

`batchViaPlaywright` uses `MAX_BATCH_DEPTH = 5` to cap execution depth, but `normalizeBatchAction` calls itself recursively with no equivalent limit. A pathologically nested payload (e.g., 50+ `batch`-in-`batch` levels) will recurse deeply during the validation pass. Node.js will eventually throw a `RangeError: Maximum call stack size exceeded`, which is caught by the `try/catch` in the route handler and returned as a 400 — but the error message is cryptic compared to the friendly `"Batch nesting depth exceeds maximum of 5"` seen at execution time.

Consider threading a `depth` parameter through `normalizeBatchAction` and throwing early when it exceeds `MAX_BATCH_DEPTH`:

```ts
function normalizeBatchAction(value: unknown, depth = 0): BrowserActRequest {
  if (depth > MAX_BATCH_DEPTH) {
    throw new Error(`Batch nesting depth exceeds maximum of ${MAX_BATCH_DEPTH}`);
  }
  // ...
  case "batch": {
    const actions = Array.isArray(raw.actions)
      ? raw.actions.map((a) => normalizeBatchAction(a, depth + 1))
      : [];
    // ...
  }
}
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/browser/pw-tools-core.interactions.ts
Line: 84-87

Comment:
**`delayMs: 0` passes normalization but is silently ignored at execution**

`normalizeBatchAction` in `agent.act.ts` includes `delayMs` when it is `!== undefined` (so `delayMs: 0` makes it into the normalized action), but this truthiness check `if (opts.delayMs)` evaluates to `false` for `0`. The hover-and-delay step is therefore skipped even when the caller explicitly passed `delayMs: 0`.

For consistency, prefer an explicit `undefined` check:

```suggestion
    if (opts.delayMs !== undefined && opts.delayMs > 0) {
      await locator.hover({ timeout });
      await new Promise((r) => setTimeout(r, opts.delayMs));
    }
```

How can I resolve this? If you propose a fix, please make it concise.

_{Last reviewed commit: 36d3e27}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3cd8e8376a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 36d3e27600

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-13T22:16:52Z

+      await batchViaPlaywright({
+        cdpUrl,
+        targetId: effectiveTargetId,
+        actions: action.actions,
+        stopOnError: action.stopOnError,


Propagate nested batch failures to the parent batch result

Handle nested batch outcomes explicitly here: this branch awaits batchViaPlaywright(...) but discards its returned results, while batchViaPlaywright internally catches action errors and reports them as { ok: false } instead of throwing. That means a nested batch with failing inner actions is recorded as { ok: true } by the outer loop, and stopOnError on the parent batch will not stop when nested actions fail, so callers can receive a successful top-level result even though part of the batch did not execute successfully.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-13T22:16:52Z

+            const result = await pw.batchViaPlaywright({
+              cdpUrl,
+              targetId: tab.targetId,
+              actions,
+              stopOnError,


Validate batch size before dispatching to Playwright

Add a route-level size check before dispatch: /act currently forwards any non-empty actions array, but batchViaPlaywright throws when actions.length > 100. That exception is a plain Error, so withRouteTabContext maps it to a 500 response, turning client input validation into an internal server error. A request with 101 actions should be rejected as a 4xx validation error instead of surfacing as a server fault.

Useful? React with 👍 / 👎.

…port (openclaw#45457) * feat(browser): add batch actions, CSS selector support, and click delayMs Adds three improvements to the browser act tool: 1. CSS selector support: All element-targeting actions (click, type, hover, drag, scrollIntoView, select) now accept an optional 'selector' parameter alongside 'ref'. When selector is provided, Playwright's page.locator() is used directly, skipping the need for a snapshot to obtain refs. This reduces roundtrips for agents that already know the DOM structure. 2. Click delay (delayMs): The click action now accepts an optional 'delayMs' parameter. When set, the element is hovered first, then after the specified delay, clicked. This enables human-like hover-before-click in a single tool call instead of three (hover + wait + click). 3. Batch actions: New 'batch' action kind that accepts an array of actions to execute sequentially in a single tool call. Supports 'stopOnError' (default true) to control whether execution halts on first failure. Results are returned as an array. This eliminates the AI inference roundtrip between each action, dramatically reducing latency and token cost for multi-step flows. Addresses: openclaw#44431, openclaw#38844 * fix(browser): address security review — batch evaluateEnabled guard, input validation, recursion limit Fixes all 4 issues raised by Greptile review: 1. Security: batch actions now respect evaluateEnabled flag. executeSingleAction and batchViaPlaywright accept evaluateEnabled param. evaluate and wait-with-fn inside batches are rejected when evaluateEnabled=false, matching the direct route guards. 2. Security: batch input validation. Each action in body.actions is validated as a plain object with a known kind string before dispatch. Applies same normalization as direct action handlers. 3. Perf: SELECTOR_ALLOWED_KINDS moved to module scope as a ReadonlySet<string> constant (was re-created on every request). 4. Security: max batch nesting depth of 5. Nested batch actions track depth and throw if MAX_BATCH_DEPTH exceeded, preventing call stack exhaustion from crafted payloads. * fix(browser): normalize batch act dispatch * fix(browser): tighten existing-session act typing * fix(browser): preserve batch type text * fix(browser): complete batch action execution * test(browser): cover batch route normalization * test(browser): cover batch interaction dispatch * fix(browser): bound batch route action inputs * fix(browser): harden batch interaction limits * test(browser): cover batch security guardrails --------- Co-authored-by: Diwakar <[email protected]>

…port (openclaw#45457) * feat(browser): add batch actions, CSS selector support, and click delayMs Adds three improvements to the browser act tool: 1. CSS selector support: All element-targeting actions (click, type, hover, drag, scrollIntoView, select) now accept an optional 'selector' parameter alongside 'ref'. When selector is provided, Playwright's page.locator() is used directly, skipping the need for a snapshot to obtain refs. This reduces roundtrips for agents that already know the DOM structure. 2. Click delay (delayMs): The click action now accepts an optional 'delayMs' parameter. When set, the element is hovered first, then after the specified delay, clicked. This enables human-like hover-before-click in a single tool call instead of three (hover + wait + click). 3. Batch actions: New 'batch' action kind that accepts an array of actions to execute sequentially in a single tool call. Supports 'stopOnError' (default true) to control whether execution halts on first failure. Results are returned as an array. This eliminates the AI inference roundtrip between each action, dramatically reducing latency and token cost for multi-step flows. Addresses: openclaw#44431, openclaw#38844 * fix(browser): address security review — batch evaluateEnabled guard, input validation, recursion limit Fixes all 4 issues raised by Greptile review: 1. Security: batch actions now respect evaluateEnabled flag. executeSingleAction and batchViaPlaywright accept evaluateEnabled param. evaluate and wait-with-fn inside batches are rejected when evaluateEnabled=false, matching the direct route guards. 2. Security: batch input validation. Each action in body.actions is validated as a plain object with a known kind string before dispatch. Applies same normalization as direct action handlers. 3. Perf: SELECTOR_ALLOWED_KINDS moved to module scope as a ReadonlySet<string> constant (was re-created on every request). 4. Security: max batch nesting depth of 5. Nested batch actions track depth and throw if MAX_BATCH_DEPTH exceeded, preventing call stack exhaustion from crafted payloads. * fix(browser): normalize batch act dispatch * fix(browser): tighten existing-session act typing * fix(browser): preserve batch type text * fix(browser): complete batch action execution * test(browser): cover batch route normalization * test(browser): cover batch interaction dispatch * fix(browser): bound batch route action inputs * fix(browser): harden batch interaction limits * test(browser): cover batch security guardrails --------- Co-authored-by: Diwakar <[email protected]> (cherry picked from commit f59b2b1)

…port (openclaw#45457) * feat(browser): add batch actions, CSS selector support, and click delayMs Adds three improvements to the browser act tool: 1. CSS selector support: All element-targeting actions (click, type, hover, drag, scrollIntoView, select) now accept an optional 'selector' parameter alongside 'ref'. When selector is provided, Playwright's page.locator() is used directly, skipping the need for a snapshot to obtain refs. This reduces roundtrips for agents that already know the DOM structure. 2. Click delay (delayMs): The click action now accepts an optional 'delayMs' parameter. When set, the element is hovered first, then after the specified delay, clicked. This enables human-like hover-before-click in a single tool call instead of three (hover + wait + click). 3. Batch actions: New 'batch' action kind that accepts an array of actions to execute sequentially in a single tool call. Supports 'stopOnError' (default true) to control whether execution halts on first failure. Results are returned as an array. This eliminates the AI inference roundtrip between each action, dramatically reducing latency and token cost for multi-step flows. Addresses: openclaw#44431, openclaw#38844 * fix(browser): address security review — batch evaluateEnabled guard, input validation, recursion limit Fixes all 4 issues raised by Greptile review: 1. Security: batch actions now respect evaluateEnabled flag. executeSingleAction and batchViaPlaywright accept evaluateEnabled param. evaluate and wait-with-fn inside batches are rejected when evaluateEnabled=false, matching the direct route guards. 2. Security: batch input validation. Each action in body.actions is validated as a plain object with a known kind string before dispatch. Applies same normalization as direct action handlers. 3. Perf: SELECTOR_ALLOWED_KINDS moved to module scope as a ReadonlySet<string> constant (was re-created on every request). 4. Security: max batch nesting depth of 5. Nested batch actions track depth and throw if MAX_BATCH_DEPTH exceeded, preventing call stack exhaustion from crafted payloads. * fix(browser): normalize batch act dispatch * fix(browser): tighten existing-session act typing * fix(browser): preserve batch type text * fix(browser): complete batch action execution * test(browser): cover batch route normalization * test(browser): cover batch interaction dispatch * fix(browser): bound batch route action inputs * fix(browser): harden batch interaction limits * test(browser): cover batch security guardrails --------- Co-authored-by: Diwakar <[email protected]>

vincentkoc self-assigned this Mar 13, 2026

openclaw-barnacle Bot added size: L maintainer Maintainer-authored PR labels Mar 13, 2026

diwakarrankawat and others added 3 commits March 13, 2026 14:11

fix(browser): normalize batch act dispatch

4e2932c

vincentkoc force-pushed the vincentkoc-code/browser-batch-route-fixes branch from 6a19174 to 4e2932c Compare March 13, 2026 21:15

vincentkoc marked this pull request as ready for review March 13, 2026 21:16

Merge branch 'main' into vincentkoc-code/browser-batch-route-fixes

3cd8e83

vincentkoc marked this pull request as draft March 13, 2026 21:20

chatgpt-codex-connector Bot reviewed Mar 13, 2026

View reviewed changes

Comment thread src/browser/routes/agent.act.ts

Comment thread src/browser/pw-tools-core.interactions.ts

Comment thread src/browser/routes/agent.act.ts Outdated

Comment thread src/browser/pw-tools-core.interactions.ts

vincentkoc added 5 commits March 13, 2026 14:25

fix(browser): tighten existing-session act typing

f8e2468

fix(browser): preserve batch type text

9bbce47

fix(browser): complete batch action execution

a1fe476

test(browser): cover batch route normalization

ef3b7fa

test(browser): cover batch interaction dispatch

af14033

openclaw-barnacle Bot added size: XL and removed size: L labels Mar 13, 2026

vincentkoc marked this pull request as ready for review March 13, 2026 22:05

vincentkoc added 4 commits March 13, 2026 18:05

Merge branch 'main' into vincentkoc-code/browser-batch-route-fixes

26eadd5

fix(browser): bound batch route action inputs

9041404

fix(browser): harden batch interaction limits

725ea1c

test(browser): cover batch security guardrails

36d3e27

vincentkoc merged commit f59b2b1 into main Mar 13, 2026
23 of 30 checks passed

vincentkoc deleted the vincentkoc-code/browser-batch-route-fixes branch March 13, 2026 22:10

chatgpt-codex-connector Bot reviewed Mar 13, 2026

View reviewed changes

github-actions Bot mentioned this pull request Mar 13, 2026

📡 Upstream Digest — 2026-03-13 22:21 UTC curtismercier/openclaw-mods#257

Open

github-actions Bot mentioned this pull request Mar 14, 2026

上游更新: v2026.3.13 — 16 P0 + 29 P1 待合并 mf-yang/openclaw-cn#510

Open

github-actions Bot mentioned this pull request Mar 15, 2026

📢 OpenClaw 新版本发布: v2026.3.13-1 xianyu110/awesome-openclaw-tutorial#32

Closed

5 tasks

diwakarrankawat mentioned this pull request Mar 18, 2026

feat(browser): batch actions, CSS selector support, and click delayMs #44934

Closed

This was referenced Apr 26, 2026

[Bug]: browser upload/file chooser flow is flaky and can misreport stale-click failures as browser control timeouts #38844

Closed

Browser tool: 7 improvements from real-world automation field test #44431

Open

steipete mentioned this pull request Jul 11, 2026

[Bug]: Browser composite waits and nested batches time out before all steps can finish #104457

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(browser): normalize batch act dispatch for selector and batch support#45457

fix(browser): normalize batch act dispatch for selector and batch support#45457
vincentkoc merged 13 commits into
mainfrom
vincentkoc-code/browser-batch-route-fixes

vincentkoc commented Mar 13, 2026

Uh oh!

aisle-research-bot Bot commented Mar 13, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Mar 13, 2026 •

edited

Loading

Comments Outside Diff (3)

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Mar 13, 2026

Uh oh!

chatgpt-codex-connector Bot Mar 13, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

vincentkoc commented Mar 13, 2026

Summary

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

User-visible / Behavior Changes

Security Impact (required)

Repro + Verification

Environment

Steps

Expected

Actual

Evidence

Human Verification (required)

Review Conversations

Compatibility / Migration

Failure Recovery (if this breaks)

Risks and Mitigations

Uh oh!

aisle-research-bot Bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Aisle Security Analysis

1. 🟡 Unbounded press delayMs allows long-running keypresses (DoS)

Description

Recommendation

2. 🟡 Unbounded recursion in batch request normalization enables denial-of-service via stack overflow / event-loop blocking

Description

Recommendation

3. 🔵 Untrusted selector/ref reflected into API error messages (log/terminal injection risk)

Description

Recommendation

Uh oh!

greptile-apps Bot commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Summary

Confidence Score: 3/5

Comments Outside Diff (3)

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Mar 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

aisle-research-bot Bot commented Mar 13, 2026 •

edited

Loading

1. 🟡 Unbounded `press` delayMs allows long-running keypresses (DoS)

greptile-apps Bot commented Mar 13, 2026 •

edited

Loading