Repository navigation
Closed
Labels
Description
Purpose
When an interaction is refused because the target is covered (reason: target_covered, node interactionBlocked: 'covered'), the geometric occlusion scan in packages/capture-kit/src/snapshot-occlusion.ts already knows which node covers the target (findCoveringNode returns it), but the annotation pass publishes only the boolean and the refusal carries nothing about the culprit. The agent must re-guess what to dismiss. Playwright's equivalent refusal names the intercepting element; agent-device should name the covering element too.
Required behavior
- The occlusion annotation pass carries a bounded description of the covering node on the covered node (e.g.
coveredBy: { kind, label?, rect }), derived from the nodefindCoveringNodealready found. - Both covered-refusal producers surface it:
src/commands/interaction/runtime/target-visibility-stages.ts#buildCoveredInteractionError(press/fill/click/longpress/hover via ref, selector, and native-ref preflight, plus the selector-readiness diagnosis probe);src/daemon/interaction/internal/find.ts#rejectCoveredFindMatch(find focus/type).
- The refusal
error.detailsgains a typedcoveredBydetail (bounded: presented role, trimmed/truncated label when the cover has one, rounded rect), and thehintnames the covering element so the recovery is concrete. - The cover's
@refis never advertised as actionable in the message, detail, or hint. - When the cover identity is unknown — backend policy annotations such as
annotateSnapshotNodesCoveredByPolicy(Android replacement surfaces) supply only a boolean — the refusal keeps today's shape: nocoveredBydetail, no guessed culprit. - Snapshot unchanged-comparison (
src/commands/capture/runtime/snapshot-unchanged.ts) treats the new node field consistently with the other presentation annotations it compares. - Update the owning help/docs recovery text that describes interaction refusals.
Observable completion conditions
- Unit tests through the public entry points:
annotateCoveredSnapshotNodesmarks a covered node with the covering node's kind/label/rect;annotateSnapshotNodesCoveredByPolicyleavescoveredByabsent.- A runtime press of a covered ref/selector rejects with
details.coveredBynaming the cover and a hint that names it; a policy-annotated covered node refuses withoutcoveredBy. - Label truncation is bounded.
- Existing refusal-shape assertions (message,
interactionBlocked,reason: target_covered) still pass. pnpm check:affected --runpasses on the pushed head.
Part of the research sweep comparing agent-device refusals with Playwright's "intercepts pointer events" diagnostics.
Activity
fukalous commentedon Oct 12, 2026
I’d like to take this issue. I’ll propagate a bounded
coveredBypayload fromfindCoveringNodeinto the target-covered refusals, including the covering element’s kind, label, and rect. I’ll update both refusal producers, unchanged snapshot comparison behavior, public-entry tests, and the related help/docs.fukalous commentedon Oct 12, 2026
I’ve now found #3495, which already implements this issue on the maintainer branch. I won’t duplicate that work; I’ll leave this one to the existing PR.
thymikee commentedon Oct 12, 2026
no worries, feel free to provide verification for it or contribute something else :) thanks!
feat(interaction): name the covering element in covered refusals
feat(interaction): name the covering element in covered refusals (cal…