Skip to content

Commit 943913a

Browse files
authored
fix(android): stop empty focusable overlays from hiding app content (#1737)
* fix(android): stop empty focusable overlays from hiding app content (#1733) The covered-subtree pruner let any `hittable` sibling condemn a lower drawing-order sibling it geometrically covers. `hittable` is `clickable ?? focusable`, so a childless, textless, id-less full-screen focusable View qualified as "agent-visible content" and dropped the sibling holding the real UI. Telegram wraps every screen in exactly such a View, which is why `snapshot` returned 1 node on every Telegram screen while the helper and stock `uiautomator dump` both saw the full tree — the loss was in the TS parser, not the Android helper. Key covering candidacy on `clickable` instead: focusability is an accessibility-traversal property and says nothing about painting over a sibling. Also generalise the existing "never condemn a marker leaf" exemption to any childless, non-clickable sibling that carries its own text or identifier, since drawing order plus geometry cannot distinguish a transparent overlay from an opaque one. * fix(android): keep descendant coverage classification on hittable Review feedback on #1733: narrowing hasActionableDescendant from `hittable` to `clickable` was over-reach. The helper emits clickable/focusable only when true, so a focusable-only control parses as clickable=undefined, hittable=true — and D-pad/TV surfaces are built almost entirely from those. A foreground surface containing them would have stopped qualifying as covering, leaving background selector/ref targets retained and actionable behind it. Descendant classification returns to the established `hittable` contract, and the leaf exemption returns to `!hittable` so a genuinely covered affordance stays condemned. The Telegram fix needs only to exclude the empty focusable wrapper itself, which is self-classification in hasOwnAgentVisibleContent — that stays on `clickable`. Replaces the test that asserted the rejected behavior with a helper-shaped regression: a foreground descendant carrying focusable without any clickable attribute must still suppress a covered target. * refactor(android): split clickable and focusable instead of collapsing them `hittable: clickable ?? focusable` collapsed two independent Android facts and made behavior depend on how a producer encodes a false attribute. The snapshot helper omits `clickable`; stock UiAutomator writes `clickable="false"`. For one focusable control those encodings gave opposite answers — verified: identical trees pruned differently and projected opposite `hittable` values. No comment can hold an invariant the representation contradicts, which is what the previous two commits tried to do. The tree now stores `clickable` and `focusable` as separate non-optional booleans, and every decision reads a named predicate: isTouchTarget, isFocusTarget, isAgentTarget, hasSemanticContent, hasDirectOcclusionEvidence, hasDescendantOcclusionEvidence, isPresentationLeaf. `canCoverSibling` consumes one derived classification (hasOcclusionEvidence) rather than choosing between raw attributes, so the self-versus-descendant substitution that caused the last review round is no longer expressible. Removing `hittable` from the tree type made the typechecker find every construction site; the public field is derived once at projection from isAgentTarget. Behavior change: a focusable, non-clickable control encoded by stock UiAutomator now projects hittable=true and participates in occlusion, matching what the helper backend already did for the same control. The Android TV test asserted the old encoding-specific value; it now asserts inclusion plus the unified projection. One encoding-parity regression replaces the patch-specific test added last round.
1 parent f18f8b0 commit 943913a

3 files changed

Lines changed: 176 additions & 37 deletions

File tree

‎src/__tests__/test-utils/android-ui-hierarchy-fixtures.ts‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,9 @@ function rawFixtureToTreeNode(raw: RawSnapshotNode): AndroidUiHierarchy {
3131
identifier: raw.identifier ?? null,
3232
packageName: raw.bundleId ?? null,
3333
rect: raw.rect,
34-
hittable: raw.hittable,
34+
// Fixtures speak the public projection; split it back into the two facts the tree stores.
35+
clickable: raw.hittable === true,
36+
focusable: false,
3537
visibleToUser: raw.visibleToUser,
3638
depth: 0,
3739
children: [],
@@ -45,6 +47,8 @@ function rawFixtureToAndroidTree(rawNodes: RawSnapshotNode[]): AndroidUiHierarch
4547
value: null,
4648
identifier: null,
4749
packageName: null,
50+
clickable: false,
51+
focusable: false,
4852
depth: -1,
4953
children: [],
5054
};

‎src/platforms/android/__tests__/ui-hierarchy.test.ts‎

Lines changed: 102 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -243,7 +243,43 @@ test('parseUiHierarchy keeps focused non-clickable Android TV nodes in interacti
243243
const focused = result.nodes.find((node) => node.label === 'Featured');
244244

245245
assert.equal(focused?.focused, true);
246-
assert.equal(focused?.hittable, false);
246+
// A D-pad control an agent can drive is a target regardless of which input model reaches it.
247+
assert.equal(focused?.hittable, true);
248+
});
249+
250+
test('parseUiHierarchy reads an omitted clickable attribute the same as clickable="false"', () => {
251+
// The snapshot helper omits false attributes; stock UiAutomator writes them out. Those are two
252+
// encodings of one control, so every derived answer — public projection and pruning alike — has
253+
// to agree. Reading an absent attribute as a value is what made them diverge.
254+
const tree = (clickable: string) => `<hierarchy>
255+
<node class="android.widget.FrameLayout" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="0">
256+
<node class="android.view.ViewGroup" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="2">
257+
<node class="android.widget.TextView" text="Foreground tile" bounds="[24,120][366,200]" enabled="true" visible-to-user="true" focusable="true" ${clickable} drawing-order="1"/>
258+
</node>
259+
<node class="android.view.ViewGroup" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="1">
260+
<node class="android.widget.TextView" text="Background row" bounds="[0,220][280,280]" enabled="true" visible-to-user="true" focusable="true" ${clickable} drawing-order="1"/>
261+
</node>
262+
</node>
263+
</hierarchy>`;
264+
265+
const omitted = parseUiHierarchy(tree(''), 800, { raw: true });
266+
const explicitFalse = parseUiHierarchy(tree('clickable="false"'), 800, { raw: true });
267+
268+
assert.deepEqual(
269+
explicitFalse.nodes.map((node) => [node.label, node.hittable]),
270+
omitted.nodes.map((node) => [node.label, node.hittable]),
271+
);
272+
// Both encodings must also agree that the foreground surface covers the background one.
273+
for (const result of [omitted, explicitFalse]) {
274+
assert.equal(
275+
result.nodes.some((node) => node.label === 'Foreground tile'),
276+
true,
277+
);
278+
assert.equal(
279+
result.nodes.some((node) => node.label === 'Background row'),
280+
false,
281+
);
282+
}
247283
});
248284

249285
test('parseUiHierarchy prunes Android nodes that are not visible to the user in raw snapshots', () => {
@@ -460,6 +496,71 @@ test('parseUiHierarchy keeps React Native content under a transparent Expo tools
460496
);
461497
});
462498

499+
test('parseUiHierarchy keeps app content under a childless focusable full-screen overlay', () => {
500+
// Telegram wraps every screen in a childless full-screen focusable View drawn above the content
501+
// container. Focusability is accessibility traversal, not paint (#1733).
502+
const xml = `<hierarchy>
503+
<node class="android.widget.FrameLayout" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="0">
504+
<node class="android.view.View" bounds="[0,0][390,844]" enabled="true" visible-to-user="true" focusable="true" drawing-order="3"/>
505+
<node class="android.widget.FrameLayout" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="1">
506+
<node class="android.widget.TextView" text="Your phone number" bounds="[24,200][366,260]" enabled="true" visible-to-user="true" drawing-order="1"/>
507+
<node class="android.widget.EditText" text="208 379 7171" bounds="[24,320][366,380]" clickable="true" focusable="true" enabled="true" visible-to-user="true" drawing-order="2"/>
508+
</node>
509+
</node>
510+
</hierarchy>`;
511+
512+
const result = parseUiHierarchy(xml, 800, { raw: true });
513+
assert.equal(
514+
result.nodes.some((node) => node.label === 'Your phone number'),
515+
true,
516+
);
517+
assert.equal(
518+
result.nodes.some((node) => node.label === '208 379 7171'),
519+
true,
520+
);
521+
});
522+
523+
test('parseUiHierarchy keeps an overlapped text leaf drawn inside a composite widget sibling', () => {
524+
// Telegram renders the "+" of "+1" as a TextView sitting inside the country-code EditText's box.
525+
const xml = `<hierarchy>
526+
<node class="android.widget.LinearLayout" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="0">
527+
<node class="android.widget.EditText" text="1" content-desc="Country code" bounds="[40,300][130,380]" clickable="true" focusable="true" enabled="true" visible-to-user="true" drawing-order="2"/>
528+
<node class="android.widget.TextView" text="+" bounds="[40,310][55,370]" enabled="true" visible-to-user="true" drawing-order="1"/>
529+
</node>
530+
</hierarchy>`;
531+
532+
const result = parseUiHierarchy(xml, 800, { raw: true });
533+
assert.equal(
534+
result.nodes.some((node) => node.label === '+'),
535+
true,
536+
);
537+
assert.equal(
538+
result.nodes.some((node) => node.label === '1'),
539+
true,
540+
);
541+
});
542+
543+
test('parseUiHierarchy still condemns a clickable leaf covered by a foreground sibling', () => {
544+
const xml = `<hierarchy>
545+
<node class="android.widget.FrameLayout" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="0">
546+
<node class="android.view.ViewGroup" bounds="[0,0][390,844]" visible-to-user="true" drawing-order="2">
547+
<node class="android.widget.Button" text="Modal action" bounds="[24,420][366,480]" clickable="true" enabled="true" visible-to-user="true" drawing-order="1"/>
548+
</node>
549+
<node class="android.widget.Button" text="Behind the modal" bounds="[0,220][280,280]" clickable="true" enabled="true" visible-to-user="true" drawing-order="1"/>
550+
</node>
551+
</hierarchy>`;
552+
553+
const result = parseUiHierarchy(xml, 800, { raw: true });
554+
assert.equal(
555+
result.nodes.some((node) => node.label === 'Modal action'),
556+
true,
557+
);
558+
assert.equal(
559+
result.nodes.some((node) => node.label === 'Behind the modal'),
560+
false,
561+
);
562+
});
563+
463564
test('parseUiHierarchy ignores attribute-name prefix spoofing', () => {
464565
const xml =
465566
"<hierarchy><node class='android.widget.TextView' hint-text='Spoofed' text='Actual' bounds='[10,20][110,60]'/></hierarchy>";

‎src/platforms/android/ui-hierarchy.ts‎

Lines changed: 69 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -193,7 +193,7 @@ function walkUiHierarchyNode(
193193
const currentIndex = include
194194
? appendAndroidSnapshotNode(state, node, parentIndex, systemChrome)
195195
: parentIndex;
196-
const nextAncestorHittable = ancestorHittable || Boolean(node.hittable);
196+
const nextAncestorHittable = ancestorHittable || isAgentTarget(node);
197197
const nextAncestorCollection = ancestorCollection || isCollectionContainerType(node.type);
198198
for (const child of node.children) {
199199
walkUiHierarchyNode(
@@ -232,7 +232,7 @@ function appendAndroidSnapshotNode(
232232
enabled: node.enabled,
233233
focused: node.focused,
234234
visibleToUser: node.visibleToUser,
235-
hittable: node.hittable,
235+
hittable: isAgentTarget(node) || undefined,
236236
depth: compactedAndroidNodeDepth(state.nodes, parentIndex),
237237
parentIndex,
238238
...(node.hiddenContentAbove ? { hiddenContentAbove: true } : {}),
@@ -255,7 +255,7 @@ function hasInteractiveDescendant(state: AndroidSnapshotBuildState, node: Androi
255255
for (const child of node.children) {
256256
if (
257257
child.visibleToUser !== false &&
258-
(child.hittable || hasInteractiveDescendant(state, child))
258+
(isAgentTarget(child) || hasInteractiveDescendant(state, child))
259259
) {
260260
state.interactiveDescendantMemo.set(node, true);
261261
return true;
@@ -421,7 +421,11 @@ export type AndroidUiHierarchy = {
421421
visibleToUser?: boolean;
422422
drawingOrder?: number;
423423
focused?: boolean;
424-
hittable?: boolean;
424+
// Two independent facts, never collapsed, and never undefined: the helper omits false attributes
425+
// while stock UiAutomator writes them out, so reading an absent attribute as a value gave two
426+
// encodings of one control opposite answers.
427+
clickable: boolean;
428+
focusable: boolean;
425429
depth: number;
426430
parentIndex?: number;
427431
hiddenContentAbove?: boolean;
@@ -466,6 +470,8 @@ export function parseUiHierarchyTree(xml: string): AndroidUiHierarchy {
466470
value: null,
467471
identifier: null,
468472
packageName: null,
473+
clickable: false,
474+
focusable: false,
469475
depth: -1,
470476
children: [],
471477
};
@@ -492,7 +498,8 @@ export function parseUiHierarchyTree(xml: string): AndroidUiHierarchy {
492498
focused: attrs.focused,
493499
visibleToUser: attrs.visibleToUser,
494500
drawingOrder: attrs.drawingOrder,
495-
hittable: attrs.clickable ?? attrs.focusable,
501+
clickable: attrs.clickable === true,
502+
focusable: attrs.focusable === true,
496503
scrollable: attrs.scrollable,
497504
canScrollForward: attrs.canScrollForward,
498505
canScrollBackward: attrs.canScrollBackward,
@@ -523,6 +530,53 @@ export function parseUiHierarchyTree(xml: string): AndroidUiHierarchy {
523530
return root;
524531
}
525532

533+
/** A node a touch can act on. */
534+
function isTouchTarget(node: AndroidNode): boolean {
535+
return node.clickable;
536+
}
537+
538+
/** A node D-pad/keyboard traversal can land on. Normal for TV controls, which are rarely clickable. */
539+
function isFocusTarget(node: AndroidNode): boolean {
540+
return node.focusable || node.focused === true;
541+
}
542+
543+
/** A node an agent can drive by either input model. This is what the public `hittable` projects. */
544+
function isAgentTarget(node: AndroidNode): boolean {
545+
return isTouchTarget(node) || isFocusTarget(node);
546+
}
547+
548+
/** Text or an address an agent can read or select by. */
549+
function hasSemanticContent(node: AndroidNode): boolean {
550+
return hasMeaningfulLabel(node) || hasMeaningfulIdentifier(node);
551+
}
552+
553+
/** Focusability is traversal, not paint, so it is not evidence a node hides anything (#1733). */
554+
function hasDirectOcclusionEvidence(node: AndroidNode): boolean {
555+
return node.visibleToUser !== false && (isTouchTarget(node) || hasSemanticContent(node));
556+
}
557+
558+
/** Evidence the node is a real surface because it contains something an agent could drive. */
559+
function hasDescendantOcclusionEvidence(node: AndroidNode, state: AndroidTreePruneState): boolean {
560+
const cached = state.actionableContentMemo.get(node);
561+
if (cached !== undefined) return cached;
562+
const result = node.children.some(
563+
(child) =>
564+
child.visibleToUser !== false &&
565+
(isAgentTarget(child) || hasDescendantOcclusionEvidence(child, state)),
566+
);
567+
state.actionableContentMemo.set(node, result);
568+
return result;
569+
}
570+
571+
/**
572+
* A childless sibling that only presents: an RN screen-level testID, or a label drawn inside a
573+
* higher sibling's box (Telegram's `+` over the country-code EditText). Geometry cannot tell a
574+
* transparent overlay from an opaque one, and exempting a leaf cannot resurrect a covered surface.
575+
*/
576+
function isPresentationLeaf(node: AndroidNode): boolean {
577+
return node.children.length === 0 && !isAgentTarget(node) && hasSemanticContent(node);
578+
}
579+
526580
function pruneAndroidInvisibleSubtrees(node: AndroidNode): void {
527581
let keptCount = 0;
528582
for (const child of node.children) {
@@ -554,16 +608,10 @@ function shouldKeepAndroidSibling(
554608
coveringCandidates: AndroidCoveringCandidate[],
555609
): boolean {
556610
return (
557-
isSemanticIdentifierMarker(node) ||
558-
!isCoveredByHigherDrawingOrderSibling(node, coveringCandidates)
611+
isPresentationLeaf(node) || !isCoveredByHigherDrawingOrderSibling(node, coveringCandidates)
559612
);
560613
}
561614

562-
function isSemanticIdentifierMarker(node: AndroidNode): boolean {
563-
// RN can emit a screen-level testID as an empty sibling beside its rendered navigator.
564-
return hasMeaningfulIdentifier(node) && !node.hittable && node.children.length === 0;
565-
}
566-
567615
function isCoveredByHigherDrawingOrderSibling(
568616
node: AndroidNode,
569617
coveringCandidates: AndroidCoveringCandidate[],
@@ -595,31 +643,18 @@ function canCoverSibling(
595643
node.visibleToUser !== false &&
596644
node.drawingOrder !== undefined &&
597645
hasPositiveRect(node) &&
598-
(hasOwnAgentVisibleContent(node) || hasActionableDescendant(node, state))
646+
hasOcclusionEvidence(node, state)
599647
);
600648
}
601649

602-
function hasOwnAgentVisibleContent(node: AndroidNode): boolean {
603-
if (node.visibleToUser === false) return false;
604-
if (node.hittable) return true;
605-
const label = node.label?.trim() ?? '';
606-
if (label && !isGenericAndroidId(label)) return true;
607-
const identifier = node.identifier?.trim() ?? '';
608-
if (identifier && !isGenericAndroidId(identifier)) return true;
609-
return false;
650+
/** The single occlusion classification. Covering is never re-derived from a raw attribute. */
651+
function hasOcclusionEvidence(node: AndroidNode, state: AndroidTreePruneState): boolean {
652+
return hasDirectOcclusionEvidence(node) || hasDescendantOcclusionEvidence(node, state);
610653
}
611654

612-
function hasActionableDescendant(node: AndroidNode, state: AndroidTreePruneState): boolean {
613-
const cached = state.actionableContentMemo.get(node);
614-
if (cached !== undefined) return cached;
615-
616-
const result = node.children.some(
617-
(child) =>
618-
child.visibleToUser !== false &&
619-
(Boolean(child.hittable) || hasActionableDescendant(child, state)),
620-
);
621-
state.actionableContentMemo.set(node, result);
622-
return result;
655+
function hasMeaningfulLabel(node: AndroidNode): boolean {
656+
const label = node.label?.trim() ?? '';
657+
return Boolean(label && !isGenericAndroidId(label));
623658
}
624659

625660
function hasPositiveRect(node: AndroidNode): node is AndroidNode & { rect: Rect } {
@@ -752,8 +787,7 @@ function shouldIncludeInteractiveAndroidNode(
752787
ancestorCollection: boolean,
753788
): boolean {
754789
if (hasNonPositiveRect(node)) return false;
755-
if (node.focused) return true;
756-
if (node.hittable) return true;
790+
if (isAgentTarget(node)) return true;
757791
if (isScrollableType(info.type) && descendantHittable) return true;
758792
return shouldIncludeInteractiveProxyNode(
759793
info,
@@ -784,7 +818,7 @@ function shouldIncludeStructuralAndroidNode(
784818
info: AndroidNodeInclusionInfo,
785819
descendantHittable: boolean,
786820
): boolean {
787-
if (node.hittable) return true;
821+
if (isAgentTarget(node)) return true;
788822
if (info.hasMeaningfulText) return true;
789823
if (info.hasMeaningfulId) return true;
790824
return descendantHittable;

0 commit comments

Comments
 (0)