Skip to content

Commit 342248f

Browse files
committed
fix(contracts): refuse to measure keyboard geometry that arrived rotated
Verified on iPhone 17 Pro with the landscape keyboard up: the screenshot shows the keyboard full width across the bottom 327 pt of an 874 x 402 viewport, and both the regular and the interactive projection report the key plane as 162 x 327 with the QWERTY columns running down the short axis and the dock's globe button at y 8. iOS hands over the landscape keyboard's rects in the keyboard's own rotated space, so a band derived from them would cover a 141 pt strip on the left edge from y 77 down — refusing a headline the keyboard is nowhere near while letting a tab-bar item at (264, 360) tap straight through into a key. A docked keyboard is wide in both orientations, 402 x 233 in measured portrait and 874 x 327 in measured landscape, so geometry that comes back taller than it is wide is refused measurement and the tap fails open, which is the same stance an unresolvable viewport gets. Landscape coverage is not lost, only iOS's rotated read of it: the table now pins the measured rotated tree as `undetermined` and pins a wide, docked landscape keyboard as measurable against the landscape viewport's own bottom edge. Recording the misfire rather than hiding it: `#2589`'s silent tap-into-a-key stays open on landscape iOS until the Apple capture reports keyboard rects in the app's orientation space. That is a capture-side fix, not a classifier rule, and gets its own issue.
1 parent 6e13b1c commit 342248f

2 files changed

Lines changed: 100 additions & 25 deletions

File tree

‎contracts/fixtures/tap-keyboard-occlusion-policy.json‎

Lines changed: 91 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"description": "Golden table for keyboard occlusion on interaction tap points (#2589), consumed by packages/contracts/src/tap-keyboard-occlusion.ts. A visible software keyboard is its own system surface: it is never a covering sibling of app content, so the same-window occlusion classifier cannot see it, and it leaves the app window rect intact, so the viewport rule passes a covered tab bar. The band runs from the topmost keyboard node down to the bottom of the viewport, across the keyboard's own columns; the tap point is the target's rect center, matching isTapPointInsideViewport. `nodes` are captured snapshot nodes as they appear in the projection the interaction resolved against — note that regular and interactive projections prune the keyboard's own container, leaving Key nodes beside app chrome. The band is derived only from keyboard geometry that is docked to the viewport bottom edge: measured on iPhone 17 Pro (iOS 26.2) the reported key plane stops 58 pt short of an 874 pt portrait viewport and lands exactly on a 402 pt landscape one, so an 80 pt docking budget is what separates a docked keyboard from an iPad floating or split keyboard, whose band stays unmeasured and fails open. `target` names either a resolved node index or a bare coordinate point; both paths read one decision and choose their own consequence.",
2+
"description": "Golden table for keyboard occlusion on interaction tap points (#2589), consumed by packages/contracts/src/tap-keyboard-occlusion.ts. A visible software keyboard is its own system surface: it is never a covering sibling of app content, so the same-window occlusion classifier cannot see it, and it leaves the app window rect intact, so the viewport rule passes a covered tab bar. The band runs from the topmost keyboard node down to the bottom of the viewport, across the keyboard's own columns; the tap point is the point the interaction dispatches: the rect center the platform aims at for a ref handed over by name, or the touch point a parent with interactive descendants resolves instead. `nodes` are captured snapshot nodes as they appear in the projection the interaction resolved against — note that regular and interactive projections prune the keyboard's own container, leaving Key nodes beside app chrome. The band is derived only from keyboard geometry that is docked to the viewport bottom edge: measured on iPhone 17 Pro (iOS 26.2) the reported key plane stops 58 pt short of an 874 pt portrait viewport and lands exactly on a 402 pt landscape one, so an 80 pt docking budget is what separates a docked keyboard from an iPad floating or split keyboard, whose band stays unmeasured and fails open. A docked keyboard is also wide: geometry that comes back taller than it is wide did not arrive in the app orientation space — iOS reports the landscape iPhone keyboard in the keyboard's own rotated space — and is refused measurement for the same reason. `target` names either a resolved node index or a bare coordinate point; both paths read one decision and choose their own consequence.",
33
"constants": {
44
"occlusionReason": "tap_keyboard_occludes_target"
55
},
@@ -1318,7 +1318,7 @@
13181318
}
13191319
},
13201320
{
1321-
"name": "a landscape iPhone keyboard fills most of the viewport height and is still the system keyboard",
1321+
"name": "the landscape iPhone keyboard reports its rects in its own rotated space, which cannot be measured",
13221322
"viewport": {
13231323
"x": 0,
13241324
"y": 0,
@@ -1343,27 +1343,40 @@
13431343
"parentIndex": 0,
13441344
"rect": {
13451345
"x": 0,
1346-
"y": 330,
1346+
"y": 338,
13471347
"width": 874,
1348-
"height": 72
1348+
"height": 64
13491349
}
13501350
},
13511351
{
13521352
"index": 2,
13531353
"type": "Button",
1354-
"label": "Form",
1354+
"label": "Home",
13551355
"parentIndex": 1,
13561356
"rect": {
1357-
"x": 390,
1358-
"y": 338,
1359-
"width": 94,
1360-
"height": 56
1357+
"x": 223,
1358+
"y": 342,
1359+
"width": 82,
1360+
"height": 36
13611361
}
13621362
},
13631363
{
13641364
"index": 3,
1365-
"type": "Keyboard",
1366-
"label": "Padding-Left",
1365+
"type": "Button",
1366+
"role": "UIKeyboardDockItemButton",
1367+
"label": "Next keyboard",
1368+
"parentIndex": 0,
1369+
"rect": {
1370+
"x": 0,
1371+
"y": 8,
1372+
"width": 65,
1373+
"height": 68
1374+
}
1375+
},
1376+
{
1377+
"index": 4,
1378+
"type": "Other",
1379+
"role": "UIKeyboardLayoutStar",
13671380
"parentIndex": 0,
13681381
"rect": {
13691382
"x": 2,
@@ -1373,10 +1386,11 @@
13731386
}
13741387
},
13751388
{
1376-
"index": 4,
1389+
"index": 5,
13771390
"type": "Key",
1378-
"label": "Q",
1379-
"parentIndex": 3,
1391+
"role": "UIAccessibilityElementKBKey",
1392+
"label": "q",
1393+
"parentIndex": 4,
13801394
"rect": {
13811395
"x": 124,
13821396
"y": 77,
@@ -1385,28 +1399,80 @@
13851399
}
13861400
},
13871401
{
1388-
"index": 5,
1402+
"index": 6,
13891403
"type": "Key",
1390-
"label": "return",
1391-
"parentIndex": 3,
1404+
"role": "UIAccessibilityElementKBKey",
1405+
"label": "t",
1406+
"parentIndex": 4,
13921407
"rect": {
1393-
"x": 700,
1394-
"y": 330,
1395-
"width": 150,
1396-
"height": 72
1408+
"x": 124,
1409+
"y": 366,
1410+
"width": 35,
1411+
"height": 36
13971412
}
13981413
}
13991414
],
14001415
"target": {
14011416
"index": 2
14021417
},
1418+
"expected": {
1419+
"kind": "undetermined"
1420+
}
1421+
},
1422+
{
1423+
"name": "a landscape keyboard reported wide and docked is measured against the landscape viewport",
1424+
"viewport": {
1425+
"x": 0,
1426+
"y": 0,
1427+
"width": 2400,
1428+
"height": 1080
1429+
},
1430+
"nodes": [
1431+
{
1432+
"index": 0,
1433+
"type": "android.widget.FrameLayout",
1434+
"bundleId": "com.example.app",
1435+
"rect": {
1436+
"x": 0,
1437+
"y": 0,
1438+
"width": 2400,
1439+
"height": 1080
1440+
}
1441+
},
1442+
{
1443+
"index": 1,
1444+
"type": "android.widget.Button",
1445+
"label": "Submit",
1446+
"bundleId": "com.example.app",
1447+
"rect": {
1448+
"x": 1100,
1449+
"y": 900,
1450+
"width": 200,
1451+
"height": 100
1452+
}
1453+
},
1454+
{
1455+
"index": 2,
1456+
"type": "android.widget.FrameLayout",
1457+
"bundleId": "com.google.android.inputmethod.latin",
1458+
"rect": {
1459+
"x": 0,
1460+
"y": 700,
1461+
"width": 2400,
1462+
"height": 380
1463+
}
1464+
}
1465+
],
1466+
"target": {
1467+
"index": 1
1468+
},
14031469
"expected": {
14041470
"kind": "occluded",
14051471
"frame": {
1406-
"x": 2,
1407-
"y": 75,
1408-
"width": 848,
1409-
"height": 327
1472+
"x": 0,
1473+
"y": 700,
1474+
"width": 2400,
1475+
"height": 380
14101476
}
14111477
}
14121478
},

‎packages/contracts/src/tap-keyboard-occlusion.ts‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,15 @@ function resolveVisibleKeyboardSurface(
187187
const bottomEdge = viewport.y + viewport.height;
188188
const reportedBottom = Math.max(...reportedRects.map((rect) => rect.y + rect.height));
189189
if (reportedBottom < bottomEdge - KEYBOARD_BOTTOM_ANCHOR_TOLERANCE) return null;
190+
// Reported geometry that arrives taller than it is wide is not in the app's orientation space. iOS
191+
// gives up the landscape iPhone keyboard's rects in the keyboard's own rotated space: measured on
192+
// iPhone 17 Pro, its key plane is 162 x 327 and its dock button reports y 8 of a 402 pt viewport,
193+
// while the screenshot shows the keyboard full width across the bottom 327 pt. A band from that
194+
// would refuse app content the keyboard is nowhere near while missing the keyboard itself, which is
195+
// worse than not measuring — see the landscape cases in the golden table.
196+
const reportedLeft = Math.min(...reportedRects.map((rect) => rect.x));
197+
const reportedRight = Math.max(...reportedRects.map((rect) => rect.x + rect.width));
198+
if (reportedRight - reportedLeft <= reportedBottom - minY) return null;
190199
const minX = Math.min(...anchorRects.map((rect) => rect.x));
191200
const maxRight = Math.max(...anchorRects.map((rect) => rect.x + rect.width));
192201
const planes = collectKeyboardPlaneIndices(nodes, surfaceNodes);

0 commit comments

Comments
 (0)