Skip to content

fix(mobile): stop dropping keyboard taps between rows - #2388

Merged
Chadpiha merged 4 commits into
OpenWhispr:mainfrom
Ker102:fix/ios-keyboard-touch-routing
Oct 7, 2026
Merged

Chadpiha merged 4 commits into
OpenWhispr:mainfrom
Ker102:fix/ios-keyboard-touch-routing

Conversation

@Ker102

@Ker102 Ker102 commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Fixes dropped taps on the iPhone keyboard: taps in the gaps between rows, in the margins at the ends of the a–l row, and in the keyboard's side and bottom padding now go to the nearest key, as on the system keyboard. On iPhones without a globe key, 123/ABC now lines up with the letter rows.

124 UIKit checks, built for Mac Catalyst and run in mobile CI, cover every half point across the keyboard below the dictation strip. The original code fails most of the routing checks. The iOS keyboard builds.

Partially addresses #2389: the missed taps are fixed, but the typing lag still needs profiling on a real iPhone. Still needs device testing. Expo Doctor flags an existing dependency mismatch.

@Ker102
Ker102 marked this pull request as ready for review September 29, 2026 12:20
@Ker102

Ker102 commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Hello, checking on this

@Chadpiha Chadpiha self-assigned this Oct 6, 2026
- Fall back to UIKit's default hit-test target when no key claims a touch,
  so a non-key control under the rows stays tappable
- Let the a and l keys take taps in the home row's 18 pt inset
- Drop the mode key's leading gap when the globe key is hidden, so 123/ABC
  lines up with the letter rows and the band to its left takes taps
- Make the hidden, transparent, noninteractive and disabled-key checks
  able to fail, and add edge-tie, wide-key and plain-button cases
- Sweep every point of the real keyboard layout at four widths, in letters
  and numbers mode, with and without the globe key
- Build the tests for Mac Catalyst at the extension's 15.1 target instead
  of booting a simulator, and remove the step from mobile CI
- Make the root padding a layout margin so the stack spans the keyboard,
  and let the rows take touches in the side and bottom padding and the
  lower half of the gap below the dictation strip
- Size each row key's target to reach its neighbours, closing the
  pixel-rounding slivers between equal-width keys
- Sweep the whole keyboard below the dictation strip from the root view
  at half-point steps, and check the strip's half of the gap and the
  padding layout
- Run the keyboard touch tests in mobile CI again
- Keep non-ASCII checkout paths valid in the runner's #sourceLocation
GHSA-pqg4-j6r4-53mv (critical, published 2026-10-06) covers shell-quote
>=1.8.4 <1.11.0. OpenWhispr#2532 cleared it for desktop with an override, but the
mobile lockfile still pins 1.9.0 (via react-devtools-core), so Mobile
CI's audit fails on every PR that touches mobile.

Move it to 1.12.0, within react-devtools-core's ^1.6.1 range and the
same version desktop uses.

@Chadpiha Chadpiha left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving: the routing fix is correct and well tested.

Root cause: each row's container is exactly one row tall, so UIKit never asked the keys about a touch in the gap between rows, the home-row margins or the keyboard's padding. About a fifth of the key area took no taps. KeyboardRowsStack now sends those taps to the nearest visible, enabled key, and the root padding is a layout margin, so the side and bottom edges reach the rows too.

Verified:

124/124 UIKit checks pass when built for Mac Catalyst, and mobile CI now runs them.
The sweep covers every half point across the keyboard below the dictation strip at 360, 393, 430 and 852 pt, in letters and numbers modes, with and without the globe key.
Each routing rule was broken on purpose (15 changes in total), and every change made the tests fail.
The keyboard still compiles for iOS.
Nothing that worked before broke: held delete, double-tap shift, the globe key's input menu, 123/ABC, space and return work as before, and the dictation strip and overlays keep their taps.

@Chadpiha
Chadpiha merged commit 63235b5 into OpenWhispr:main Oct 7, 2026
20 checks passed
Chadpiha added a commit that referenced this pull request Oct 22, 2026
* feat(mobile): rebuild the iOS keyboard to Apple's iOS 26 layout

The keys now match Apple's iOS 26 keyboard on every iPhone width, in light
and dark, measured against Apple's keyboard in the same text field:

- Key sizes, gaps, edge margins, corner curves and row positions; one key
  colour; lowercase letters in Apple's keyboard font (SF Compact) until shift.
- Apple's 123 and #+= pages with curly quotes; space returns to letters once
  something was typed there, the apostrophe at once.
- A bottom row that follows the field: a blue Search/Go/Send/Done return key,
  greyed while empty, and Apple's extra keys for URL ("." "/" ".com"), email
  ("@" "."), web-search and Twitter fields. This brings back the ".com" and
  "@" keys removed in #2359, to match Apple's layout.
- Apple's key pop-ups, which stay up ~50ms after release as Apple's do.
- Capitalisation follows the field's autocapitalization type.
- The toolbar keeps tone, the dictation pill (now showing the OpenWhispr
  logo) and the agent button, one row gap above the keys. The space bar is
  blank, as Apple's is. The keyboard opens at its final height.

Taps in the gaps between keys and in the keyboard's padding now land:
#2388's nearest-key routing is carried over to the new layout, and the
keyboard view is 0.1% opaque instead of clear, because iOS sends an
extension no touches where it is fully transparent (in the simulator, gap
taps reached the keyboard only with this change). The touch tests compile
every keyboard source and allow for the half-point side margin's rounding.

Adds the capitalisation rule's tests to mobile CI.

Co-Authored-By: Claude Opus 5.5 <[email protected]>

* fix(mobile): keep non-clean prebuilds working with the keyboard plugin

CocoaPods rewrites the OpenWhisprKeyboard group with only a path, so the
name-only group lookup missed it, created an empty group and failed the
prebuild ("another file reference already uses that path") on any ios/
folder that had been through pod install. The lookup now matches name or
path and prefers the group that already holds the keyboard sources.

The Jest tests now cover a CocoaPods-style group, an empty duplicate group
and a project that already has the keyboard target (one target, its build
settings, one embed), check the plugin's file list against ios/*.swift,
and clean up their temp folders. Drops the unused Resources phase mapping.

* fix(mobile): fix review findings in the rebuilt iOS keyboard

- A greyed return key keeps its taps and ignores them, instead of passing
  them to the space, ".com" or "." key beside it.
- The return key redraws only when its look changes, not on every key.
- Shift ends after a letter by the field's capitalization type, so hosts
  that never report their text don't type in capitals; delete re-checks
  shift, and shift and the return key follow the field when it appears.
- Twitter fields show return and drop @ and # on the 123 and #+= pages,
  as Apple's do (iOS 26.5 simulator).
- Straight quotes where the field turns smart quotes off, and by default
  in URL, email and password fields.
- Layout follows rotation and a late globe-key flag, not only width; key
  touch reach follows the laid-out width; a view shorter than the
  keyboard squeezes the rows instead of pushing the toolbar off screen.
- The keyboard grows 40pt while the agent review card shows.
- A docked iPad keyboard keeps its previous 309pt height and top gap.
- VoiceOver labels for shift/caps lock, delete, the page keys and quotes.

The touch tests stub the field, the globe key, landscape and iPad, and
now cover every iPhone width in portrait and landscape, all pages, iPad,
the URL/email/web search/Twitter rows and a greyed return key, plus the
behaviour above (365 checks). The runner compiles the other keyboard
sources as their own files, and the padding check allows half a pixel.

* fix(mobile): keep the keyboard steady and usable in fields that hide their text

- The keyboard keeps the review card's extra 40pt while a regenerate or
  follow-up runs, instead of shrinking and growing again under the host app;
  it gives the height back once the action ends without the card.
- Hosts that never report their text: the keyboard remembers the field it
  typed into (documentIdentifier). Where that field has reported no text,
  textDidChange, delete and a dictated insert leave shift alone instead of
  capitalising mid-word, and a greyed return key still sends. A field that
  reports text but none around the cursor leaves shift alone too.
- All keyboard inserts go through typeText().

The touch tests cover the card's height through regenerate, error, an action
ending without the card and dismiss; shift and return in a field that hides
its text; return greying again once a field is emptied; the late globe key on
an iPad; and touch reach in landscape, where the keys aren't rebuilt (376
checks). Each of these fails with its fix reverted.

---------

Co-authored-by: Claude Opus 5.5 <[email protected]>
Co-authored-by: Chadpiha <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants