Skip to content

Can't navigate composing candidates list with arrow keys on Mac #85328

Closed

Description

@justinmc

When using composing keyboards on native Mac, the arrow keys can be used to navigate the composing tooltip. In Flutter, the arrow keys continue to be intercepted by Flutter when the composing tooltip is visible, so it doesn't receive the keys. We should not intercept the keys when composing is happening.

Native Mac Flutter Mac
text_edit_composing text_edit_composing_flutter

Steps to reproduce

  1. Run any app with a text input, like the one given below, on a Mac.
  2. Set your keyboard to a composing keyboard, such as Chinese Pinyin.
  3. Type some text so that the composing tooltip appears, such as "ni".
  4. Attempt to navigate in the tooltip using the arrow keys.

Result: The cursor moves, not the selection in the composing tooltip.
Expected result: The selection in the composing tooltip moves.

Code
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        // This is the theme of your application.
        //
        // Try running your application with "flutter run". You'll see the
        // application has a blue toolbar. Then, without quitting the app, try
        // changing the primarySwatch below to Colors.green and then invoke
        // "hot reload" (press "r" in the console where you ran "flutter run",
        // or simply save your changes to "hot reload" in a Flutter IDE).
        // Notice that the counter didn't reset back to zero; the application
        // is not restarted.
        primarySwatch: Colors.blue,
        // This makes the visual density adapt to the platform that you run
        // the app on. For desktop platforms, the controls will be smaller and
        // closer together (more dense) than on mobile platforms.
        visualDensity: VisualDensity.adaptivePlatformDensity,
      ),
      home: MyHomePage(title: 'Flutter Demo Home Page'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  // This widget is the home page of your application. It is stateful, meaning
  // that it has a State object (defined below) that contains fields that affect
  // how it looks.

  // This class is the configuration for the state. It holds the values (in this
  // case the title) provided by the parent (in this case the App widget) and
  // used by the build method of the State. Fields in a Widget subclass are
  // always marked "final".

  final String title;

  @override
  _MyHomePageState createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  int _counter = 0;

  void _incrementCounter() {
    setState(() {
      // This call to setState tells the Flutter framework that something has
      // changed in this State, which causes it to rerun the build method below
      // so that the display can reflect the updated values. If we changed
      // _counter without calling setState(), then the build method would not be
      // called again, and so nothing would appear to happen.
      _counter++;
    });
  }

  @override
  Widget build(BuildContext context) {
    // This method is rerun every time setState is called, for instance as done
    // by the _incrementCounter method above.
    //
    // The Flutter framework has been optimized to make rerunning build methods
    // fast, so that you can just rebuild anything that needs updating rather
    // than having to individually change instances of widgets.
    return Scaffold(
      appBar: AppBar(
        // Here we take the value from the MyHomePage object that was created by
        // the App.build method, and use it to set our appbar title.
        title: Text(widget.title),
      ),
      body: Center(
        // Center is a layout widget. It takes a single child and positions it
        // in the middle of the parent.
        child: TextField(),
      ),
      floatingActionButton: FloatingActionButton(
        onPressed: _incrementCounter,
        tooltip: 'Increment',
        child: Icon(Icons.add),
      ), // This trailing comma makes auto-formatting nicer for build methods.
    );
  }
}

Activity

  1. added
    a: text inputEntering text in a text field or keyboard related problems
    platform-macosBuilding on or for macOS specifically
    on Jun 25, 2021
  2. self-assigned this
    on Jun 25, 2021
  3. maheshj01 commented on Jul 12, 2021

    @maheshj01
    Member

    Reproducible on the latest stable and the master channel, When the composable tooltip is visible, using the side arrow keys the cursor moves in the TextField instead of navigating through the tooltip suggestions.

    flutter doctor -v
    [✓] Flutter (Channel stable, 2.2.3, on macOS 11.4 20F71 darwin-arm, locale en-IN)
        • Flutter version 2.2.3 at /Users/mahesh/Documents/flutter
        • Framework revision f4abaa0735 (10 days ago), 2021-07-01 12:46:11 -0700
        • Engine revision 241c87ad80
        • Dart version 2.13.4
    
    [✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
        • Android SDK at /Users/mahesh/Library/Android/sdk
        • Platform android-30, build-tools 30.0.3
        • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
        • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
        • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
        • All Android licenses accepted.
    
    [✓] Xcode - develop for iOS and macOS
        • Xcode at /Applications/Xcode.app/Contents/Developer
        • Xcode 12.4, Build version 12D4e
        • CocoaPods version 1.10.1
    
    [✓] Chrome - develop for the web
        • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
    
    [✓] Android Studio (version 4.2)
        • Android Studio at /Applications/Android Studio.app/Contents
        • Flutter plugin can be installed from:
          🔨 https://plugins.jetbrains.com/plugin/9212-flutter
        • Dart plugin can be installed from:
          🔨 https://plugins.jetbrains.com/plugin/6351-dart
        • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    
    [✓] IntelliJ IDEA Community Edition (version 2021.1.2)
        • IntelliJ at /Applications/IntelliJ IDEA CE.app
        • Flutter plugin version 57.0.5
        • Dart plugin version 211.7233
    
    [✓] VS Code (version 1.58.0)
        • VS Code at /Applications/Visual Studio Code.app/Contents
        • Flutter extension version 3.24.0
    
    [✓] Connected device (2 available)
        • macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
        • Chrome (web)    • chrome • web-javascript • Google Chrome 91.0.4472.114
    
    • No issues found!
    
    [✓] Flutter (Channel master, 2.4.0-1.0.pre.149, on macOS 11.4 20F71 darwin-arm, locale en-IN)
        • Flutter version 2.4.0-1.0.pre.149 at /Users/mahesh/Documents/flutter_master
        • Upstream repository https://github.com/flutter/flutter.git
        • Framework revision d71c0eb1d8 (3 days ago), 2021-07-09 06:01:04 -0400
        • Engine revision 024cc455fe
        • Dart version 2.14.0 (build 2.14.0-293.0.dev)
    
    [✓] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
        • Android SDK at /Users/mahesh/Library/Android/sdk
        • Platform android-30, build-tools 30.0.3
        • ANDROID_HOME = /Users/mahesh/Library/Android/sdk
        • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
        • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
        • All Android licenses accepted.
    
    [✓] Xcode - develop for iOS and macOS
        • Xcode at /Applications/Xcode.app/Contents/Developer
        • Xcode 12.4, Build version 12D4e
        • CocoaPods version 1.10.1
    
    [✓] Chrome - develop for the web
        • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome
    
    [✓] Android Studio (version 4.2)
        • Android Studio at /Applications/Android Studio.app/Contents
        • Flutter plugin can be installed from:
          🔨 https://plugins.jetbrains.com/plugin/9212-flutter
        • Dart plugin can be installed from:
          🔨 https://plugins.jetbrains.com/plugin/6351-dart
        • Java version OpenJDK Runtime Environment (build 11.0.8+10-b944.6916264)
    
    [✓] IntelliJ IDEA Community Edition (version 2021.1.2)
        • IntelliJ at /Applications/IntelliJ IDEA CE.app
        • Flutter plugin version 57.0.5
        • Dart plugin version 211.7233
    
    [✓] VS Code (version 1.58.0)
        • VS Code at /Applications/Visual Studio Code.app/Contents
        • Flutter extension version 3.24.0
    
    [✓] Connected device (2 available)
        • macOS (desktop) • macos  • darwin-arm64   • macOS 11.4 20F71 darwin-arm
        • Chrome (web)    • chrome • web-javascript • Google Chrome 91.0.4472.114
    
    • No issues found!
    
  4. cbracken commented on Jul 28, 2021

    @cbracken
    Member

    Keep in mind that arrow keys behave differently depending on the language and IME.

    For example, with Japanese input on macOS, left and right arrow should move the cursor within the composing region when the candidates drop-down is not present (either before it comes up, or if you hit ESC), and when it is visible, it's navigated using up/down (right/left have no effect either on cursor position or candidate selection):

    Screen.Recording.2021-07-28.at.15.09.47.mov

    In this case:

    1. I typed the keys 'nihonngo'
    2. the candidates list popped up automatically based on my input history
    3. I pressed ESC to hide the candidates list
    4. I moved the cursor around using left/right arrow keys
    5. I pressed space to bring up the candidates again
    6. I pressed up/down to move between candidates
    7. I pressed ESC to hide the candidates
    8. I changed the active composing region using shift left/right

    At least on macOS, I believe that so long as the candidates list is visible, the correct behaviour is to hand the keypresses to the IME. Once the candidates list is hidden (e.g. via ESC), then we're back to moving the cursor. Since changes from the IME trigger us to update the text editing state from the engine back to the framework, I think it may be safe to always send those keypresses to the IME in composing mode... but I'd verify the above test case steps work as expected.

  5. cbracken commented on Jul 28, 2021

    @cbracken
    Member

    I know this issue is mac-specific, but it's worth noting that behaviour on Linux (GTK) is quite different to this.

  6. added
    P1High-priority issues at the top of the work list
    on Oct 5, 2021
  7. changed the title [-]Can't navigate composing tooltip with arrow keys on Mac[/-] [+]Can't navigate composing candidates list with arrow keys on Mac[/+] on Oct 5, 2021
  8. 21 remaining items

  9. cbracken commented on Jun 3, 2022

    @cbracken
    Member

    Forward the actions from NSStandardKeyBindingResponding to the framework and bind them to appropriate intents

    Agreed this seems like potentially a cleaner path forward; also nice that it allows for stripping out some more embedder-specific code from the framework. @LongCatIsLooong / @dkwingsmt was this considered during the key event work for macOS? If so, do you remember why it was rejected?

  10. knopp commented on Jun 4, 2022

    @knopp
    Member

    PR to fix emoji picker. Not sure why it doesn't show as reference here.

  11. knopp commented on Jun 5, 2022

    @knopp
    Member

    PR for engine and framework to use editing intents instead from engine of hardcoded shortcuts in framework. These are bit raw but any feedback on whether this goes in the right direction is appreciated. Together with flutter-team-archive/engine#33827 this should be a proper solution for all IMEs (CJK, Latin and Emoji).

  12. reopened this on Jun 15, 2022
  13. maheshj01 commented on Jun 21, 2022

    @maheshj01
    Member

    @knopp Was this issue accidentally reopened? Also, the linked PRs seem to be merged.

  14. knopp commented on Jun 21, 2022

    @knopp
    Member

    It wasn't accidentally reopened. flutter-team-archive/engine#33838 and #105407 have not been merged yet. They are needed to completely fix the issue.

  15. unlocked this conversation on Jun 21, 2022
  16. Repository owner moved this from Todo to Done in Flutter Desktopon Jul 29, 2022
  17. added
    r: fixedIssue is closed as already fixed in a newer version
    on Jul 29, 2022
  18. github-actions commented on Aug 12, 2022

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  19. locked as resolved and limited conversation to collaborators on Aug 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

P1High-priority issues at the top of the work lista: desktopRunning on desktopa: text inputEntering text in a text field or keyboard related problemscustomer: todofound in release: 2.2Found to occur in 2.2found in release: 2.4Found to occur in 2.4frameworkflutter/packages/flutter repository. See also f: labels.has reproducible stepsThe issue has been confirmed reproducible and is ready to work onplatform-macosBuilding on or for macOS specificallyr: fixedIssue is closed as already fixed in a newer version

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions