Skip to content

Japanese IME input is not working properly on macOS #160935

@hidea

Description

@hidea

Steps to reproduce

This is a report following the this issue.
singerdmx/flutter-quill#2178

Enter a sentence with two or more phrases using Japanese IME.
Example: きょうはいえにかえります This sentence consists of three phrases, recognized during IME conversion as: きょうは いえに かえります

Press the space key to convert using IME, then press Enter to confirm.

Expected results

今日は家に帰ります

Actual results

今日は家に帰ります家に帰ります

Code sample

Code sample

Type a sentence and then press enter.

After insertText  is called from macOS NSTextInputClient 

flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm

- (void)insertText:(id)string replacementRange:(NSRange)range {
  if (_activeModel == nullptr) {
    return;
  }
  _eventProducedOutput |= true;

  FML_LOG(INFO) << "insertText"
   << " string=" << string;
   << " range=" << range.location << " " << range.length;

  if (range.location != NSNotFound) {
    // The selected range can actually have negative numbers, since it can start
    // at the end of the range if the user selected the text going backwards.
    // Cast to a signed type to determine whether or not the selection is reversed.
    long signedLength = static_cast<long>(range.length);
    long location = range.location;
    long textLength = _activeModel->text_range().end();

    size_t base = std::clamp(location, 0L, textLength);
    size_t extent = std::clamp(location + signedLength, 0L, textLength);

    _activeModel->SetSelection(flutter::TextRange(base, extent));
  }

  flutter::TextRange oldSelection = _activeModel->selection();
  flutter::TextRange composingBeforeChange = _activeModel->composing_range();
  flutter::TextRange replacedRange(-1, -1);

  std::string textBeforeChange = _activeModel->GetText().c_str();
  std::string utf8String = [string UTF8String];

  FML_LOG(INFO) << "textBeforeChange=" << textBeforeChange;
  FML_LOG(INFO) << "utf8String=" << utf8String;

  _activeModel->AddText(utf8String);

  FML_LOG(INFO) << "textAfterChange=" << _activeModel->GetText().c_str();

  if (_activeModel->composing()) {
    replacedRange = composingBeforeChange;
    _activeModel->CommitComposing();
    _activeModel->EndComposing();
  } else {
    replacedRange = range.location == NSNotFound
                        ? flutter::TextRange(oldSelection.base(), oldSelection.extent())
                        : flutter::TextRange(range.location, range.location + range.length);
  }

  if (_enableDeltaModel) {
    [self updateEditStateWithDelta:flutter::TextEditingDelta(textBeforeChange, replacedRange, utf8String)];
  } else {
    [self updateEditState];
  }
}

[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(745)] insertText string=1 range=9223372036854775807 0
[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(770)] textBeforeChange=今日は家に帰ります
[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(771)] utf8String=今日は家に帰ります
[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(775)] textAfterChange=今日は家に帰ります家に帰ります

Problem occurred after _activeModel->AddText.

Next, check AddText.

flutter/shell/platform/common/text_input_model.cc

void TextInputModel::AddText(const std::u16string& text) {
  FML_LOG(INFO) << "AddText:" 
    << " text=" << fml::Utf16ToUtf8(text) 
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  DeleteSelected();

  FML_LOG(INFO) << "after DeleteSelected: "
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  if (composing_) {
    // Delete the current composing text, set the cursor to composing start.
    text_.erase(composing_range_.start(), composing_range_.length());
    selection_ = TextRange(composing_range_.start());
    composing_range_.set_end(composing_range_.start() + text.length());

    FML_LOG(INFO) << "after erace composing_range: "
      << " text_=" << fml::Utf16ToUtf8(text_) 
      << " selection_=" << selection_.start() << " " << selection_.length()
      << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();
  }
  size_t position = selection_.position();
  text_.insert(position, text);
  selection_ = TextRange(position + text.length());

  FML_LOG(INFO) << "after text_.insert: "
    << " position=" << position
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();
}

[INFO:flutter/shell/platform/common/text_input_model.cc(151)] AddText: text=今日は家に帰ります text_=今日は家に帰ります selection_=0 3 composing_range_=0 9
[INFO:flutter/shell/platform/common/text_input_model.cc(159)] after DeleteSelected:  text_=家に帰ります selection_=0 0 composing_range_=0 0
[INFO:flutter/shell/platform/common/text_input_model.cc(170)] after erace composing_range:  text_=家に帰ります selection_=0 0 composing_range_=0 9
[INFO:flutter/shell/platform/common/text_input_model.cc(180)] after text_.insert:  position=0 text_=今日は家に帰ります家に帰ります selection_=9 0 composing_range_=0 9

It is expected that text_ in composing_range_ would be deleted before text_.insert, but this is not the case.

DeleteSelected deletes only selection_ range and clears composing_range_.

Next, check DeleteSelected.

flutter/shell/platform/common/text_input_model.cc

bool TextInputModel::DeleteSelected() {
  if (selection_.collapsed()) {
    return false;
  }

  FML_LOG(INFO) << "DeleteSelected: "
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  size_t start = selection_.start();
  text_.erase(start, selection_.length());
  selection_ = TextRange(start);

  FML_LOG(INFO) << "after text_.erase: "
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  if (composing_) {
    // This occurs only immediately after composing has begun with a selection.
    composing_range_ = selection_;
  }
  return true;
}

[INFO:flutter/shell/platform/common/text_input_model.cc(115)] DeleteSelected:  text_=今日は家に帰ります selection_=0 3 composing_range_=0 9
[INFO:flutter/shell/platform/common/text_input_model.cc(124)] after text_.erase:  text_=家に帰ります selection_=0 0 composing_range_=0 0

“This occurs only immediately after composing has begun with a selection. “

However, in macOS Japanese IME, selection_ indicates only a portion of the range of an unfinalized string that is being converted. The entire conversion range is composing_range_.

In this example, selection_ is the range from the beginning of composing_range_, and in Japanese conversion, selection_ can also be the middle part of composing_range_.

It would be quicker to fix this in text_input_model.cc, but this is a common code for each device, so it will affect other devices that do not have the problem.

So I fixed it with insertText in FlutterTextInputPlugin.mm mentioned at the beginning of this article.

flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm

- (void)insertText:(id)string replacementRange:(NSRange)range {
  if (_activeModel == nullptr) {
    return;
  }
  _eventProducedOutput |= true;

  if (range.location != NSNotFound) {
    // The selected range can actually have negative numbers, since it can start
    // at the end of the range if the user selected the text going backwards.
    // Cast to a signed type to determine whether or not the selection is reversed.
    long signedLength = static_cast<long>(range.length);
    long location = range.location;
    long textLength = _activeModel->text_range().end();

    size_t base = std::clamp(location, 0L, textLength);
    size_t extent = std::clamp(location + signedLength, 0L, textLength);

    _activeModel->SetSelection(flutter::TextRange(base, extent));
  }
+ else if (_activeModel->composing() && !(_activeModel->composing_range() == _activeModel->selection())) {
+   // When confirmed by Japanese IME, string replaces range of composing_range.
+   // If selection == composing_range there is no problem.
+   // If selection ! = composing_range the range of selection is only a part of composing_range.
+   // Since _activeModel->AddText is processed first for selection, the finalization of the conversion 
+   // cannot be processed correctly unless selection == composing_range or selection.collapsed().
+   // Since _activeModel->SetSelection fails if (composing_ && !range.collapsed()), 
+   // selection == composing_range will failed.
+   // Therefore, the selection cursor should only be placed at the beginning of composing_range.
+   flutter::TextRange composing_range = _activeModel->composing_range();
+   _activeModel->SetSelection(flutter::TextRange(composing_range.start()));
+ }

  flutter::TextRange oldSelection = _activeModel->selection();
  flutter::TextRange composingBeforeChange = _activeModel->composing_range();
  flutter::TextRange replacedRange(-1, -1);

  std::string textBeforeChange = _activeModel->GetText().c_str();
  std::string utf8String = [string UTF8String];

  _activeModel->AddText(utf8String);

  if (_activeModel->composing()) {
    replacedRange = composingBeforeChange;
    _activeModel->CommitComposing();
    _activeModel->EndComposing();
  } else {
    replacedRange = range.location == NSNotFound
                        ? flutter::TextRange(oldSelection.base(), oldSelection.extent())
                        : flutter::TextRange(range.location, range.location + range.length);
  }

  if (_enableDeltaModel) {
    [self updateEditStateWithDelta:flutter::TextEditingDelta(textBeforeChange, replacedRange,
                                                             utf8String)];
  } else {
    [self updateEditState];
  }
}

This solved the problem.
What do you think?
Not only this example, but I have tried several Japanese input patterns and they work fine.
Not only the standard macOS Japanese IME, but also 3rd party ones worked fine.

Screenshots or Video

Type a sentence and then press enter.

After insertText  is called from macOS NSTextInputClient 

flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm

- (void)insertText:(id)string replacementRange:(NSRange)range {
  if (_activeModel == nullptr) {
    return;
  }
  _eventProducedOutput |= true;

  FML_LOG(INFO) << "insertText"
   << " string=" << string;
   << " range=" << range.location << " " << range.length;

  if (range.location != NSNotFound) {
    // The selected range can actually have negative numbers, since it can start
    // at the end of the range if the user selected the text going backwards.
    // Cast to a signed type to determine whether or not the selection is reversed.
    long signedLength = static_cast<long>(range.length);
    long location = range.location;
    long textLength = _activeModel->text_range().end();

    size_t base = std::clamp(location, 0L, textLength);
    size_t extent = std::clamp(location + signedLength, 0L, textLength);

    _activeModel->SetSelection(flutter::TextRange(base, extent));
  }

  flutter::TextRange oldSelection = _activeModel->selection();
  flutter::TextRange composingBeforeChange = _activeModel->composing_range();
  flutter::TextRange replacedRange(-1, -1);

  std::string textBeforeChange = _activeModel->GetText().c_str();
  std::string utf8String = [string UTF8String];

  FML_LOG(INFO) << "textBeforeChange=" << textBeforeChange;
  FML_LOG(INFO) << "utf8String=" << utf8String;

  _activeModel->AddText(utf8String);

  FML_LOG(INFO) << "textAfterChange=" << _activeModel->GetText().c_str();

  if (_activeModel->composing()) {
    replacedRange = composingBeforeChange;
    _activeModel->CommitComposing();
    _activeModel->EndComposing();
  } else {
    replacedRange = range.location == NSNotFound
                        ? flutter::TextRange(oldSelection.base(), oldSelection.extent())
                        : flutter::TextRange(range.location, range.location + range.length);
  }

  if (_enableDeltaModel) {
    [self updateEditStateWithDelta:flutter::TextEditingDelta(textBeforeChange, replacedRange, utf8String)];
  } else {
    [self updateEditState];
  }
}

[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(745)] insertText string=1 range=9223372036854775807 0
[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(770)] textBeforeChange=今日は家に帰ります
[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(771)] utf8String=今日は家に帰ります
[INFO:flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm(775)] textAfterChange=今日は家に帰ります家に帰ります

Problem occurred after _activeModel->AddText.

Next, check AddText.

flutter/shell/platform/common/text_input_model.cc

void TextInputModel::AddText(const std::u16string& text) {
  FML_LOG(INFO) << "AddText:" 
    << " text=" << fml::Utf16ToUtf8(text) 
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  DeleteSelected();

  FML_LOG(INFO) << "after DeleteSelected: "
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  if (composing_) {
    // Delete the current composing text, set the cursor to composing start.
    text_.erase(composing_range_.start(), composing_range_.length());
    selection_ = TextRange(composing_range_.start());
    composing_range_.set_end(composing_range_.start() + text.length());

    FML_LOG(INFO) << "after erace composing_range: "
      << " text_=" << fml::Utf16ToUtf8(text_) 
      << " selection_=" << selection_.start() << " " << selection_.length()
      << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();
  }
  size_t position = selection_.position();
  text_.insert(position, text);
  selection_ = TextRange(position + text.length());

  FML_LOG(INFO) << "after text_.insert: "
    << " position=" << position
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();
}

[INFO:flutter/shell/platform/common/text_input_model.cc(151)] AddText: text=今日は家に帰ります text_=今日は家に帰ります selection_=0 3 composing_range_=0 9
[INFO:flutter/shell/platform/common/text_input_model.cc(159)] after DeleteSelected:  text_=家に帰ります selection_=0 0 composing_range_=0 0
[INFO:flutter/shell/platform/common/text_input_model.cc(170)] after erace composing_range:  text_=家に帰ります selection_=0 0 composing_range_=0 9
[INFO:flutter/shell/platform/common/text_input_model.cc(180)] after text_.insert:  position=0 text_=今日は家に帰ります家に帰ります selection_=9 0 composing_range_=0 9

It is expected that text_ in composing_range_ would be deleted before text_.insert, but this is not the case.

DeleteSelected deletes only selection_ range and clears composing_range_.

Next, check DeleteSelected.

flutter/shell/platform/common/text_input_model.cc

bool TextInputModel::DeleteSelected() {
  if (selection_.collapsed()) {
    return false;
  }

  FML_LOG(INFO) << "DeleteSelected: "
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  size_t start = selection_.start();
  text_.erase(start, selection_.length());
  selection_ = TextRange(start);

  FML_LOG(INFO) << "after text_.erase: "
    << " text_=" << fml::Utf16ToUtf8(text_) 
    << " selection_=" << selection_.start() << " " << selection_.length()
    << " composing_range_=" << composing_range_.start() << " " << composing_range_.length();

  if (composing_) {
    // This occurs only immediately after composing has begun with a selection.
    composing_range_ = selection_;
  }
  return true;
}

[INFO:flutter/shell/platform/common/text_input_model.cc(115)] DeleteSelected:  text_=今日は家に帰ります selection_=0 3 composing_range_=0 9
[INFO:flutter/shell/platform/common/text_input_model.cc(124)] after text_.erase:  text_=家に帰ります selection_=0 0 composing_range_=0 0

“This occurs only immediately after composing has begun with a selection. “

However, in macOS Japanese IME, selection_ indicates only a portion of the range of an unfinalized string that is being converted. The entire conversion range is composing_range_.

In this example, selection_ is the range from the beginning of composing_range_, and in Japanese conversion, selection_ can also be the middle part of composing_range_.

It would be quicker to fix this in text_input_model.cc, but this is a common code for each device, so it will affect other devices that do not have the problem.

So I fixed it with insertText in FlutterTextInputPlugin.mm mentioned at the beginning of this article.

flutter/shell/platform/darwin/macos/framework/Source/FlutterTextInputPlugin.mm

- (void)insertText:(id)string replacementRange:(NSRange)range {
  if (_activeModel == nullptr) {
    return;
  }
  _eventProducedOutput |= true;

  if (range.location != NSNotFound) {
    // The selected range can actually have negative numbers, since it can start
    // at the end of the range if the user selected the text going backwards.
    // Cast to a signed type to determine whether or not the selection is reversed.
    long signedLength = static_cast<long>(range.length);
    long location = range.location;
    long textLength = _activeModel->text_range().end();

    size_t base = std::clamp(location, 0L, textLength);
    size_t extent = std::clamp(location + signedLength, 0L, textLength);

    _activeModel->SetSelection(flutter::TextRange(base, extent));
  }
+ else if (_activeModel->composing() && !(_activeModel->composing_range() == _activeModel->selection())) {
+   // When confirmed by Japanese IME, string replaces range of composing_range.
+   // If selection == composing_range there is no problem.
+   // If selection ! = composing_range the range of selection is only a part of composing_range.
+   // Since _activeModel->AddText is processed first for selection, the finalization of the conversion 
+   // cannot be processed correctly unless selection == composing_range or selection.collapsed().
+   // Since _activeModel->SetSelection fails if (composing_ && !range.collapsed()), 
+   // selection == composing_range will failed.
+   // Therefore, the selection cursor should only be placed at the beginning of composing_range.
+   flutter::TextRange composing_range = _activeModel->composing_range();
+   _activeModel->SetSelection(flutter::TextRange(composing_range.start()));
+ }

  flutter::TextRange oldSelection = _activeModel->selection();
  flutter::TextRange composingBeforeChange = _activeModel->composing_range();
  flutter::TextRange replacedRange(-1, -1);

  std::string textBeforeChange = _activeModel->GetText().c_str();
  std::string utf8String = [string UTF8String];

  _activeModel->AddText(utf8String);

  if (_activeModel->composing()) {
    replacedRange = composingBeforeChange;
    _activeModel->CommitComposing();
    _activeModel->EndComposing();
  } else {
    replacedRange = range.location == NSNotFound
                        ? flutter::TextRange(oldSelection.base(), oldSelection.extent())
                        : flutter::TextRange(range.location, range.location + range.length);
  }

  if (_enableDeltaModel) {
    [self updateEditStateWithDelta:flutter::TextEditingDelta(textBeforeChange, replacedRange,
                                                             utf8String)];
  } else {
    [self updateEditState];
  }
}

This solved the problem.
What do you think?
Not only this example, but I have tried several Japanese input patterns and they work fine.
Not only the standard macOS Japanese IME, but also 3rd party ones worked fine.

Logs

Logs
[Paste your logs here]

Flutter Doctor output

Doctor output
[✓] Flutter (Channel beta, 3.28.0-0.1.pre, on macOS 15.2 24C101 darwin-arm64, locale ja-JP)
    • Flutter version 3.28.0-0.1.pre on channel beta at /Users/hidea/Documents/workspace/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 3e493a3e4d (2 weeks ago), 2024-12-12 05:59:24 +0900
    • Engine revision 2ba456fd7f
    • Dart version 3.7.0 (build 3.7.0-209.1.beta)
    • DevTools version 2.41.0

[✓] Android toolchain - develop for Android devices (Android SDK version 34.0.0)
    • Android SDK at /Users/hidea/Library/Android/sdk
    • Platform android-35, build-tools 34.0.0
    • ANDROID_HOME = /Users/hidea/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
      This is the JDK bundled with the latest Android Studio installation on this machine.
      To manually set the JDK path, use: `flutter config --jdk-dir="path/to/jdk"`.
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS (Xcode 16.2)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 16C5032a
    ! CocoaPods 1.15.2 out of date (1.16.2 is recommended).
        CocoaPods is a package manager for iOS or macOS platform code.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/to/platform-plugins
      To update CocoaPods, see https://guides.cocoapods.org/using/getting-started.html#updating-cocoapods

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2023.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.9+0-17.0.9b1087.7-11185874)

[✓] VS Code (version 1.96.2)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.102.0

[✓] Connected device (4 available)
    • 15Pro+新刊.netで新刊チェック (mobile)    • 00008130-001E19812210001C • ios            • iOS 18.1.1 22B91
    • macOS (desktop)                 • macos                     • darwin-arm64   • macOS 15.2 24C101 darwin-arm64
    • Mac Designed for iPad (desktop) • mac-designed-for-ipad     • darwin         • macOS 15.2 24C101 darwin-arm64
    • Chrome (web)                    • chrome                    • web-javascript • Google Chrome 131.0.6778.205

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work lista: internationalizationSupporting other languages or locales. (aka i18n)a: text inputEntering text in a text field or keyboard related problemsfound in release: 3.27Found to occur in 3.27found in release: 3.28Found to occur in 3.28fyi-text-inputFor the attention of Text Input teamhas reproducible stepsThe issue has been confirmed reproducible and is ready to work onplatform-macBuilding on or for macOS specificallyr: fixedIssue is closed as already fixed in a newer versionteam-macosOwned by the macOS platform teamtriaged-macosTriaged by the macOS platform team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions