-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Fix extra numbers showing up when enabling VoiceControl #163593
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix extra numbers showing up when enabling VoiceControl #163593
Conversation
| return traits; | ||
| } | ||
|
|
||
| - (BOOL)accessibilityRespondsToUserInteraction { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chunhtai I'm worried this logic might break if/when we add more "system" actions. Would you have any recommendations on how to future-proof this method?
For example, should we push kSystemActions down to //flutter/lib/ui/semantics/semantics_node.h?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, that sounds like a better way of keeping track of this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also should we implement this in SemanticsObject class instead? there are subclasses such as switch that extends from SemanticsObject, for example, a disabled switch probability shouldn't responseToUserInteraction as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a good question. I believe FlutterSwitchSemanticsObject works as expected as it overrides traits, and I didn't see any issues Material 3 switches. However, I think we might need a similar fix for FlutterScrollableSemanticsObject.
Voice Control shows incorrect numbers on the Material 3 demo, and I suspected FlutterScrollableSemanticsObject but I didn't have the time to root cause it. @LouiseHsu could you check if pushing this logic down to SemanticsObject removes the Voice Control numbers that show up on the Material 3 demo app's grey outline used to group controls?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried this with our fix, and the numbers did change, but not for the number on the grey bar
| Before Fix | After Fix |
|---|---|
![]() |
![]() |
Interestingly if i comment out the other sibling containers the marker disappears. I think the root cause is different than the fix for these two issues. Ill try to find a minimal repro and file a seperate bug for it, but I won't address it in this PR.

engine/src/flutter/shell/platform/darwin/ios/framework/Source/accessibility_bridge_test.mm
Outdated
Show resolved
Hide resolved
engine/src/flutter/shell/platform/darwin/ios/framework/Source/accessibility_bridge_test.mm
Outdated
Show resolved
Hide resolved
engine/src/flutter/shell/platform/darwin/ios/framework/Source/accessibility_bridge_test.mm
Show resolved
Hide resolved
| return traits; | ||
| } | ||
|
|
||
| - (BOOL)accessibilityRespondsToUserInteraction { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, that sounds like a better way of keeping track of this.
| return traits; | ||
| } | ||
|
|
||
| - (BOOL)accessibilityRespondsToUserInteraction { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
also should we implement this in SemanticsObject class instead? there are subclasses such as switch that extends from SemanticsObject, for example, a disabled switch probability shouldn't responseToUserInteraction as well.
engine/src/flutter/shell/platform/darwin/ios/framework/Source/accessibility_bridge_test.mm
Show resolved
Hide resolved
697d3c8 to
e841467
Compare
chunhtai
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, just one question
| } | ||
|
|
||
| - (BOOL)accessibilityRespondsToUserInteraction { | ||
| if ((self.node.actions & flutter::~kSystemActions) != 0) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am surprise this works, shouldn't this be ~flutter::kSystemActions
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
it doesnt work cuz the tests are failing 😅. My bad I was copy and pasting my changes cuz I had to force push to an earlier commit
| const int kScrollableSemanticsActions = | ||
| kVerticalScrollSemanticsActions | kHorizontalScrollSemanticsActions; | ||
|
|
||
| const int kSystemActions = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@chunhtai What do you think of this wording?
| const int kSystemActions = | |
| // Actions that aren't user-initiated. | |
| const int kSystemActions = |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Im not @chunhtai but I agree
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SGTM
Roll Flutter from 1659206 to 2e570ca (75 revisions) flutter/flutter@1659206...2e570ca 2025-02-27 [email protected] Roll Skia from ac08df9c8d82 to e5a33102401c (1 revision) (flutter/flutter#164288) 2025-02-27 [email protected] Roll Skia from 6018dff0233a to ac08df9c8d82 (1 revision) (flutter/flutter#164275) 2025-02-27 [email protected] Roll Dart SDK from 4a218fbffc80 to fcda71ce147b (1 revision) (flutter/flutter#164276) 2025-02-27 [email protected] Roll Skia from 0f2e106c9abc to 6018dff0233a (1 revision) (flutter/flutter#164266) 2025-02-27 [email protected] Add windowing channel support to Linux embedder (flutter/flutter#163180) 2025-02-27 [email protected] Roll Dart SDK from 7fa5901bd8a3 to 4a218fbffc80 (2 revisions) (flutter/flutter#164260) 2025-02-27 [email protected] Roll Skia from fdd97386193e to 0f2e106c9abc (4 revisions) (flutter/flutter#164259) 2025-02-27 [email protected] Run more builds faster (flutter/flutter#164125) 2025-02-27 [email protected] Increase customer test timeout to 60 minutes (flutter/flutter#164239) 2025-02-27 [email protected] Reland "Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools" (flutter/flutter#164240) 2025-02-27 [email protected] Use the Dart isolate ownership API on the root isolate (flutter/flutter#163703) 2025-02-26 [email protected] Roll Fuchsia Linux SDK from g-2SJtblPjjaH7Egy... to 1elkOxihZuTEiTXzY... (flutter/flutter#164243) 2025-02-26 [email protected] Show Linux driver information in flutter doctor (flutter/flutter#163980) 2025-02-26 [email protected] Implement opacity `FlutterMutator` for hcpp (flutter/flutter#164147) 2025-02-26 [email protected] Roll Fuchsia Linux SDK from g-2SJtblPjjaH7Egy... to 1elkOxihZuTEiTXzY... (flutter/flutter#164232) 2025-02-26 [email protected] Roll Skia from 47b84f354604 to fdd97386193e (8 revisions) (flutter/flutter#164221) 2025-02-26 [email protected] [Impeller] Reland: move AHB check into Flutter main, don't disable ImageReader on 29. (flutter/flutter#164201) 2025-02-26 [email protected] Roll Dart SDK from 80865748abe0 to 7fa5901bd8a3 (3 revisions) (flutter/flutter#164226) 2025-02-26 [email protected] Update fuchsia_test_scripts_version to the latest version (flutter/flutter#164123) 2025-02-26 [email protected] Guard against zero item extent for carousel (flutter/flutter#163310) 2025-02-26 [email protected] [Impeller] work around for crashy Nexus 5 Driver. (flutter/flutter#164040) 2025-02-26 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools (#163890)" (flutter/flutter#164209) 2025-02-26 [email protected] Add localization for `Back` and `Cancel` buttons in CupertinoNavigationBar (flutter/flutter#162581) 2025-02-26 [email protected] Added compilation failure if a max ubo size is exceeded (flutter/flutter#164038) 2025-02-26 [email protected] Roll Skia from 5336e20498d9 to 47b84f354604 (2 revisions) (flutter/flutter#164188) 2025-02-26 [email protected] Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools (flutter/flutter#163890) 2025-02-26 [email protected] Roll Skia from 3ce6f25dc13e to 5336e20498d9 (9 revisions) (flutter/flutter#164174) 2025-02-26 [email protected] Roll Dart SDK from b8292dfeaa67 to 80865748abe0 (9 revisions) (flutter/flutter#164168) 2025-02-26 [email protected] [Engine] Remove dead code for RoundedSuperellipse (flutter/flutter#164163) 2025-02-26 [email protected] [Impeller] detect mediatek soc and fall back to GLES. (flutter/flutter#164126) 2025-02-26 [email protected] [Impeller] make DLOG into LOG for startup errors. (flutter/flutter#164110) 2025-02-26 [email protected] Intercept error when iOS 18.4 crashes with JIT mode and give guided error (flutter/flutter#164072) 2025-02-26 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Impeller] move AHB check to Vulkan, use Vulkan surface on 29. (#164109)" (flutter/flutter#164166) 2025-02-26 [email protected] Run new gallery transition perf benchmark on Galaxy S24 (flutter/flutter#163665) 2025-02-26 [email protected] [Impeller] move AHB check to Vulkan, use Vulkan surface on 29. (flutter/flutter#164109) 2025-02-26 [email protected] [iOS] Add platform view to integration_test example (flutter/flutter#164144) 2025-02-26 [email protected] Fix minor issues in documentation of WidgetsApp (flutter/flutter#163942) 2025-02-26 [email protected] Fix extra numbers showing up when enabling VoiceControl (flutter/flutter#163593) 2025-02-26 [email protected] Set SliverResizingHeader's maxScrollObstructionExtent to minExtent (flutter/flutter#162955) 2025-02-26 [email protected] Roll Dart SDK from aea6fff33f06 to b8292dfeaa67 (1 revision) (flutter/flutter#163973) 2025-02-26 [email protected] [DisplayList] Delete all legacy Skia-oriented method overloads in DlCanvas (flutter/flutter#164054) 2025-02-25 [email protected] Clean up leak tracker instrumentation tech debt. (flutter/flutter#164070) 2025-02-25 [email protected] Revert "Marks Linux_pixel_7pro service_extensions_test to be flaky" (flutter/flutter#163882) 2025-02-25 [email protected] Check if simctl is installed before trying to list devices or runtimes (flutter/flutter#163895) 2025-02-25 [email protected] Update dragDevices doc to include default PointerDeviceKind.trackpad (flutter/flutter#163898) 2025-02-25 [email protected] Update multiple flutters benchmark test to latest gradle and agp and gradle defined dependencies (flutter/flutter#164029) ...
Roll Flutter from 1659206 to 2e570ca (75 revisions) flutter/flutter@1659206...2e570ca 2025-02-27 [email protected] Roll Skia from ac08df9c8d82 to e5a33102401c (1 revision) (flutter/flutter#164288) 2025-02-27 [email protected] Roll Skia from 6018dff0233a to ac08df9c8d82 (1 revision) (flutter/flutter#164275) 2025-02-27 [email protected] Roll Dart SDK from 4a218fbffc80 to fcda71ce147b (1 revision) (flutter/flutter#164276) 2025-02-27 [email protected] Roll Skia from 0f2e106c9abc to 6018dff0233a (1 revision) (flutter/flutter#164266) 2025-02-27 [email protected] Add windowing channel support to Linux embedder (flutter/flutter#163180) 2025-02-27 [email protected] Roll Dart SDK from 7fa5901bd8a3 to 4a218fbffc80 (2 revisions) (flutter/flutter#164260) 2025-02-27 [email protected] Roll Skia from fdd97386193e to 0f2e106c9abc (4 revisions) (flutter/flutter#164259) 2025-02-27 [email protected] Run more builds faster (flutter/flutter#164125) 2025-02-27 [email protected] Increase customer test timeout to 60 minutes (flutter/flutter#164239) 2025-02-27 [email protected] Reland "Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools" (flutter/flutter#164240) 2025-02-27 [email protected] Use the Dart isolate ownership API on the root isolate (flutter/flutter#163703) 2025-02-26 [email protected] Roll Fuchsia Linux SDK from g-2SJtblPjjaH7Egy... to 1elkOxihZuTEiTXzY... (flutter/flutter#164243) 2025-02-26 [email protected] Show Linux driver information in flutter doctor (flutter/flutter#163980) 2025-02-26 [email protected] Implement opacity `FlutterMutator` for hcpp (flutter/flutter#164147) 2025-02-26 [email protected] Roll Fuchsia Linux SDK from g-2SJtblPjjaH7Egy... to 1elkOxihZuTEiTXzY... (flutter/flutter#164232) 2025-02-26 [email protected] Roll Skia from 47b84f354604 to fdd97386193e (8 revisions) (flutter/flutter#164221) 2025-02-26 [email protected] [Impeller] Reland: move AHB check into Flutter main, don't disable ImageReader on 29. (flutter/flutter#164201) 2025-02-26 [email protected] Roll Dart SDK from 80865748abe0 to 7fa5901bd8a3 (3 revisions) (flutter/flutter#164226) 2025-02-26 [email protected] Update fuchsia_test_scripts_version to the latest version (flutter/flutter#164123) 2025-02-26 [email protected] Guard against zero item extent for carousel (flutter/flutter#163310) 2025-02-26 [email protected] [Impeller] work around for crashy Nexus 5 Driver. (flutter/flutter#164040) 2025-02-26 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools (#163890)" (flutter/flutter#164209) 2025-02-26 [email protected] Add localization for `Back` and `Cancel` buttons in CupertinoNavigationBar (flutter/flutter#162581) 2025-02-26 [email protected] Added compilation failure if a max ubo size is exceeded (flutter/flutter#164038) 2025-02-26 [email protected] Roll Skia from 5336e20498d9 to 47b84f354604 (2 revisions) (flutter/flutter#164188) 2025-02-26 [email protected] Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools (flutter/flutter#163890) 2025-02-26 [email protected] Roll Skia from 3ce6f25dc13e to 5336e20498d9 (9 revisions) (flutter/flutter#164174) 2025-02-26 [email protected] Roll Dart SDK from b8292dfeaa67 to 80865748abe0 (9 revisions) (flutter/flutter#164168) 2025-02-26 [email protected] [Engine] Remove dead code for RoundedSuperellipse (flutter/flutter#164163) 2025-02-26 [email protected] [Impeller] detect mediatek soc and fall back to GLES. (flutter/flutter#164126) 2025-02-26 [email protected] [Impeller] make DLOG into LOG for startup errors. (flutter/flutter#164110) 2025-02-26 [email protected] Intercept error when iOS 18.4 crashes with JIT mode and give guided error (flutter/flutter#164072) 2025-02-26 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Impeller] move AHB check to Vulkan, use Vulkan surface on 29. (#164109)" (flutter/flutter#164166) 2025-02-26 [email protected] Run new gallery transition perf benchmark on Galaxy S24 (flutter/flutter#163665) 2025-02-26 [email protected] [Impeller] move AHB check to Vulkan, use Vulkan surface on 29. (flutter/flutter#164109) 2025-02-26 [email protected] [iOS] Add platform view to integration_test example (flutter/flutter#164144) 2025-02-26 [email protected] Fix minor issues in documentation of WidgetsApp (flutter/flutter#163942) 2025-02-26 [email protected] Fix extra numbers showing up when enabling VoiceControl (flutter/flutter#163593) 2025-02-26 [email protected] Set SliverResizingHeader's maxScrollObstructionExtent to minExtent (flutter/flutter#162955) 2025-02-26 [email protected] Roll Dart SDK from aea6fff33f06 to b8292dfeaa67 (1 revision) (flutter/flutter#163973) 2025-02-26 [email protected] [DisplayList] Delete all legacy Skia-oriented method overloads in DlCanvas (flutter/flutter#164054) 2025-02-25 [email protected] Clean up leak tracker instrumentation tech debt. (flutter/flutter#164070) 2025-02-25 [email protected] Revert "Marks Linux_pixel_7pro service_extensions_test to be flaky" (flutter/flutter#163882) 2025-02-25 [email protected] Check if simctl is installed before trying to list devices or runtimes (flutter/flutter#163895) 2025-02-25 [email protected] Update dragDevices doc to include default PointerDeviceKind.trackpad (flutter/flutter#163898) 2025-02-25 [email protected] Update multiple flutters benchmark test to latest gradle and agp and gradle defined dependencies (flutter/flutter#164029) ...
Roll Flutter from 1659206 to 2e570ca (75 revisions) flutter/flutter@1659206...2e570ca 2025-02-27 [email protected] Roll Skia from ac08df9c8d82 to e5a33102401c (1 revision) (flutter/flutter#164288) 2025-02-27 [email protected] Roll Skia from 6018dff0233a to ac08df9c8d82 (1 revision) (flutter/flutter#164275) 2025-02-27 [email protected] Roll Dart SDK from 4a218fbffc80 to fcda71ce147b (1 revision) (flutter/flutter#164276) 2025-02-27 [email protected] Roll Skia from 0f2e106c9abc to 6018dff0233a (1 revision) (flutter/flutter#164266) 2025-02-27 [email protected] Add windowing channel support to Linux embedder (flutter/flutter#163180) 2025-02-27 [email protected] Roll Dart SDK from 7fa5901bd8a3 to 4a218fbffc80 (2 revisions) (flutter/flutter#164260) 2025-02-27 [email protected] Roll Skia from fdd97386193e to 0f2e106c9abc (4 revisions) (flutter/flutter#164259) 2025-02-27 [email protected] Run more builds faster (flutter/flutter#164125) 2025-02-27 [email protected] Increase customer test timeout to 60 minutes (flutter/flutter#164239) 2025-02-27 [email protected] Reland "Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools" (flutter/flutter#164240) 2025-02-27 [email protected] Use the Dart isolate ownership API on the root isolate (flutter/flutter#163703) 2025-02-26 [email protected] Roll Fuchsia Linux SDK from g-2SJtblPjjaH7Egy... to 1elkOxihZuTEiTXzY... (flutter/flutter#164243) 2025-02-26 [email protected] Show Linux driver information in flutter doctor (flutter/flutter#163980) 2025-02-26 [email protected] Implement opacity `FlutterMutator` for hcpp (flutter/flutter#164147) 2025-02-26 [email protected] Roll Fuchsia Linux SDK from g-2SJtblPjjaH7Egy... to 1elkOxihZuTEiTXzY... (flutter/flutter#164232) 2025-02-26 [email protected] Roll Skia from 47b84f354604 to fdd97386193e (8 revisions) (flutter/flutter#164221) 2025-02-26 [email protected] [Impeller] Reland: move AHB check into Flutter main, don't disable ImageReader on 29. (flutter/flutter#164201) 2025-02-26 [email protected] Roll Dart SDK from 80865748abe0 to 7fa5901bd8a3 (3 revisions) (flutter/flutter#164226) 2025-02-26 [email protected] Update fuchsia_test_scripts_version to the latest version (flutter/flutter#164123) 2025-02-26 [email protected] Guard against zero item extent for carousel (flutter/flutter#163310) 2025-02-26 [email protected] [Impeller] work around for crashy Nexus 5 Driver. (flutter/flutter#164040) 2025-02-26 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools (#163890)" (flutter/flutter#164209) 2025-02-26 [email protected] Add localization for `Back` and `Cancel` buttons in CupertinoNavigationBar (flutter/flutter#162581) 2025-02-26 [email protected] Added compilation failure if a max ubo size is exceeded (flutter/flutter#164038) 2025-02-26 [email protected] Roll Skia from 5336e20498d9 to 47b84f354604 (2 revisions) (flutter/flutter#164188) 2025-02-26 [email protected] Add a buildtools directory and move third_party/ninja to the project root in order to match the expectations of depot_tools (flutter/flutter#163890) 2025-02-26 [email protected] Roll Skia from 3ce6f25dc13e to 5336e20498d9 (9 revisions) (flutter/flutter#164174) 2025-02-26 [email protected] Roll Dart SDK from b8292dfeaa67 to 80865748abe0 (9 revisions) (flutter/flutter#164168) 2025-02-26 [email protected] [Engine] Remove dead code for RoundedSuperellipse (flutter/flutter#164163) 2025-02-26 [email protected] [Impeller] detect mediatek soc and fall back to GLES. (flutter/flutter#164126) 2025-02-26 [email protected] [Impeller] make DLOG into LOG for startup errors. (flutter/flutter#164110) 2025-02-26 [email protected] Intercept error when iOS 18.4 crashes with JIT mode and give guided error (flutter/flutter#164072) 2025-02-26 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[Impeller] move AHB check to Vulkan, use Vulkan surface on 29. (#164109)" (flutter/flutter#164166) 2025-02-26 [email protected] Run new gallery transition perf benchmark on Galaxy S24 (flutter/flutter#163665) 2025-02-26 [email protected] [Impeller] move AHB check to Vulkan, use Vulkan surface on 29. (flutter/flutter#164109) 2025-02-26 [email protected] [iOS] Add platform view to integration_test example (flutter/flutter#164144) 2025-02-26 [email protected] Fix minor issues in documentation of WidgetsApp (flutter/flutter#163942) 2025-02-26 [email protected] Fix extra numbers showing up when enabling VoiceControl (flutter/flutter#163593) 2025-02-26 [email protected] Set SliverResizingHeader's maxScrollObstructionExtent to minExtent (flutter/flutter#162955) 2025-02-26 [email protected] Roll Dart SDK from aea6fff33f06 to b8292dfeaa67 (1 revision) (flutter/flutter#163973) 2025-02-26 [email protected] [DisplayList] Delete all legacy Skia-oriented method overloads in DlCanvas (flutter/flutter#164054) 2025-02-25 [email protected] Clean up leak tracker instrumentation tech debt. (flutter/flutter#164070) 2025-02-25 [email protected] Revert "Marks Linux_pixel_7pro service_extensions_test to be flaky" (flutter/flutter#163882) 2025-02-25 [email protected] Check if simctl is installed before trying to list devices or runtimes (flutter/flutter#163895) 2025-02-25 [email protected] Update dragDevices doc to include default PointerDeviceKind.trackpad (flutter/flutter#163898) 2025-02-25 [email protected] Update multiple flutters benchmark test to latest gradle and agp and gradle defined dependencies (flutter/flutter#164029) ...



Fixes #158477 and #156368.
The excess numbers in both PRs are caused by all
SemanticObjectsreturningYESforaccessibilityRespondsToUserInteraction, even if it has no semantic actions. For example, a SemanticObject with just a label has semantic information (the label) but no action. This PR adds a check, ensuring that anSemanticObjectshas at least one accessible action before returningYESPre-launch Checklist
///).