Repository navigation
[iOS a11y] Use a real UIScrollView as a11y scrollable container - #195731
Conversation
| constexpr int kScrollableSemanticsActions = | ||
| kVerticalScrollSemanticsActions | kHorizontalScrollSemanticsActions; | ||
| kVerticalScrollSemanticsActions | kHorizontalScrollSemanticsActions | | ||
| static_cast<int32_t>(SemanticsAction::kScrollToOffset); |
There was a problem hiding this comment.
This is related (because a11y hit-testing was relying on isFocusable instead of isAccessibilityElement), but mostly a drive-by fix since hit testing no longer relies on isFocusable.
| SemanticsObject* semanticsObject = nil; | ||
| if ([element isKindOfClass:[SemanticsObjectContainer class]]) { | ||
| semanticsObject = ((SemanticsObjectContainer*)element).semanticsObject; | ||
| } else if ([element isKindOfClass:[FlutterSemanticsScrollView class]]) { |
There was a problem hiding this comment.
We can probably eliminate this additional branch if we make SemanticsObjectContainer a protocol, but that would require a larger change.
There was a problem hiding this comment.
I am not opposed to this change, but just curious is it possible that the root element is a FlutterSemanticsScrollView? or this is just a safe precaution
There was a problem hiding this comment.
typo, I meant to say making SemanticsObjectContainer a protocol since the scroll view replaces the SemanticsObjectContainer for scrollable subtrees.
is it possible that the root element is a FlutterSemanticsScrollView? or this is just a safe precaution
You mean this?
([semanticsObject uid] == kRootNodeId)
? semanticsObject.bridgeView
: semanticsObject.parent.accessibilityContainer;
I think gemini just copied this from the SemanticsObject implementation.
There was a problem hiding this comment.
The method only called on root view, my question is that whether it is possible the root node is a scrollable, since the condition [element isKindOfClass:[FlutterSemanticsScrollView class]] will only be true if it is. I don't think framework has a code path that makes the root node to be scrollable.
There was a problem hiding this comment.
Oh in that case the code is just copied (since FlutterSemanticsScrollView is now a special case of SemanticsObjectContainer). I don't think it's possible to have a FlutterSemanticsScrollView root.
There was a problem hiding this comment.
Gemini said the framework won't produce such a semantics tree but the engine / embedder does not make such a guarantee.
| return [super isAccessibilityElement]; | ||
| } | ||
|
|
||
| - (id)accessibilityContainer { |
There was a problem hiding this comment.
This is the FlutterScrollableSemanticsObject class. It now reports a real UIScrollView as its a11y container.
| #pragma mark - Semantic object property accesser | ||
|
|
||
| - (NSArray<SemanticsObject*>*)children { | ||
| if (_inDealloc) { |
There was a problem hiding this comment.
drive by fix, but this should be safe.
| } | ||
|
|
||
| return [self isFocusable]; | ||
| // If the node is scrollable AND hidden OR |
There was a problem hiding this comment.
the isFocusable method is inlined.
There was a problem hiding this comment.
Code Review
This pull request updates the iOS accessibility implementation for scrollable regions by introducing a custom FlutterSemanticsScrollView that acts as an accessibility container for its children. This change ensures that scrollable elements are correctly recognized by UIKit's accessibility runtime and improves the accessibility hierarchy for nested elements. The changes include updates to SemanticsObject and FlutterSemanticsScrollView to handle hit testing and accessibility container logic, along with comprehensive unit tests to verify the new behavior. My feedback includes suggestions to optimize performance by caching weak property lookups into strong local variables and refining the hit-testing logic to avoid redundant coordinate conversions.
| SemanticsObject* semanticsObject = nil; | ||
| if ([element isKindOfClass:[SemanticsObjectContainer class]]) { | ||
| semanticsObject = ((SemanticsObjectContainer*)element).semanticsObject; | ||
| } else if ([element isKindOfClass:[FlutterSemanticsScrollView class]]) { |
There was a problem hiding this comment.
I am not opposed to this change, but just curious is it possible that the root element is a FlutterSemanticsScrollView? or this is just a safe precaution
| if (!self.semanticsObject.bridge) { | ||
| return nil; | ||
| } | ||
| if ([self.semanticsObject hasChildren]) { |
There was a problem hiding this comment.
I am a bit confused, we already have the branching logic in FlutterScrollableSemanticsObject which will pick the correct container based on whether there is children or not, couldn't we just delegate to self.semanticsObject.accessibilityContainer
There was a problem hiding this comment.
The FlutterScrollableSemanticsObject.accessibilityContainer implementation returns this scroll view as container. They are different nodes in the tree (the scroll view replaces the SemanticsObjectContainer in the tree, not FlutterScrollableSemanticsObject).
@implementation FlutterScrollableSemanticsObject
- (id)accessibilityContainer {
if ([self hasChildren]) {
if (!self.bridge) {
return nil;
}
// Expose `scrollView` directly as the container instead of wrapping it in a
// `SemanticsObjectContainer` so child elements have this `UIScrollView` as their
// `accessibilityContainer` ancestor.
return self.scrollView;
}
return [super accessibilityContainer];
}There was a problem hiding this comment.
If you look at SemanticsObjectContainer.accessibilityContainer:
- (id)accessibilityContainer {
return ([self.semanticsObject uid] == kRootNodeId)
? self.semanticsObject.bridgeView
: self.semanticsObject.parent.accessibilityContainer;
}
There was a problem hiding this comment.
what i meant is why don't we differ all the accessibilityContainer logic to the FlutterScrollableSemanticsObject.accessibilityContainer ?
They are different nodes in the tree
FlutterScrollableSemanticsObject is not on the final UIA tree sent to the iOS, only FlutterSemanticsScrollView is on the UIA tree.
I meant something like this
@implementation FlutterSemanticsScrollView
- (id)accessibilityContainer {
return self.semanticsObject.accessibilityContainer;
}
@end
@implementation FlutterScrollableSemanticsObject
- (id)accessibilityContainer {
if (!bridge) return nil;
if (hasChildren) {
return self.scrollView;
}
return parent.accessibilityContainer;
}
@end
There was a problem hiding this comment.
Ahh right FlutterScrollableSemanticsObject doesn't appear in the UIA tree. But I think the implementation above does not work because the FlutterSemanticsScrollView would then return itself as its own accessibility container. I think the FlutterScrollableSemanticsObject.accessibilityContainer implementation has to stay that way because it's used by its own children to get their container.
There was a problem hiding this comment.
ah i see, because the children may do things like
return self.parent.accessibilityContainer;
I wonder if it will be easier if we do something like
return self.parent.nativeAccessible.accessibilityContainer;
so that all the call will be directed to the node in UIA tree.
There was a problem hiding this comment.
Gemini was able to get rid of the hasChildren check in the scrollable semantics object and the UIScrollView subclass' accessibilityContainer implementation (added an assert and updated some comments).
There was a problem hiding this comment.
(The commit dismissed the approval so requesting another review)
| } | ||
|
|
||
| - (id)accessibilityContainer { | ||
| if ([self hasChildren]) { |
There was a problem hiding this comment.
Just curious when it has no child, can the uiscrollview still serve as an empty container? this way we don't need this kind of condition branch and just always return self.semanticsObject.parent.accessibilityContainer;
There was a problem hiding this comment.
When the scroll view has no children the nativeAccessibility of the scrollable SemanticsObject is the UIScrollView so I don't think we should make that the container too.
chunhtai
left a comment
There was a problem hiding this comment.
LGTM, left a comment in #195731 (comment)
but i am fine either way
|
Failed to create CP due to merge conflicts. |
flutter/flutter@09175cd...e36b370 2026-11-07 [email protected] Roll Fuchsia Linux SDK from KIc0XZbhLmvDpTluZ... to Sf2yvdTiFXAKkfLkD... (flutter/flutter#196160) 2026-11-07 [email protected] Roll pub packages (flutter/flutter#196155) 2026-11-07 [email protected] [iOS a11y] Use a real `UIScrollView` as a11y scrollable container (flutter/flutter#195731) 2026-11-07 [email protected] Add ParagraphStyle.hyphenCharacter to dart:ui (flutter/flutter#195345) 2026-11-07 [email protected] Roll pub packages (flutter/flutter#196143) 2026-11-06 [email protected] [Linux] Fix layer position when compositing without glBlitFramebuffer (flutter/flutter#196083) 2026-11-06 [email protected] Add support for riscv64 desktop linux engine (flutter/flutter#178712) 2026-11-06 [email protected] Add ResizeImagePolicy.cover (flutter/flutter#194665) 2026-11-06 [email protected] Roll Skia from f533d4034900 to b11315d0a096 (6 revisions) (flutter/flutter#196138) 2026-11-06 [email protected] iOS: Remove dead code/unused imports from FlutterPlatformViewsController (flutter/flutter#196084) 2026-11-06 [email protected] Replace pull_request_target workflows with pull_request + workflow_run (flutter/flutter#195536) 2026-11-06 [email protected] [master] Update Flutter DEPS to Dart 3b174b7217fe09b21da836ea623932b5f041b091 (flutter/flutter#196004) 2026-11-06 [email protected] Fix android_hardware_smoke_test retry mechanisms (flutter/flutter#195708) 2026-11-06 [email protected] Roll Skia from 452b9c1e5db6 to f533d4034900 (1 revision) (flutter/flutter#196129) 2026-11-06 [email protected] [flutter_tools] Use `ErrorHandlingFileSystem.deleteIfExists` for `IOSCoreDeviceControl` temp directory cleanup (flutter/flutter#195913) 2026-11-06 [email protected] Adds guideline tests to flutter create samples (flutter/flutter#195827) 2026-11-06 [email protected] refactor(flutter_tools): unify OptionDescriptor identity and addTo (flutter/flutter#196076) 2026-11-06 [email protected] Roll pub packages (flutter/flutter#196114) 2026-11-06 [email protected] Roll Skia from 5ce64e63a0e7 to 452b9c1e5db6 (5 revisions) (flutter/flutter#196111) 2026-11-06 [email protected] Add PlatformDispatcher.onAlwaysUse24HourFormatChanged callback (flutter/flutter#194670) 2026-11-06 [email protected] Roll Packages from 0478162 to 63881f9 (10 revisions) (flutter/flutter#196108) 2026-11-06 [email protected] Roll Skia from 793b8ac73d4b to 5ce64e63a0e7 (4 revisions) (flutter/flutter#196107) If this roll has caused a breakage, revert this CL and set the roller to dry run mode using the controls here: https://autoroll.skia.org/r/flutter-packages Please CC [email protected],[email protected] on the revert to ensure that a human is aware of the problem. To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose To report a problem with the AutoRoller itself, please file a bug: https://issues.skia.org/issues/new?component=1389291&template=1850622 Documentation for the AutoRoller is here: https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Fixes #192774 (and maybe other scrolling related a11y issues). I intend to cherry pick this patch to the beta branch if merged, since the diff is surprisingly small.
This PR does 3 things:
hiddenSemanticsObjectsglobalRectinstead of an empty rect forhiddenSemanticsObjectsFlutterSemanticsScrollView(a realUIScrollViewsubclass) as theaccessibilityContainerof scroll items, instead of the originalSemanticsObjectsubclass.Why is 3 needed?
Instead of exposing a11y APIs for scrollable containers, the iOS a11y system assumes all scrollable containers are
UIScrollViews and usesUIScrollViewAPIs to determine relevant geometry information (it relies on[obj isKindOfClass:[UIScrollView class]]checks):Concrete Binary Evidence That
UIScrollViewIs Hardcoded inUIAccessibility.framework&UIKit.axbundle(No Non-UIScrollViewWorkaround)Disassembling UIAccessibility.framework and UIKit.axbundle (iOS 27.1) proves that iOS explicitly checks isKindOfClass:[UIScrollView class] (loaded from GOT slot 0x221b44f40) and/or ___UIAccessibilityCastAsClass([UIScrollView class], ...) across the scroll, visibility, and opaque-element pipeline:
Evidence 1: _accessibilityVisibleFrame: explicitly checks [scrollParent isKindOfClass:[UIScrollView class]] (0x1F069DADC)
iOS separates an element's layout geometry (accessibilityFrame, AX attribute 2003) from its on-screen clipped geometry (_accessibilityVisibleFrame, AX attribute 2053 / -[VOTElement visibleFrame]):
In ___55-[NSObject(AXPrivCategory) _accessibilityVisibleFrame:]_block_invoke (UIAccessibility.framework @ 0x1F069DABC):
0x1F069DADC: adrp x8, 201895 ; 0x221b44000
0x1F069DAE0: ldr x0, [x8, #0xf40] ; OBJC_CLASS$_UIScrollView
0x1F069DAE4: bl _objc_opt_class
0x1F069DAE8: mov x1, x0
0x1F069DAEC: ldr x0, [sp, #0x8] ; scrollParent
0x1F069DAF0: bl _objc_opt_isKindOfClass ; [scrollParent isKindOfClass:[UIScrollView class]]
0x1F069DAF4: tbz w0, #0x0, 0x1f069db5c ; <-- IF NOT UIScrollView, SKIP CLIPPING ENTIRELY!
0x1F069DAF8: ldr x0, [sp, #0x8]
0x1F069DAFC: bl _objc_msgSend$_accessibilityContentFrame
...
0x1F069DB10: bl _CGRectIntersection ; Intersect child frame with UIScrollView._accessibilityContentFrame
Even if a custom NSObject container overrides _accessibilityIsScrollAncestor to return YES, _accessibilityVisibleFrame: refuses to clip children against it unless [scrollParent isKindOfClass:[UIScrollView class]] is true.
Evidence 2: _axFrameForBoundsCheck: (0x1F069E5C4) and _accessibilityScrollAncestorForSelector: (0x1F067FB24) explicitly check [scrollParent isKindOfClass:[UIScrollView class]]
Normally, UIAccessibility discards any element whose accessibilityFrame lies outside the UIWindow screen bounds (y > screenHeight or y < 0) as outOfBounds = YES. However, off-screen cached children inside a scroll view must have off-screen coordinates (y = 902, 958, ...) while still remaining eligible for VoiceOver swipe navigation. _axFrameForBoundsCheck:YES (UIAccessibility.framework @ 0x1F069E5C4) solves this by substituting the outermost scroll parent's on-screen frame in place of the child's off-screen frame during the screen-bounds check—if and only if [scrollParent isKindOfClass:[UIScrollView class]] (0x1F069E66C):
0x1F069E648: bl _objc_msgSend$_axOutermostScrollParent
0x1F069E650: mov x20, x0
0x1F069E654: cbz x0, 0x1f069e6b8
0x1F069E658: adrp x8, 201894 ; 0x221b44000
0x1F069E65C: ldr x0, [x8, #0xf40] ; OBJC_CLASS$_UIScrollView
0x1F069E660: bl _objc_opt_class
0x1F069E664: mov x1, x0
0x1F069E668: mov x0, x20
0x1F069E66C: bl _objc_opt_isKindOfClass ; [scrollParent isKindOfClass:[UIScrollView class]]
When a user focuses a child item and performs a 3-finger swipe (accessibilityScroll:) or when VoiceOver triggers scroll-to-visible, UIAccessibility walks up the hierarchy via _accessibilityScrollAncestorForSelector:alwaysAllowRefreshAction: (0x1F067FB24) and _accessibilityFindScrollParent: (0x1F0680E00), both of which check [ancestor isKindOfClass:[UIScrollView class]] (0x1F067FBC4 and 0x1F0680E80).
Evidence 3: _accessibilityOpaqueElementScrollToDirection: & _accessibilityScrollOpaqueElementIntoView:previousScroller: require UIScrollView (0x1F06983C8 & 0x1F0697744)
When VoiceOver reaches the last cached child inside an opaque scroll container (or when entering an opaque scroll container whose current viewport has no visible children) during _accessibilitySearchSubtreesAfterChildElement:..., UIAccessibility calls _accessibilityOpaqueElementScrollToDirection: to advance the scroll container's contentOffset so that newly-materialized lazy items can be discovered and focused.
Evidence 4: -[UIScrollViewAccessibility _accessibilityFirstLastOpaqueElement:options:] in UIKit.axbundle (0x1C8DFA64C)
Controls which child receives initial VoiceOver focus when swiping into a scroll view from an adjacent sibling outside the scroll view (invoked via _accessibilityFirstOpaqueElementWithOptions: at 0x1C8DFA9DC when swiping right/down into the scroll view, or _accessibilityLastOpaqueElementWithOptions: at 0x1C8DFB5D0 when swiping left/up into the scroll view from below).
The base -[NSObject(AXPrivCategory) _accessibilityFirstLastOpaqueElement:options:] (0x1F0696488) does not filter by viewport visibility—it simply returns firstObject or lastObject of _accessibilitySortedElementsWithinWithOptions:. Only UIScrollViewAccessibility (+[UIScrollViewAccessibility safeCategoryTargetClassName] -> @"UIScrollView") overrides _accessibilityFirstLastOpaqueElement:options: to query [UIScrollView visibleBounds] (0x1C8DFA77C), intersect each child's accessibilityFrame with visibleBounds in screen coordinates (0x1C8DFA898), and skip off-screen cached children whose visible intersection is smaller than childHeight / _accessibilityVisibleItemDenominator (0x1C8DFA8E0).
When
accessibilityFramereturns[self globalRect]without a UIScrollView ancestor in the accessibilityContainer chain,-[NSObject(AXPrivCategory) _accessibilityVisibleFrame:]has[[self _axOutermostScrollParent] _accessibilityContentFrame] == nil. As a result, it only clipsaccessibilityFrameagainst the screen bounds rather than the scroll view's viewport. Which means those hidden elements will get a label when voice control is on, among other things.With this patch:
voiceover_scroll_fix.mp4
Pre-launch Checklist
///).If you need help, consider asking for advice on the #hackers-new channel on Discord.
If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.
Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the
gemini-code-assistbot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.