Skip to content

Commit aae258a

Browse files
smocerLongCatIsLooongcbracken
authored
[iOS] Rebind accessibility bridge after view controller changes (flutter#187167)
Keep the existing `AccessibilityBridge` when `PlatformViewIOS` detaches from an owner `FlutterViewController`, and rebind it when a controller/view is attached again. In add-to-app integrations that reuse a `FlutterEngine`, the engine can detach from one `FlutterViewController` and later attach to another one. When semantics are already enabled, clearing the bridge during detach drops the UIKit accessibility representation of the current semantics tree. The Flutter UI continues to render, but VoiceOver / Accessibility Inspector can no longer see Flutter fields and buttons after reattachment. This change preserves the bridge across owner-controller detach/reattach, skips semantics updates while no owner controller is attached, and rebinds the bridge to the new owner controller/view before semantics updates resume. Fixes flutter#186582 ## Pre-launch Checklist - [x] I read the [Contributor Guide] and followed the process outlined there for submitting PRs. - [x] I read the [AI contribution guidelines] and understand my responsibilities, or I am not using AI tools. - [x] I read the [Tree Hygiene] wiki page, which explains my responsibilities. - [x] I read and followed the [Flutter Style Guide], including [Features we expect every widget to implement]. - [x] I signed the [CLA]. - [x] I listed at least one issue that this PR fixes in the description above. - [x] I updated/added relevant documentation (doc comments with `///`). - [x] I added new tests to check the change I am making, or this PR is [test-exempt]. - [x] I followed the [breaking change policy] and added [Data Driven Fixes] where supported. - [x] All existing and new tests are passing. [Contributor Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview [AI contribution guidelines]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines [Tree Hygiene]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md [test-exempt]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests [Flutter Style Guide]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md [Features we expect every widget to implement]: https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement [CLA]: https://cla.developers.google.com/ [breaking change policy]: https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes [Data Driven Fixes]: https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md --------- Co-authored-by: Yegor Butyrin <[email protected]> Co-authored-by: LongCatIsLooong <[email protected]> Co-authored-by: Chris Bracken <[email protected]>
1 parent 332efcd commit aae258a

13 files changed

Lines changed: 1004 additions & 108 deletions

‎engine/src/flutter/shell/platform/darwin/ios/framework/Headers/FlutterEngine.h‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -332,8 +332,10 @@ FLUTTER_DARWIN_EXPORT
332332
* You can subscribe to semantics updates via `NSNotificationCenter` by adding
333333
* an observer for the name `FlutterSemanticsUpdateNotification`. The `object`
334334
* parameter will be the `FlutterViewController` associated with the semantics
335-
* update. This will asynchronously fire after a semantics tree has actually
336-
* built (which may be some time after the frame has been rendered).
335+
* update. This will asynchronously fire after a semantics tree has actually
336+
* built and has been applied to a loaded Flutter view (which may be some time after the frame has
337+
* been rendered). If the view is not loaded when the semantics tree is built, the notification is
338+
* deferred until the cached semantics are applied when the view loads.
337339
*/
338340
- (void)ensureSemanticsEnabled;
339341

‎engine/src/flutter/shell/platform/darwin/ios/framework/Headers/FlutterViewController.h‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,10 @@ NS_ASSUME_NONNULL_BEGIN
2323
/**
2424
* The name used for semantic update notifications via `NSNotificationCenter`.
2525
*
26+
* A notification is posted after a semantics update is applied to a loaded Flutter view. If an
27+
* update arrives before the associated `FlutterViewController` loads its view, the notification is
28+
* deferred until the cached semantics are applied when the view loads.
29+
*
2630
* The object passed as the sender is the `FlutterViewController` associated
2731
* with the update.
2832
*/

‎engine/src/flutter/shell/platform/darwin/ios/framework/Source/SemanticsObject.h‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -120,6 +120,14 @@ constexpr float kScrollExtentMaxForInf = 1000;
120120
*/
121121
- (void)accessibilityBridgeDidFinishUpdate;
122122

123+
/**
124+
* Called when the root view associated with this node's accessibility bridge changes.
125+
*
126+
* Subclasses can override this method if they own native UIKit views that must
127+
* remain attached to the current Flutter view.
128+
*/
129+
- (void)accessibilityBridgeDidChangeView;
130+
123131
#pragma mark - Designated initializers
124132

125133
- (instancetype)init __attribute__((unavailable("Use initWithBridge instead")));

‎engine/src/flutter/shell/platform/darwin/ios/framework/Source/SemanticsObject.mm‎

Lines changed: 57 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -151,6 +151,9 @@ - (UIAccessibilityTraits)accessibilityTraits {
151151

152152
@interface FlutterScrollableSemanticsObject ()
153153
@property(nonatomic) FlutterSemanticsScrollView* scrollView;
154+
- (UIView*)reattachScrollViewToCurrentBridgeView;
155+
- (void)updateScrollViewForCurrentBridgeView;
156+
- (void)updateScrollViewGeometry;
154157
@end
155158

156159
@implementation FlutterScrollableSemanticsObject
@@ -164,10 +167,7 @@ - (instancetype)initWithBridge:(fml::WeakPtr<flutter::AccessibilityBridgeIos>)br
164167
[_scrollView setShowsVerticalScrollIndicator:NO];
165168
[_scrollView setContentInset:UIEdgeInsetsZero];
166169
[_scrollView setContentInsetAdjustmentBehavior:UIScrollViewContentInsetAdjustmentNever];
167-
UIView* containerView = self.bridgeView;
168-
if (containerView) {
169-
[containerView addSubview:_scrollView];
170-
}
170+
[self reattachScrollViewToCurrentBridgeView];
171171
}
172172
return self;
173173
}
@@ -177,6 +177,36 @@ - (void)dealloc {
177177
}
178178

179179
- (void)accessibilityBridgeDidFinishUpdate {
180+
[self updateScrollViewForCurrentBridgeView];
181+
}
182+
183+
- (void)accessibilityBridgeDidChangeView {
184+
[self updateScrollViewForCurrentBridgeView];
185+
}
186+
187+
- (id)nativeAccessibility {
188+
return _scrollView;
189+
}
190+
191+
// private methods
192+
193+
- (UIView*)reattachScrollViewToCurrentBridgeView {
194+
UIView* view = self.bridgeView;
195+
if (_scrollView.superview != view) {
196+
[_scrollView removeFromSuperview];
197+
[view addSubview:_scrollView];
198+
}
199+
return view;
200+
}
201+
202+
- (void)updateScrollViewForCurrentBridgeView {
203+
UIView* view = [self reattachScrollViewToCurrentBridgeView];
204+
if (view) {
205+
[self updateScrollViewGeometry];
206+
}
207+
}
208+
209+
- (void)updateScrollViewGeometry {
180210
// In order to make iOS think this UIScrollView is scrollable, the following
181211
// requirements must be true.
182212
// 1. contentSize must be bigger than the frame size.
@@ -185,20 +215,14 @@ - (void)accessibilityBridgeDidFinishUpdate {
185215
// Once the requirements are met, the iOS uses contentOffset to determine
186216
// what scroll actions are available. e.g. If the view scrolls vertically and
187217
// contentOffset is 0.0, only the scroll down action is available.
188-
self.scrollView.frame = self.accessibilityFrame;
189-
self.scrollView.contentSize = [self contentSizeInternal];
218+
_scrollView.frame = self.accessibilityFrame;
219+
_scrollView.contentSize = [self contentSizeInternal];
190220
// See the documentation on `isDoingSystemScrolling`.
191-
if (!self.scrollView.isDoingSystemScrolling) {
192-
[self.scrollView setContentOffset:self.contentOffsetInternal animated:NO];
221+
if (!_scrollView.isDoingSystemScrolling) {
222+
[_scrollView setContentOffset:self.contentOffsetInternal animated:NO];
193223
}
194224
}
195225

196-
- (id)nativeAccessibility {
197-
return self.scrollView;
198-
}
199-
200-
// private methods
201-
202226
- (float)scrollExtentMax {
203227
if (!self.bridge) {
204228
return 0.0f;
@@ -240,7 +264,7 @@ - (CGSize)contentSizeInternal {
240264

241265
- (CGPoint)contentOffsetInternal {
242266
CGPoint result;
243-
CGPoint origin = self.scrollView.frame.origin;
267+
CGPoint origin = _scrollView.frame.origin;
244268
const SkRect& rect = self.node.rect;
245269
if (self.node.actions & flutter::kVerticalScrollSemanticsActions) {
246270
result = ConvertPointToGlobal(self, CGPointMake(rect.x(), rect.y() + [self scrollPosition]));
@@ -278,11 +302,17 @@ - (instancetype)initWithBridge:(fml::WeakPtr<flutter::AccessibilityBridgeIos>)br
278302
uid:(int32_t)uid {
279303
FML_DCHECK(bridge) << "bridge must be set";
280304
FML_DCHECK(uid >= kRootNodeId);
305+
flutter::AccessibilityBridgeIos* accessibility_bridge = bridge.get();
306+
if (!accessibility_bridge) {
307+
return nil;
308+
}
281309
// Initialize with the UIView as the container.
282310
// The UIView will not necessarily be accessibility parent for this object.
283311
// The bridge informs the OS of the actual structure via
284312
// `accessibilityContainer` and `accessibilityElementAtIndex`.
285-
self = [super initWithAccessibilityContainer:bridge ? bridge->view() : nil];
313+
UIView* accessibilityContainer =
314+
accessibility_bridge->AccessibilityElementInitializationContainer();
315+
self = [super initWithAccessibilityContainer:accessibilityContainer];
286316

287317
if (self) {
288318
_weakBridge = bridge;
@@ -346,7 +376,7 @@ - (BOOL)hasChildren {
346376

347377
- (UIView*)bridgeView {
348378
flutter::AccessibilityBridgeIos* bridge = [self bridge];
349-
return bridge ? bridge->view() : nil;
379+
return bridge ? bridge->ViewIfLoaded() : nil;
350380
}
351381

352382
- (void)setSemanticsNode:(const flutter::SemanticsNode*)node {
@@ -356,6 +386,9 @@ - (void)setSemanticsNode:(const flutter::SemanticsNode*)node {
356386
- (void)accessibilityBridgeDidFinishUpdate { /* Do nothing by default */
357387
}
358388

389+
- (void)accessibilityBridgeDidChangeView { /* Do nothing by default */
390+
}
391+
359392
/**
360393
* Whether calling `setSemanticsNode:` with `node` would cause a layout change.
361394
*/
@@ -925,11 +958,17 @@ @implementation SemanticsObjectContainer {
925958

926959
- (instancetype)initWithSemanticsObject:(SemanticsObject*)semanticsObject {
927960
FML_DCHECK(semanticsObject) << "semanticsObject must be set";
961+
flutter::AccessibilityBridgeIos* accessibility_bridge = semanticsObject.bridge;
962+
if (!accessibility_bridge) {
963+
return nil;
964+
}
928965
// Initialize with the UIView as the container.
929966
// The UIView will not necessarily be accessibility parent for this object.
930967
// The bridge informs the OS of the actual structure via
931968
// `accessibilityContainer` and `accessibilityElementAtIndex`.
932-
self = [super initWithAccessibilityContainer:semanticsObject.bridgeView];
969+
UIView* accessibilityContainer =
970+
accessibility_bridge->AccessibilityElementInitializationContainer();
971+
self = [super initWithAccessibilityContainer:accessibilityContainer];
933972

934973
if (self) {
935974
_semanticsObject = semanticsObject;

‎engine/src/flutter/shell/platform/darwin/ios/framework/Source/SemanticsObjectTest.mm‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -799,6 +799,90 @@ - (void)testSemanticsObjectContainerAccessibilityFrameCoversEntireScreen {
799799
XCTAssertTrue(CGRectEqualToRect(container.accessibilityFrame, UIScreen.mainScreen.bounds));
800800
}
801801

802+
- (void)testRootSemanticsObjectContainerAccessibilityContainerIsBridgeView {
803+
flutter::testing::MockAccessibilityBridge* mock = new flutter::testing::MockAccessibilityBridge();
804+
mock->isVoiceOverRunningValue = true;
805+
fml::WeakPtrFactory<flutter::AccessibilityBridgeIos> factory(mock);
806+
fml::WeakPtr<flutter::AccessibilityBridgeIos> bridge = factory.GetWeakPtr();
807+
808+
flutter::SemanticsNode root;
809+
root.id = kRootNodeId;
810+
811+
FlutterSemanticsObject* rootObject = [[FlutterSemanticsObject alloc] initWithBridge:bridge
812+
uid:kRootNodeId];
813+
[rootObject setSemanticsNode:&root];
814+
815+
SemanticsObjectContainer* container =
816+
static_cast<SemanticsObjectContainer*>(rootObject.accessibilityContainer);
817+
818+
XCTAssertEqual(container.accessibilityContainer, mock->view());
819+
}
820+
821+
- (void)testBridgeViewIsNilWhileBridgeIsAliveAndViewIsNotLoaded {
822+
flutter::testing::MockAccessibilityBridge* mock = new flutter::testing::MockAccessibilityBridge();
823+
mock->isViewLoaded = false;
824+
fml::WeakPtrFactory<flutter::AccessibilityBridgeIos> factory(mock);
825+
fml::WeakPtr<flutter::AccessibilityBridgeIos> bridge = factory.GetWeakPtr();
826+
827+
flutter::SemanticsNode root;
828+
root.id = kRootNodeId;
829+
root.rect = SkRect::MakeXYWH(0, 0, 100, 100);
830+
831+
FlutterSemanticsObject* rootObject = [[FlutterSemanticsObject alloc] initWithBridge:bridge
832+
uid:kRootNodeId];
833+
[rootObject setSemanticsNode:&root];
834+
835+
XCTAssertNotNil(mock->view());
836+
XCTAssertTrue(rootObject.bridge != nullptr);
837+
XCTAssertNil(rootObject.bridgeView);
838+
XCTAssertTrue(CGRectEqualToRect(rootObject.accessibilityFrame, CGRectZero));
839+
840+
SemanticsObjectContainer* container =
841+
static_cast<SemanticsObjectContainer*>(rootObject.accessibilityContainer);
842+
XCTAssertNotNil(container);
843+
XCTAssertNil(container.accessibilityContainer);
844+
}
845+
846+
- (void)testChildSemanticsObjectContainerAccessibilityContainerIsParentContainer {
847+
flutter::testing::MockAccessibilityBridge* mock = new flutter::testing::MockAccessibilityBridge();
848+
mock->isVoiceOverRunningValue = true;
849+
fml::WeakPtrFactory<flutter::AccessibilityBridgeIos> factory(mock);
850+
fml::WeakPtr<flutter::AccessibilityBridgeIos> bridge = factory.GetWeakPtr();
851+
852+
flutter::SemanticsNode root;
853+
root.id = kRootNodeId;
854+
root.childrenInTraversalOrder.push_back(1);
855+
856+
flutter::SemanticsNode child;
857+
child.id = 1;
858+
child.childrenInTraversalOrder.push_back(2);
859+
860+
flutter::SemanticsNode grandchild;
861+
grandchild.id = 2;
862+
863+
FlutterSemanticsObject* rootObject = [[FlutterSemanticsObject alloc] initWithBridge:bridge
864+
uid:kRootNodeId];
865+
[rootObject setSemanticsNode:&root];
866+
867+
FlutterSemanticsObject* childObject = [[FlutterSemanticsObject alloc] initWithBridge:bridge
868+
uid:1];
869+
[childObject setSemanticsNode:&child];
870+
871+
FlutterSemanticsObject* grandchildObject = [[FlutterSemanticsObject alloc] initWithBridge:bridge
872+
uid:2];
873+
[grandchildObject setSemanticsNode:&grandchild];
874+
875+
rootObject.children = @[ childObject ];
876+
childObject.children = @[ grandchildObject ];
877+
878+
SemanticsObjectContainer* rootContainer =
879+
static_cast<SemanticsObjectContainer*>(rootObject.accessibilityContainer);
880+
SemanticsObjectContainer* childContainer =
881+
static_cast<SemanticsObjectContainer*>(childObject.accessibilityContainer);
882+
883+
XCTAssertEqual(childContainer.accessibilityContainer, rootContainer);
884+
}
885+
802886
- (void)testFlutterSemanticsObjectAttributedStringsDoNotCrashWhenEmpty {
803887
flutter::testing::MockAccessibilityBridge* mock = new flutter::testing::MockAccessibilityBridge();
804888
mock->isVoiceOverRunningValue = true;

‎engine/src/flutter/shell/platform/darwin/ios/framework/Source/SemanticsObjectTestMocks.h‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,8 @@ class MockAccessibilityBridge : public AccessibilityBridgeIos {
3838
}
3939
bool isVoiceOverRunning() const override { return isVoiceOverRunningValue; }
4040
UIView* view() const override { return view_; }
41+
UIView* ViewIfLoaded() const override { return isViewLoaded ? view_ : nil; }
42+
UIView* AccessibilityElementInitializationContainer() const override { return view_; }
4143
UIView<UITextInput>* textInputView() override { return nil; }
4244

4345
NSString* GetDefaultLocale() override { return mockedLocale; }
@@ -55,6 +57,7 @@ class MockAccessibilityBridge : public AccessibilityBridgeIos {
5557
void AccessibilityObjectDidLoseFocus(int32_t id) override {}
5658
FlutterPlatformViewsController* GetPlatformViewsController() const override { return nil; }
5759
std::vector<SemanticsActionObservation> observations;
60+
bool isViewLoaded = true;
5861
bool isVoiceOverRunningValue;
5962
NSString* mockedLocale;
6063

@@ -70,6 +73,8 @@ class MockAccessibilityBridgeNoWindow : public AccessibilityBridgeIos {
7073
}
7174
bool isVoiceOverRunning() const override { return isVoiceOverRunningValue; }
7275
UIView* view() const override { return view_; }
76+
UIView* ViewIfLoaded() const override { return isViewLoaded ? view_ : nil; }
77+
UIView* AccessibilityElementInitializationContainer() const override { return view_; }
7378
UIView<UITextInput>* textInputView() override { return nil; }
7479
void DispatchSemanticsAction(int32_t id, SemanticsAction action) override {
7580
SemanticsActionObservation observation(id, action);
@@ -86,6 +91,7 @@ class MockAccessibilityBridgeNoWindow : public AccessibilityBridgeIos {
8691
NSString* GetDefaultLocale() override { return nil; }
8792
FlutterPlatformViewsController* GetPlatformViewsController() const override { return nil; }
8893
std::vector<SemanticsActionObservation> observations;
94+
bool isViewLoaded = true;
8995
bool isVoiceOverRunningValue;
9096

9197
private:

‎engine/src/flutter/shell/platform/darwin/ios/framework/Source/accessibility_bridge.h‎

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -27,8 +27,9 @@ namespace flutter {
2727
class PlatformViewIOS;
2828

2929
/**
30-
* An accessibility instance is bound to one `FlutterViewController` and
31-
* `FlutterView` instance.
30+
* An accessibility instance is bound to the `FlutterEngine` through
31+
* `PlatformViewIOS` and can be rebound to one `FlutterViewController` at a
32+
* time.
3233
*
3334
* It helps populate the UIView's accessibilityElements property from Flutter's
3435
* semantics nodes.
@@ -53,8 +54,14 @@ class AccessibilityBridge final : public AccessibilityBridgeIos {
5354
std::unique_ptr<IosDelegate> ios_delegate = nullptr);
5455
~AccessibilityBridge();
5556

57+
// Rebinds the cached semantics tree from `previousView` to `viewController` without recreating
58+
// the bridge. A non-nil new view controller must belong to the same engine because the platform
59+
// views controller is fixed for the lifetime of the bridge.
60+
void SetViewController(FlutterViewController* viewController, FlutterView* previousView);
5661
void UpdateSemantics(flutter::SemanticsNodeUpdates nodes,
5762
const flutter::CustomAccessibilityActionUpdates& actions);
63+
// Returns true when a root semantics object is cached in the bridge.
64+
bool HasSemantics() const;
5865
void HandleEvent(NSDictionary<NSString*, id>* annotatedEvent);
5966
void DispatchSemanticsAction(int32_t id, flutter::SemanticsAction action) override;
6067
void DispatchSemanticsAction(int32_t id,
@@ -68,6 +75,12 @@ class AccessibilityBridge final : public AccessibilityBridgeIos {
6875

6976
UIView* view() const override { return view_controller_.view; }
7077

78+
UIView* ViewIfLoaded() const override { return view_controller_.viewIfLoaded; }
79+
80+
UIView* AccessibilityElementInitializationContainer() const override {
81+
return ViewIfLoaded() ?: accessibility_element_init_container_;
82+
}
83+
7184
bool isVoiceOverRunning() const override { return view_controller_.isVoiceOverRunning; }
7285

7386
fml::WeakPtr<AccessibilityBridge> GetWeakPtr();
@@ -79,6 +92,10 @@ class AccessibilityBridge final : public AccessibilityBridgeIos {
7992
void clearState();
8093

8194
private:
95+
bool AccessibilityElementsWereInstalledByBridge(NSArray* elements) const;
96+
void ClearAccessibilityElementsIfOwnedByBridge(UIView* view);
97+
void UpdateAccessibilityElementsForCurrentView();
98+
void NotifySemanticsObjectsViewChanged();
8299
SemanticsObject* GetOrCreateObject(int32_t id, flutter::SemanticsNodeUpdates& updates);
83100
SemanticsObject* FindNextFocusableIfNecessary();
84101
// Finds the first focusable SemanticsObject rooted at the parent. This includes the parent itself
@@ -90,6 +107,7 @@ class AccessibilityBridge final : public AccessibilityBridgeIos {
90107
NSMutableArray<NSNumber*>* doomed_uids);
91108

92109
__weak FlutterViewController* view_controller_;
110+
UIView* accessibility_element_init_container_;
93111
PlatformViewIOS* platform_view_;
94112
__weak FlutterPlatformViewsController* platform_views_controller_;
95113

0 commit comments

Comments
 (0)