[WIP] Semantics integration test for Android #19005
Conversation
|
cc @yjbanov @goderbauer - I'm not quite sure what I need to do to correctly wire this up as an integration test. This also includes some additions to flutter_test which I think are related to overall testing improvements, but I can split that out into a separate PR if that would be easier to track |
goderbauer
left a comment
goderbauer
left a comment
There was a problem hiding this comment.
+1 for adding integrations tests. We should figure out a similar way for iOS.
Also, can we split this into at least 3 PRs for easier reviewing:
- PR for the changes to flutter_test
- PR for the changes to flutter_driver
- PR for adding the actual integration tests
| Widget build(BuildContext context) { | ||
| return new MaterialApp( | ||
| routes: <String, WidgetBuilder>{ | ||
| 'SelectionControls': (BuildContext context) => new SelectionControlsPage(), |
There was a problem hiding this comment.
Should these maybe be static consts on SelectionControlPage and TextFieldsPage? e.g. SelectionControlPage.routeName and TextFieldsPage.routeName.
| result.put("flags", flags); | ||
| Rect nodeRect = new Rect(); | ||
| node.getBoundsInScreen(nodeRect); | ||
| rect.put("left", nodeRect.left / mScreenDensity); |
There was a problem hiding this comment.
Why do we need to correct these values with the density?
There was a problem hiding this comment.
This is just so the rects will be in the same space as the rects on the SemanticsData. So you could check that a checkbox is 48 by 48
| /// A checkbox widget. | ||
| static const String checkBox = 'android.widget.CheckBox'; | ||
|
|
||
| /// The default className if none is provided by flutter. |
| static const String textView = 'android.widget.TextView'; | ||
| } | ||
|
|
||
| /// Action constants Taken from [AccessibilityAction]. |
There was a problem hiding this comment.
This is Android's android.view.accessibility.AccessibilityNodeInfo.AccessibilityAction? Maybe make that clearer? Also, I am not sure if dart doc can link to that, so maybe use enclose it in `` instead of []?
| return id == typedOther.id; | ||
| } | ||
|
|
||
| /// Taken from [AccessibilityAction.ACTION_FOCUS]. |
| testWidgets('BackButton semantics', (WidgetTester tester) async { | ||
| final SemanticsHandle handle = tester.ensureSemantics(); | ||
| await tester.pumpWidget( | ||
| new MaterialApp( |
There was a problem hiding this comment.
nit: add trailing commas and only indent by two spaces.
There was a problem hiding this comment.
| return result.changedState; | ||
| } | ||
|
|
||
| Future<int> getSemanticsId(SerializableFinder finder, { Duration timeout = _kShortTimeout}) async { |
|
|
||
|
|
||
| /// A Flutter driver command that retrieves a semantics id using a specified | ||
| /// finder. |
There was a problem hiding this comment.
What happens if the thing identified by finder doesn't own a semantics node?
| /// A Flutter driver command that retrieves a semantics id using a specified | ||
| /// finder. | ||
| /// | ||
| /// Semantics must be enabled before using this command. |
There was a problem hiding this comment.
Link to how to enable semantics?
| await tap(backButton); | ||
| } | ||
|
|
||
| /// Attempts to find the [SemanticsData] of first result from [finder]. |
There was a problem hiding this comment.
document what happens if the object identified by finder doesn't own its own semantics node?
There was a problem hiding this comment.
I'd even make it 4 PRs: |
|
It seems a bit... overly complicated that we are sending the semantic data from the framework to the Java code, then back to the framework, then to the driver. Ideally we'd want to check the actual data from Android, as if we were Talkback. Is there some way we can do that instead? Reading the data using the Android accessibility APIs directly? |
|
I don't believe we can use TalkBack directly - in theory we could write our own accessibility service, but there is the added complexity of trying to synchronize three separate processes instead of just two. Ultimately TalkBack just sees the (maybe this is technically an end to end test?) |
|
How do regular android apps test their accessibility? |
|
A combination of manual testing and a few automated lints like missing labels |
5 participants
This test still needs more config, and requires talkback to be enabled to function. This can be done through adb:
adb shell settings put secure enabled_accessibility_services com.google.android.marvin.talkback/com.google.android.marvin.talkback.TalkBackServicecan be disabled (apparently) with same command replaced with null for the value
Also adds additional finders/matcher for semantics