Background

The Finder API is based off of navigating the Element tree. For interacting with the Semantics tree, we've been using tester.semantics.find (previously tester.getSemantics) which is based on the Finder API and is implemented as follows:

• Use the Finder to get a singular Element
• On that Element, use findRenderObject to get the closest render object
  • The findRenderObject method will return the element if it happens to also be a render object, or the nearest descendant of the element that is a RenderObject
• Once we have the closest RenderObject, find the related SemanticsNode to that RenderObject
  • If the current RenderObject does not have a node, or that node is part of a merged node, we navigate up the RenderObject tree until we find a standalone SemanticsNode

The important parts here to note are:

• We navigate down the RenderObject tree to find a RenderObject from an Element
  • This RenderObject is not guaranteed to have a SemanticsNode
• We then navigate up the RenderObject tree to find a SemanticsNode

The problem

TLDR;

Translating between the Element, RenderObject, and Semantics trees isn't consistent enough to be able to use the existing Finder API (which is based on the Element tree) to find specific SemanticsNodes in the Semantics tree.

Deep dive

The current method of finding SemanticsNodes related to the Element tree is a heuristic that doesn't consistently return the SemanticsNode that one would expect. For example, take a look at the following test:

void main() {
  testWidgets('Scratch', (WidgetTester tester) async {
    await tester.pumpWidget(const MaterialApp(
      home: Material(child: TextField()),
    ));

    expect(
      tester.semantics.find(find.byType(TextField)),
      containsSemantics(isTextField: true),
    );
  });
}

This test is very straightforward. When we try to find the SemanticsNode for a TextField the isTextField flag should be true, but this test ends up failing with the current implementation of find.

What went wrong

Looking at the implementation of find we can see a potential issue there that might resolve the issue. The findRenderObject method just looks for the closest descendent RenderObject. It doesn't really care whether or not that RenderObject has any semantics attached to it.

Turns out this is exactly the issue for the TextField case. The RenderObject returned is a RenderMouseRegion with no semantics attached to it. Looking at the RenderObject tree shows us that a couple steps further down the tree we can find a semantically relevant node:

RenderSemanticsAnnotations#efb04
│ needs compositing
│ creator: Semantics ← AnimatedBuilder ← IgnorePointer ←
│   TextFieldTapRegion ← MouseRegion ← TextField ← DefaultTextStyle
│   ← AnimatedDefaultTextStyle ← _InkFeatures-[GlobalKey#95de3 ink
│   renderer] ← NotificationListener<LayoutChangedNotification> ←
│   PhysicalModel ← AnimatedPhysicalModel ← ⋯
│ parentData: <none> (can use size)
│ constraints: BoxConstraints(w=800.0, h=600.0)
│ semantics node: SemanticsNode#4
│ size: Size(800.0, 600.0)

Which relates to the SemanticsNode we would expect:

└─SemanticsNode#4
    Rect.fromLTRB(0.0, 0.0, 800.0, 600.0)
    actions: tap
    flags: isTextField
    textDirection: ltr
    currentValueLength: 0

So, this gives us a potential solution! Lets adapt find to not just look for the closest descendant RenderObject but the closest descendant that also has semantics. Here's our new test with a fixedFind method:

void main() {
  testWidgets('Scratch', (WidgetTester tester) async {
    await tester.pumpWidget(const MaterialApp(
      home: Material(child: TextField()),
    ));

    debugDumpSemanticsTree();
    debugPrint(tester.semantics.find(find.byType(TextField)).toString());

    expect(
      fixedFind(find.byType(TextField)),
      containsSemantics(isTextField: true),
    );
  });
}

fixedFind(Finder finder) {
    final Iterable<Element> candidates = finder.evaluate();
    final Element element = candidates.single;

    // Find the closest render object that has semantics data.
    final List<RenderObject> nextQueue = <RenderObject>[];
    RenderObject? renderObject = element.findRenderObject();
    SemanticsNode? result = renderObject?.debugSemantics;
    while (renderObject != null && result == null) {
      renderObject.visitChildren(nextQueue.add);
      if (nextQueue.isEmpty) {
        renderObject = null;
      } else {
        renderObject = nextQueue.removeAt(0);
        result = renderObject.debugSemantics;
      }
    }

    if (result == null) {
      throw StateError('No semantics data found.');
    }

    // Get the merged node if the related node is part of a merger
    while(result!.isMergedIntoParent) {
      result = result.parent;
    }
    return result;
}

And tada! The test now passes and find returns the SemanticsNode that we would expect from a TextField!

Success?

We solved the issue where we got the ancestor of the expected SemanticsNode! Perfect! Now let's make sure it works everywhere. Let's start with this test:

void main() {
  testWidgets('Scratch', (WidgetTester tester) async {
    await tester.pumpWidget( MaterialApp(
      home: ListView(
        children: [1, 2, 3, 4]
          .map((i) => Text('Testing $i', semanticsLabel: 'Semantics $i'))
          .toList(),
        ),
    ));

    expect(
      fixedFind(find.text('Testing 1')),
      containsSemantics(label: 'Testing 1'),
    );
  });
}

Nothing particularly out of the ordinary here, we've got a list of items, we're finding one of those items by text and making sure it has the correct semantics label. Unfortunately, this test now fails with a StateError saying: "No semantics data found.". Looking at the implementation of fixedFind, we can see that this happens when we've walked down the RenderObject tree and found no RenderObject that has a related SemanticsNode.

If you dig really deep, the root cause for this specific case is that being in an indexed scrollable moves the semantics up the render tree from the Element that is found by the find.text finder. If the text is not in an indexed scrollable, the semantics are down the tree where the implementation of fixedFind would expect it.

That just adds to the inconsistency of the Element -> RenderObject -> Semantics translation though. In the end, there isn't a heuristic for this translation that we could use that we could guarantee we're getting the "right" SemanticsNode for a given Element.

The solution

Create an API that is parallel to the Element based Finder API, but specifically for searching the Semantics tree.

• SemanticsFinder will be based around the Finder API
  • Possibly by extracting a FinderBase<TEvaluation> interface, though this isn't strictly necessary
• The finders defined by flutter_test will be exposed through a CommonSemanticsFinders class, and be exposed through a find.semantics property on the existing global find property
  • This mirrors the addition of the semantics property to WidgetController in 107866: Add support for verifying SemanticsNode ordering in widget tests #113133 while not requiring a secondary import, and not polluting the global namespace with a new semantics property.
• For an initial implementation, we'll need the following finders:
  • ancestor
    • Takes two SemanticsFinders and evaluates to all ancestors of of that meet the expectations of matching, optionally inclusive with matchesRoot
  • descendant
    • Takes two SemanticsFinders and evaluates to all descendants of of that meet the expectations of matching, optionally inclusive with matchesRoot
  • byLabel
    • Evaluates to all SemanticsNodes in the tree that has a label matching the given Pattern.
  • byValue
    • Evaluates to all SemanticsNodes in the tree that has a value matching the given Pattern.
  • byHint
    • Evaluates to all SemanticsNodes in the tree that has a hint matching the given Pattern.
  • byAction
    • Evaluates to all SemanticsNodes in the tree that has the given SemanticsAction.
  • byAnyAction
    • Evaluates to all SemanticsNodes in the tree that has at least one of the given SemanticsActions.
  • byFlag
    • Evaluates to all SemanticsNodes in the tree that has the given SemanticsFlag.
  • byAnyFlag
    • Evaluates to all SemanticsNodes in the tree that has at least one of the given SemanticsFlags.