Skip to content

DragScrollActivity disables tap actions in the semantics tree #130162

@yjbanov

Description

@yjbanov

Is there an existing issue for this?

Steps to reproduce

  1. This will help a ton: edit rendering/binding.dart and add debugDumpSemanticsTree(); as the first line in _handleSemanticsUpdate.
  2. Run flutter run -d web-server --web-port=8888 on the code sample provided below.
  3. Open Chrome and point it at http://localhost:8888.
  4. Open Chrome DevTools and enable the mobile emulation mode (this causes Chrome to emulate taps, like on Android).
  5. Long-press on the text next to the checkbox. Do not press the checkbox itself (this is key!)

Expected results

All printed semantics trees should contain a tap action on the node corresponding to the combination of the checkbox and the text next to it, like this (SemanticsNode#5 is the interesting one; others were stripped of information for readability):

SemanticsNode#0
 └─SemanticsNode#1
   └─SemanticsNode#2
     └─SemanticsNode#3
       └─SemanticsNode#4
         └─SemanticsNode#6
           └─SemanticsNode#5
               Rect.fromLTRB(0.0, 0.0, 390.0, 32.0)
               tags: RenderViewport.twoPane
               actions: tap
               flags: hasCheckedState, isFocused, hasEnabledState, isEnabled,
                 isFocusable
               label: "this is a visible label"
               textDirection: ltr

Actual results

Upon receiving a pointerdown event, if the pointerup doesn't come soon enough ScrollableState initiates a drag activity calling setIgnorePointer to true. Importantly the pointerup event would still produce a click/tap event and win in the gesture arena. The drag activity starts before the tap is detected.

at RenderIgnorePointer.describeSemanticsConfiguration$1
at RenderIgnorePointer.get$_semanticsConfiguration
at RenderIgnorePointer.markNeedsSemanticsUpdate$0
at RenderIgnorePointer.set$ignoring
at ScrollableState.setIgnorePointer$1
at ScrollPositionWithSingleContext.beginActivity$1
at ScrollPositionWithSingleContext.beginActivity$1
at ScrollableState._scrollable$_handleDragStart$1

The reason ScrollPositionWithSingleContext ignores the pointer is because DragScrollActivity.shouldIgnorePointer returns true. If I change the implementation of the getter to return false, the problem goes away.

This causes a semantic update, leading to the following semantics tree being synced into the engine prior to the pointerup event:

SemanticsNode#0
 └─SemanticsNode#1
   └─SemanticsNode#2
     └─SemanticsNode#3
       └─SemanticsNode#4
         └─SemanticsNode#6
           └─SemanticsNode#5
               Rect.fromLTRB(0.0, 0.0, 390.0, 32.0)
               tags: RenderViewport.twoPane
               flags: hasCheckedState, isFocused, hasEnabledState, isEnabled,
                 isFocusable
               label: "this is a visible label"
               textDirection: ltr

Notice that actions: tap is gone. This tells the engine that the node is no longer tappable and causes the engine to stop sending tap events to it.

Shortly after, another semantics update reinstates the tap action. However, by then it's too late as the click was ignored.

To summarize, the sequence of events is as follows:

  • Initial semantics update adds the tap action to the checkbox
  • pointerdown happens
  • Drag start => ignore pointer injected
  • Semantics update removes the tap action
  • pointerup happens
  • click happens - because the node no longer has the tap action the click is ignored
  • Another semantics update adds the tap action back, but it's too late

Code sample

Code sample
import 'package:flutter/foundation.dart';
import 'package:flutter/material.dart';
import 'package:flutter/rendering.dart';

void main() {
  runApp(const MyApp());
  if (kIsWeb) {
    SemanticsBinding.instance.ensureSemantics();
  }
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Accessibility tester app',
      theme: ThemeData(
        colorScheme: ColorScheme.fromSeed(seedColor: Colors.deepPurple),
        useMaterial3: true,
      ),
      home: const MyHomePage(title: 'Accessibility tester app'),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({super.key, required this.title});

  final String title;

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  @override
  Widget build(BuildContext context) {
    return const Scaffold(
      body: CheckboxTest(),
    );
  }
}

class CheckboxTest extends StatefulWidget {
  const CheckboxTest({ super.key });

  @override
  State<CheckboxTest> createState() => _CheckboxTestState();
}

class _CheckboxTestState extends State<CheckboxTest> {
  bool _value1 = false;

  @override
  Widget build(BuildContext context) {
    // ListView is necessary for the scrolling infra to kick in.
    return ListView(
      children: <Widget>[
        Row(
          children: [
            Checkbox(
              autofocus: true,
              value: _value1,
              onChanged: (bool? value) {
                setState(() {
                  _value1 = value!;
                });
              },
            ),
            const Text('this is a visible label'),
          ],
        ),
      ],
    );
  }
}

Screenshots or Video

Screenshots / Video demonstration
checkbox-a11y-issue.mov

Logs

No response

Flutter Doctor output

Doctor output
Doctor summary (to see all details, run flutter doctor -v):
[!] Flutter (Channel [user-branch], 3.12.0-4.0.pre.316, on macOS 13.4.1 22F82 darwin-arm64, locale en)
    ! Flutter version 3.12.0-4.0.pre.316 on channel [user-branch] at /Users/yjbanov/code/flutter/flutter
      Currently on an unknown channel. Run `flutter channel` to switch to an official channel.
      If that doesn't fix the issue, reinstall Flutter by following instructions at
      https://flutter.dev/docs/get-started/install.
    ! Upstream repository unknown source is not a standard remote.
      Set environment variable "FLUTTER_GIT_URL" to unknown source to dismiss this error.
[!] Android toolchain - develop for Android devices (Android SDK version 33.0.2)
    ✗ cmdline-tools component is missing
      Run `path/to/sdkmanager --install "cmdline-tools;latest"`
      See https://developer.android.com/studio/command-line for more details.
    ✗ Android license status unknown.
      Run `flutter doctor --android-licenses` to accept the SDK licenses.
      See https://flutter.dev/docs/get-started/install/macos#android-setup for more details.
[✓] Xcode - develop for iOS and macOS (Xcode 14.2)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2022.2)
[✓] VS Code (version 1.79.2)
[✓] Connected device (2 available)
[✓] Network resources

! Doctor found issues in 2 categories.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work lista: accessibilityAccessibility, e.g. VoiceOver or TalkBack. (aka a11y)f: gesturesflutter/packages/flutter/gestures repository.f: scrollingViewports, list views, slivers, etc.frameworkflutter/packages/flutter repository. See also f: labels.platform-webWeb applications specificallyteam-frameworkOwned by Framework teamtriaged-frameworkTriaged by Framework team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions