Skip to content

[Android] - Jitter/jank problem when panning content #129150

@matthew-carroll

Description

@matthew-carroll

When panning and zooming content on Android tablets, we're experiencing a repeated "jitter" effect, which has rendered an important app unshippable.

The Problem

We have a tile-based PDF renderer, which renders textures for every tile. Our PDF tiles are pannable and zoomable.

We noticed that as the user panned around the PDF, there was a slight but noticeable jitter that would appear almost all the time. I call it a "jitter" instead of "jank" because typically jank is characterized by a single frame that's very slow, resulting in a single, noticeable jump in the UI. The behavior we're seeing is more like a repeated start-and-stop behavior. It's more like Flutter is rendering every other frame, or something like that.

Here's a video taken directly from an Android tablet, showing the jitter. This reproduction isn't using our PDF system. It's based on the standard Flutter widgets that I included below in the "Reproduction" section.

https://youtu.be/cwhRwdM_2es

Despite the visual indication that something is going wrong with the frame rate, the Flutter profiler shows very few red jank frames.
Screenshot 2023-06-19 at 8 10 24 PM

Why it Matters

This jitter might seem minor as compared to some other Flutter jank issues. However, this is a ship-blocking issue for my client.

My client is trying to release a premium PDF reading experience, built with Flutter. Every member of the team, and every early tester of this app, notices this problem, and complains about it. It feels cheap and buggy. This can't be what my client ships.

If this problem can't be solved, my client will likely have to throw out all of their work on this reading experience, and rebuild it without Flutter.

I've elaborated on these business implications with @tvolkert

Hypotheses We've Evaluated

We've worked quite a bit with @dnfield to try to root cause and solve this jitter. The primary issue with this jitter is that none of our performance signals indicate a problem. The Flutter profiler shows very few jank frames. The Android profiler doesn't show anything that my team is able to identify as an issue. The device reports a high frame rate. We have no mechanism through which to drill down, locate, and resolve the issue. Instead, we're forced to throw things at at the wall and hope something sticks.

Here are some of things we've thrown at the wall.

Maybe its a texture-specific problem: We reworked our tile system to use typical bitmap images instead of texture layers. The problem persists.

Maybe the images are too big: We tried decoding our bitmap images at smaller resolutions. We then tried shrinking the images on disk to make them only 400px wide. The problem persists.

Maybe the issues is that you're using TextureView on Android instead of SurfaceView: We switched back to SurfaceView and the problem persists.

Maybe the issue is the touch input system, rather than the rendering system: We created panning animations to completely bypass the touch input system. The problem persists.

Maybe it's a manufacturing issue with that one Redmi tablet: We've reproduced the issue on multiple Redmi tablets.

Maybe it's an issue with Redmi tablets: We've reproduced the issue (though less of an issue) on the Samsung Galaxy Tab S6 Lite. We've also reproduced it on the client's custom tablet.

Maybe it's an issue with the tile layout implementation: We threw away all of our code and reproduced the jitter with regular Flutter widgets, as described in the "Reproduction" section.

Maybe things would work better on Impeller: At a couple points in this process we tried Impeller. The problem persists.

Reproduction

The following Flutter code reproduces a jitter effect when running animated panning around small images displayed in an InteractiveViewer. A video of the reproduction is shown in "The Problem" section.

For a direct comparison, this code should be run on a Xiaomi Redmi tablet. That said, we've experienced the same problem, to a lesser degree, on a Samsung Galaxy Tab S6 Lite.

Code Sample
import 'package:flutter/material.dart';

void main() {
  runApp(
    const MaterialApp(
      home: _InteractiveViewerDemo(),
    ),
  );
}

class _InteractiveViewerDemo extends StatefulWidget {
  const _InteractiveViewerDemo({super.key});

  @override
  State<_InteractiveViewerDemo> createState() => _InteractiveViewerDemoState();
}

class _InteractiveViewerDemoState extends State<_InteractiveViewerDemo> with TickerProviderStateMixin {
  late final TransformationController _controller;

  int _currentImageIndex = 0;

  @override
  void initState() {
    super.initState();
    _controller = TransformationController();
  }

  final _verticalAnimationDistance = 800;
  AnimationController? _verticalPanningAnimation;
  double? _previousFrameVerticalOffset;
  void _toggleVerticalPanningAnimation() {
    if (_verticalPanningAnimation == null) {
      // Start the animation
      _previousFrameVerticalOffset = 0;
      _verticalPanningAnimation = AnimationController(
        vsync: this,
        duration: const Duration(seconds: 2),
      )
        ..addListener(() {
          final offsetAtTime =
              _verticalAnimationDistance * Curves.easeInOut.transform(_verticalPanningAnimation!.value);
          _controller.value.translate(0.0, _previousFrameVerticalOffset! - offsetAtTime);
          _controller.notifyListeners();
          _previousFrameVerticalOffset = offsetAtTime;
        })
        ..addStatusListener((status) {
          switch (status) {
            case AnimationStatus.dismissed:
              _verticalPanningAnimation!.forward();
              break;
            case AnimationStatus.completed:
              _verticalPanningAnimation!.reverse();
              break;
            case AnimationStatus.forward:
            case AnimationStatus.reverse:
              // TODO: Handle this case.
              break;
          }
        })
        ..forward();
    } else {
      // Stop the animation
      _verticalPanningAnimation!.dispose();
      _verticalPanningAnimation = null;
    }
  }

  final _horizontalAnimationDistance = 400;
  AnimationController? _horizontalPanningAnimation;
  double? _previousFrameHorizontalOffset;
  void _toggleHorizontalPanningAnimation() {
    if (_horizontalPanningAnimation == null) {
      // Start the animation
      _previousFrameHorizontalOffset = 0;
      _horizontalPanningAnimation = AnimationController(
        vsync: this,
        duration: const Duration(seconds: 2),
      )
        ..addListener(() {
          final offsetAtTime =
              _horizontalAnimationDistance * Curves.easeInOut.transform(_horizontalPanningAnimation!.value);
          _controller.value.translate(_previousFrameHorizontalOffset! - offsetAtTime, 0);
          _controller.notifyListeners();
          _previousFrameHorizontalOffset = offsetAtTime;
        })
        ..addStatusListener((status) {
          switch (status) {
            case AnimationStatus.dismissed:
              _horizontalPanningAnimation!.forward();
              break;
            case AnimationStatus.completed:
              _horizontalPanningAnimation!.reverse();
              break;
            case AnimationStatus.forward:
            case AnimationStatus.reverse:
              // TODO: Handle this case.
              break;
          }
        })
        ..forward();
    } else {
      // Stop the animation
      _horizontalPanningAnimation!.dispose();
      _horizontalPanningAnimation = null;
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: InteractiveViewer(
        transformationController: _controller,
        constrained: false,
        alignment: Alignment.center,
        child: _buildImageGrid(),
      ),
      floatingActionButton: _buildAnimationButtons(),
    );
  }

  Widget _buildImageGrid() {
    _currentImageIndex = 0;

    return Column(
      children: [
        _buildImageRow(),
        _buildImageRow(),
        _buildImageRow(),
        _buildImageRow(),
      ],
    );
  }

  Widget _buildImageRow() {
    return Row(
      children: [
        _buildImage(),
        _buildImage(),
        _buildImage(),
        _buildImage(),
      ],
    );
  }

  Widget _buildImage() {
    final image = SizedBox(
      width: 400,
      height: 500,
      child: Image.asset(
        _photos[_currentImageIndex],
        fit: BoxFit.fill,
      ),
    );

    _currentImageIndex += 1;
    _currentImageIndex %= _photos.length;

    return image;
  }

  Widget _buildAnimationButtons() {
    return Column(
      mainAxisSize: MainAxisSize.min,
      children: [
        FloatingActionButton(
          onPressed: _toggleHorizontalPanningAnimation,
          child: const Icon(Icons.compare_arrows),
        ),
        const SizedBox(height: 12),
        FloatingActionButton(
          onPressed: _toggleVerticalPanningAnimation,
          child: const Icon(Icons.arrow_downward_sharp),
        ),
      ],
    );
  }
}

const _photos = [
  "assets/image-4_small.jpeg",
  "assets/image-5_small.jpeg",
  "assets/image-8_small.jpeg",
  "assets/image-9_small.jpeg",
  "assets/image-12_small.jpeg",
  "assets/image-13_small.jpeg",
];

Flutter version:

Flutter 3.10.5 • channel stable • https://github.com/flutter/flutter.git
Framework • revision 796c8ef792 (6 days ago) • 2023-06-13 15:51:02 -0700
Engine • revision 45f6e00911
Tools • Dart 3.0.5 • DevTools 2.23.1

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work listc: performanceRelates to speed or footprint issues (see "perf:" labels)engineflutter/engine related. See also e: labels.found in release: 3.10Found to occur in 3.10found in release: 3.12Found to occur in 3.12has reproducible stepsThe issue has been confirmed reproducible and is ready to work onplatform-androidAndroid applications specificallyteam-engineOwned by Engine teamtriaged-engineTriaged by Engine team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions