[web] VideoElement does not open in fullscreen when using CanvasKit

Steps to Reproduce:

1). Run the example below

code sample

import 'dart:ui' as ui;
import 'dart:html';
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      debugShowCheckedModeBanner: false,
      title: 'Example',
      home: DefaultTabController(length: 3, child: Example()),
    );
  }
}


class Example extends StatefulWidget{
  @override
  State<StatefulWidget> createState() {
    return ExampleState();
  }

}

class ExampleState extends State<Example>{

  int _index = 0;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text("Example"),
        centerTitle: true,
        bottom: TabBar(
          onTap: (value){
            setState(() {
              _index = value;
            });
          },
          tabs: [
            Tab(text: "1",),
            Tab(text: "2"),
            Tab(text: "3"),
          ],
        ),
      ),
      body: IndexedStack(
        index: _index,
        children: [
          Center(child: Text("1")),
          Container(
                        height: MediaQuery.of(context).size.height/1.5,
                        width: MediaQuery.of(context).size.width,
                        child: AspectRatio(
                          aspectRatio: 3,
                          child: WebVideoPlayer(
                            autoPlay: true,
                            src: "https://file-examples-com.github.io/uploads/2017/04/file_example_MP4_480_1_5MG.mp4",
                          ),
                        ),
                      ),
          // Text("2"),
          Center(child: Text("3")),
        ],
      ),
    );
  }

}

class WebVideoElement extends StatefulWidget {
  const WebVideoElement({Key key, this.src, this.width, this.height, this.startAt,this.autoplay,this.controls})
      : super(key: key);

  final int width;
  final int height;
  final String src;
  final double startAt;
  final bool autoplay;
  final bool controls;

  @override
  _WebVideoElementState createState() => _WebVideoElementState();
}

class _WebVideoElementState extends State<WebVideoElement> {
  @override
  void initState() {
    super.initState();
    ui.platformViewRegistry.registerViewFactory(
        widget.src,
        (int viewId) {
          final video = VideoElement()
            ..width = widget.width
            ..height = widget.height
            ..src = widget.src + '#t=${widget.startAt}'
            ..autoplay = widget.autoplay
            ..controls = widget.controls
            ..style.border = 'none';
          return video ;
        });
  }

  @override
  Widget build(BuildContext context) {
    return Directionality(
      textDirection: TextDirection.ltr,
      child: HtmlElementView(viewType: widget.src),
    );
  }
}

2). Run the app with flutter run -d chrome --release --web-renderer canvaskit
3). Run the app with flutter run -d chrome --release --web-renderer HTML

Fullscreen works with the HTML renderer but not with the canvaskit renderer.
I don't believe that this has anything to do with TabView, but I left it in the example just incase it provides additional insight.

Video for Reference:

Desktop.2021.02.01.-.21.41.59.08.DVR_Trim.mp4

I can reproduce this with CanvasKit. Works as expected using HTML renderer.

code sample

import 'dart:ui' as ui;
import 'dart:html';
import 'package:flutter/material.dart';

void main() {
  runApp(MyApp());
}

class MyApp extends StatelessWidget {
  // This widget is the root of your application.
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      debugShowCheckedModeBanner: false,
      title: 'Example',
      home: DefaultTabController(length: 3, child: Example()),
    );
  }
}


class Example extends StatefulWidget{
  @override
  State<StatefulWidget> createState() {
    return ExampleState();
  }

}

class ExampleState extends State<Example>{

  int _index = 0;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text("Example"),
        centerTitle: true,
        bottom: TabBar(
          onTap: (value){
            setState(() {
              _index = value;
            });
          },
          tabs: [
            Tab(text: "1",),
            Tab(text: "2"),
            Tab(text: "3"),
          ],
        ),
      ),
      body: IndexedStack(
        index: _index,
        children: [
          Center(child: Text("1")),
          Container(
            height: MediaQuery.of(context).size.height/1.5,
            width: MediaQuery.of(context).size.width,
            child: AspectRatio(
              aspectRatio: 3,
              child: WebVideoElement(
                autoplay: true,
                controls: true,
                src: "https://file-examples-com.github.io/uploads/2017/04/file_example_MP4_480_1_5MG.mp4",
              ),
            ),
          ),
          // Text("2"),
          Center(child: Text("3")),
        ],
      ),
    );
  }

}

class WebVideoElement extends StatefulWidget {
  const WebVideoElement({Key key, this.src, this.width, this.height, this.startAt,this.autoplay,this.controls})
      : super(key: key);

  final int width;
  final int height;
  final String src;
  final double startAt;
  final bool autoplay;
  final bool controls;

  @override
  _WebVideoElementState createState() => _WebVideoElementState();
}

class _WebVideoElementState extends State<WebVideoElement> {
  @override
  void initState() {
    super.initState();
    ui.platformViewRegistry.registerViewFactory(
        widget.src,
            (int viewId) {
          final video = VideoElement()
            ..width = widget.width
            ..height = widget.height
            ..src = widget.src + '#t=${widget.startAt}'
            ..autoplay = widget.autoplay
            ..controls = widget.controls
            ..style.border = 'none';
          return video ;
        });
  }

  @override
  Widget build(BuildContext context) {
    return Directionality(
      textDirection: TextDirection.ltr,
      child: HtmlElementView(viewType: widget.src),
    );
  }
}

flutter doctor -v

[✓] Flutter (Channel master, 1.26.0-18.0.pre.123, on macOS 11.1 20C69 darwin-x64, locale en)
    • Flutter version 1.26.0-18.0.pre.123 at /Users/pedromassango/Code/flutter_master
    • Framework revision 9fd3c22eb9 (20 minutes ago), 2021-02-02 01:06:04 -0500
    • Engine revision bb2bbde4d0
    • Dart version 2.12.0 (build 2.12.0-279.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/pedromassango/Library/Android/sdk
    • Platform android-30, build-tools 30.0.2
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.1, Build version 12A7403
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] IntelliJ IDEA Community Edition (version 2020.3.2)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 52.2.5
    • Dart plugin version 203.6912

[✓] VS Code (version 1.52.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.18.1

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 11.1 20C69 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 88.0.4324.96

! Doctor found issues in 1 category.

@ditman

Something tells me this is caused by loss of state. WDYT @ditman?

@yjbanov

@yjbanov it might be, but this needs some more digging. Do the Tabs have anything to do with the behavior? It's odd, because full-screen mode should not change the layout of the page, but I might be wrong :S

I can reproduce it with an iframe and a youtube embed.

I can reproduce it running on --web-renderer html and flutter 2.0.1

Any workarounds?

@Bilonik

@Bilonik Nope, this issue is still blocking my release flutter web application. Performance with HTML is abysmal so I have disabled the option to open in full screen until this is fixed.

One alternative might be to use chewie or something like that, but I was having problems with chewie earlier so I don't know if that will work.

I'm still trying to debug the canvaskit case. The issue seems different. I'm attempting to use JS to hook up to mutation observers on the DOM. It might be related to HtmlElementViews reloading, but I haven't confirmed yet!

Looking at the screen recordings here, I am actually fairly confident that this is a sub case of #79065 (I did not find the more specific issue before).

Essentially, you can tell from the screen recordings that the Flutter scene is being resized in HTML and exactly in that moment, the video player full screen is discarded. So I assume that this is exactly #79065, where the Flutter pane is resized, which causes the iframe to reload.

Yes, this is definitely happening on resize, I can see the handleMetricsChanged firing when the app goes full screen:

I just didn't get a proper stack trace confirming what it looks like, that resize event to the video player being destroyed and re-created.

A way to test this theory might be to make the video player a fixed size in the top-left corner of the screen, and see if that goes full-screen normally? (I understand this is not the desired layout, but that might be a way to indirectly verify that the resize is the culprit)

@ditman

I tested on the current master (c4ba26e) to check if we can improve on the situation using the approaches proposed by @ditman.

Top left (fixed size) anchor

Here I tried to anchor the VideoElement in the top left corner with a fixed sized (on the Flutter layout side) as you suggested @ditman.
The layout is very simple:

MaterialApp -> Scaffold -> (body) Stack ->
Positioned (top: 0, left: 0) -> SizedBox (width: 480, height: 270) -> HtmlElementView

I also added a centered one and a bottom right positioned one for the screen recordings.

Screen.Recording.2021-03-29.at.20.30.48.mov

As you can see, the Flutter layout does not seem to affect the behavior.
If I swap the HTML element with an iframe to illustrate the #79065 behavior, you can see it more clearly:

Screen.Recording.2021-03-29.at.20.32.31.mov

Also note that if you look at the #79065 fully integrated example, you will actually see a compound effect of the two issues:

Full screen being blocked.
iframe being reloaded (because we are using JW Player and [web] Cannot register HTML Element in the Regular DOM (JW Player) #65157 prevents using a div.

This might be why I jumped to conclusions quickly, but it appears I can reproduce the issues in isolation consistently.

Div wrapping

When wrapping in a div (as suggested on Discord), the only notable change is that the sizing behaves strangely. Namely I need to explicitly set a width and height on the iframe/video element (otherwise it will size itself smaller inside of the div).

You can see that the same behavior still occurs (view inspector):

Screen.Recording.2021-03-29.at.20.41.37.mov

The same applies for VideoElement and trying to use full screen:

Screen.Recording.2021-03-29.at.20.45.11.mov

@ditman

Workaround

After a considerable amount of pain, I was able to successfully build a workaround/hack for this:

720p.mp4

This solves the issue with JW Player in particular - I detailed the problem in #79065.

Approach

The approach is somewhat simple:

I added a new button to the video player that will add a top-level div to the HTML body and create a new instance of the same video ("theater mode").
The same div is also pushed whenever the user double clicks or presses F on their keyboard (I catch the fullscreen event).
I cannot go into fullscreen right away because as you can see, CanvasKit blocks it when you double click or press F (the issue described here).
Once we are in the 100% width 100% height div that is in the HTML body on top of the Flutter glass pane, we can also go into fullscreen (not on Safari, unfortunately).

The whole logic becomes very complex when you consider syncing between the two player instances (the one in the overlay div and the one in Flutter), handling every possible fullscreen entry, etc.
Moreover, there is added complexity because I needed to build hacks like #79550 (comment) for edge cases, where the platform view was suddenly not correctly displayed or the full screen would also be canceled from the "theater mode" on Safari, etc.

One potential improvement I realized was that we could trigger fullscreen from code when the user clicks the custom button. The problem I see is that it would not be consistent with the double click / pressing F and this "theater mode" is a smoother transition for that sake.
Overall, I would say this is a valid workaround that we can live it ~ still, it would be extremely beneficial if this worked 🙏

Considerations

There are some more things I found out need to be kept in mind - they might also explain why my hack is the way it is:

Browsers block fullscreen from code if there was no user interaction at that moment. It means that you can go fullscreen on click and on key press but not arbitrarily from code. This is why we cannot launch the overlayed div into fullscreen when CanvasKit "closes" the "failed fullscreen attempt" from the player inside of Flutter. We require another click from the user then.
Trying to launch the overlayed video player into fullscreen in parallel also does not work because the "failed fullscreen attempt" from CanvasKit will also close the fullscreen of the overlayed div.
Pausing the video instance in Flutter on launching the overlay is not enough when the user double clicks. You need to make sure that you listen to all mutations and events in a way that always syncs the state and makes sure the instance in Flutter is paused while the overlayed one is playing etc.

cc @ditman

@ditman

@ditman its stop working on latest master (web-renderer html)

:(

@Bilonik

@Bilonik AFAIK we haven't changed much in master recently for this to either work or not work, so I'm not sure. HOWEVER, we have a POC of the slot rendering technique now, so we can test this. Let me try and build the example app here with my experimental engine.

I have deployed the repro app with my experimental engine here, and fullscreen seems to work normally again?

https://dit-fullscreen-video.web.app

If so, let's close this in favor of #80524, which is where all the code regarding the new SLOT-based rendering is happening!

@ditman

@ditman Experimental engine looks great! The SLOT-based rendering looks very promising!

@ditman

I also confirm that @ditman's approach is working as expected.

Closing this as fixed, please follow up on #80524 for more updates.

@ditman

@ditman I upgraded Flutter on master to receive the PR. The example still doesn't open in fullscreen. Is there a flag I need to pass through when running the app?

Edit; The engine hasn't finished rolling out yet. My bad! #83066

Doctor summary (to see all details, run flutter doctor -v):
[√] Flutter (Channel master, 2.3.0-13.0.pre.83, on Microsoft Windows [Version 10.0.19042.928], locale en-US)
[√] Android toolchain - develop for Android devices (Android SDK version 30.0.1)
[√] Chrome - develop for the web
[√] Visual Studio - develop for Windows (Visual Studio Community 2019 16.7.4)
[√] Android Studio (version 4.0)
[√] IntelliJ IDEA Community Edition (version 2020.2)
[√] IntelliJ IDEA Ultimate Edition (version 2021.1)
[√] VS Code (version 1.56.0)
[√] Connected device (4 available)

This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

Navigation Menu

Repository navigation

[web] VideoElement does not open in fullscreen when using CanvasKit #75195

10 remaining items

Top left (fixed size) anchor

Div wrapping

Workaround

Approach

Considerations

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

[web] VideoElement does not open in fullscreen when using CanvasKit #75195

Description

Activity

pedromassangocode commented on Feb 2, 2021

yjbanov commented on Feb 4, 2021

ditman commented on Feb 4, 2021

CodeDoctorDE commented on Feb 8, 2021

Bilonik commented on Mar 9, 2021

Bilonik commented on Mar 18, 2021

msoham123 commented on Mar 18, 2021

10 remaining items

ditman commented on Mar 24, 2021

creativecreatorormaybenot commented on Mar 29, 2021

ditman commented on Mar 29, 2021

creativecreatorormaybenot commented on Mar 29, 2021

Top left (fixed size) anchor

Div wrapping

creativecreatorormaybenot commented on Apr 1, 2021

Workaround

Approach

Considerations

Bilonik commented on Apr 16, 2021

ditman commented on Apr 19, 2021

ditman commented on Apr 19, 2021

msoham123 commented on Apr 20, 2021

pedromassangocode commented on Apr 20, 2021

msoham123 commented on May 21, 2021

github-actions commented on Aug 1, 2021

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions