[video_player] Edited videos are distorted when played on iOS #71299
Description
When I run the sample video_player plugin app with a particular mp4 file, the playback is incorrect. The Video doesn't appear to fit on the screen properly. This only effects iOS, everything is perfect on Android. The offending mp4 file has rotation metadata in the file and I suspect that the native iOS code relating to the video player is not handling the rotation metadata correctly.
Steps to Reproduce
Run the video_player sample project and replace the mp4 url with the following url: https://squiredev.com/golf.mp4
Expected results:
I expected the video to play normally like it would any other mp4 file but the video is not fitting onto the screen correctly
Actual results:
The video plays however it is not fitting onto the screen properly. I noticed that the FFMPEG output for the file has a rotation value in the meta data, maybe this is being ignored?
Here is a screenshot of how it looks on iOS
If I re-encode the file using FFMPEG then the video plays perfectly as expected. Here is the command I use to re-encode the file:
ffmpeg -i golf.mp4 golf_reencoded.mp4
Here is a screenshot of how the re-encoded file looks:
FFMPEG outputs:
Here is the FFMPEG output of the original mp4 file:
$ ffmpeg -i golf.mp4
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'golf.mp4':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2020-11-26T07:39:11.000000Z
com.apple.quicktime.location.accuracy.horizontal: 12.631108
com.apple.quicktime.location.ISO6709: +51.9998+000.9766+046.756/
com.apple.quicktime.make: Apple
com.apple.quicktime.model: iPhone XR
com.apple.quicktime.software: 14.1
com.apple.quicktime.creationdate: 2020-11-18T10:46:15+0000
Duration: 00:00:03.80, start: 0.000000, bitrate: 3307 kb/s
Stream #0:0(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 175 kb/s (default)
Metadata:
creation_time : 2020-11-26T07:39:11.000000Z
handler_name : Core Media Audio
Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720, 3120 kb/s, 30.24 fps, 29.97 tbr, 600 tbn, 1200 tbc (default)
Metadata:
rotate : 90
creation_time : 2020-11-26T07:39:11.000000Z
handler_name : Core Media Video
encoder : H.264
Side data:
displaymatrix: rotation of -90.00 degrees
Here is the FFMPEG output of the re-encoded file, the re-encoded file plays perfectly in the video_player sample app. It may be worth noting that during the re-encoding process, FFMPEG drops the meta/rotation data and resizes the video from 1280x720 to 720x1280 . This leads me to think this could be a problem in the iOS native Video Player, relating to the handling of the meta/rotation data. If you would like to test the re-encoded file file yourself then it can be found at the following url: https://squiredev.com/golf_reencoded.mp4
$ ffmpeg -i golf_reencoded.mp4
ffmpeg version 4.2.4-1ubuntu0.1 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 9 (Ubuntu 9.3.0-10ubuntu2)
configuration: --prefix=/usr --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
libavutil 56. 31.100 / 56. 31.100
libavcodec 58. 54.100 / 58. 54.100
libavformat 58. 29.100 / 58. 29.100
libavdevice 58. 8.100 / 58. 8.100
libavfilter 7. 57.100 / 7. 57.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 5.100 / 5. 5.100
libswresample 3. 5.100 / 3. 5.100
libpostproc 55. 5.100 / 55. 5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'golf_reencoded.mp4':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
encoder : Lavf58.29.100
Duration: 00:00:03.84, start: 0.000000, bitrate: 2105 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 720x1280, 1967 kb/s, 29.97 fps, 29.97 tbr, 30k tbn, 59.94 tbc (default)
Metadata:
handler_name : Core Media Video
Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 127 kb/s (default)
Metadata:
handler_name : Core Media Audio
$ flutter doctor -v
[✓] Flutter (Channel dev, 1.24.0-10.2.pre, on Linux, locale en_GB.UTF-8)
• Flutter version 1.24.0-10.2.pre at /home/brx/software/flutter
• Framework revision 022b333 (8 days ago), 2020-11-18 11:35:09 -0800
• Engine revision 07c1eed46b
• Dart version 2.12.0 (build 2.12.0-29.10.beta)
[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
• Android SDK at /home/brx/Android/Sdk
• Platform android-30, build-tools 30.0.2
• Java binary at: /home/brx/android-studio/jre/bin/java
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
• All Android licenses accepted.
[✓] Chrome - develop for the web
• Chrome at google-chrome
[✓] Android Studio (version 4.0)
• Android Studio at /home/brx/android-studio
• Flutter plugin version 49.0.2
• Dart plugin version 193.7547
• Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6222593)
[✓] VS Code (version 1.47.3)
• VS Code at /usr/share/code
• Flutter extension version 3.14.1
[✓] Connected device (2 available)
• Web Server (web) • web-server • web-javascript • Flutter Tools
• Chrome (web) • chrome • web-javascript • Google Chrome 84.0.4147.125
• No issues found!