Skip to content

[camera] Low Resolution in Camera Image Stream, ResolutionPreset Constructor Parameter is ignored #78247

@Joshua27

Description

@Joshua27

Hi,
I would like to process images from the camera while previewing the camera images using the camera plugin. I do not want to transform the preview image or draw anything on the previewed image.
From my understanding the function CameraController#startImageStream() is supposed to be used for such tasks.
This works as expected except of the point that the images send to the image stream only have 720p resolution or lower regardless what ResolutionPreset is set when constructing the CameraController

Steps to Reproduce

Goal: Process images from the camera in best available resolution (send to isolates to not block the main isolate) while previewing the camera images.

Most important parts of the implementation:

Create controller:

_controller = CameraController(
  cameraDescription,
  ResolutionPreset.max,
  enableAudio: false,
);

Show camera preview using widget:

CameraPreview(_controller)

Listen to image stream:

_controller.startImageStream((image) {
        print("Image size: ${image.width}x${image.height}");
      });

The CameraDescription is provided by onNewCameraSelected() as demonstrated in the example provided along with this camera plugin.
To reproduce this issue you can run the example provided in this package and listen to the image stream while previewing the camera images.

Expected results: Maximum available image resolution in the image stream. My devices should have 4k resolution (or at least Full-HD).

Actual results: Always 720p (or lower if requested).

Tested Devices: Samsung Galaxy S10 and S10e

The constructor parameter of CameraController defining ResolutionPreset is ignored/overwritten if it is greater than ResolutionPreset.high.
As already pointed out in issue #49420, the following function in the camera plugin is the reason for this behavior:

static Size computeBestPreviewSize(String cameraName, ResolutionPreset preset) {
    if (preset.ordinal() > ResolutionPreset.high.ordinal()) {
      preset = ResolutionPreset.high;
    }
    CamcorderProfile profile =
        getBestAvailableCamcorderProfileForResolutionPreset(cameraName, preset);
    return new Size(profile.videoFrameWidth, profile.videoFrameHeight);
  }

I see the point that the resolution has to be scaled down for preview.
However, there should be a possibility to process the camera images in the specified resolution.
It should be up to the developer to not block the main thread when listening to an image stream.

My current workaround is to build the camera plugin locally and change CameraUtils#computeBestPreviewSize() as follows:

static Size computeBestPreviewSize(String cameraName, ResolutionPreset preset) {
    //if (preset.ordinal() > ResolutionPreset.high.ordinal()) {
    //  preset = ResolutionPreset.high;
    //}
    CamcorderProfile profile =
        getBestAvailableCamcorderProfileForResolutionPreset(cameraName, preset);
    return new Size(profile.videoFrameWidth, profile.videoFrameHeight);
  }

Of course, this now slows down the camera preview on older devices or is really slow if 4K resolution is used.
Yet, I couldn't find a way to downscale the image before being previewed using above initialization.
I was able to improve performance for previewing a little by changing line 35 of camera_preview.dart from

CameraPlatform.instance.buildPreview(controller.cameraId)

to

Texture(textureId: controller.cameraId, filterQuality: FilterQuality.none)

The default setting of the parameter filterQuality is FilterQuality.low, which is currently used.
According to the documentation of FilterQuality the setting FilterQuality.none is faster.

Summary: Is it currently possible to process the camera images with the requested resolution while previewing the camera images in a lower resolution?

If this is not the case, I would like to request the feature to add another image stream to CameraController which is not used for previewing as described above. The images in the stream used for previewing can be downscaled, that's fine.
Alternatively, one might be able to add an additional constructor parameter and field to CameraController which defines the preview size (use to downscale before actual preview) while the images send to the single image stream have the requested resolution.
Of course, if I select a large preview size the performance is bad. But this decision should be adjustable and up to the developer.

Output of `flutter-doctor -v`
[✓] Flutter (Channel stable, 2.0.2, on Linux, locale de_DE.UTF-8) • Flutter version 2.0.2 at /home/joshua/snap/flutter/common/flutter • Framework revision 8962f6d (vor 4 Tagen), 2021-03-11 13:22:20 -0800 • Engine revision 5d8bf81 • Dart version 2.12.1

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
• Android SDK at /home/joshua/Android/Sdk
• Platform android-30, build-tools 30.0.2
• Java binary at: /usr/lib/jvm/java-8-oracle/bin/java
• Java version Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
! Some Android licenses not accepted. To resolve this, run: flutter doctor --android-licenses

[✗] Chrome - develop for the web (Cannot find Chrome executable at google-chrome)
! Cannot find Chrome. Try setting CHROME_EXECUTABLE to a Chrome executable.

[!] Android Studio (not installed)
• Android Studio not found; download from https://developer.android.com/studio/index.html
(or visit https://flutter.dev/docs/get-started/install/linux#android-setup for detailed instructions).

[✓] IntelliJ IDEA Ultimate Edition (version 2019.1)
• IntelliJ at /home/joshua/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/191.6183.87
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] IntelliJ IDEA Ultimate Edition (version 2020.3)
• IntelliJ at /home/joshua/.local/share/JetBrains/Toolbox/apps/IDEA-U/ch-0/203.7148.57
• Flutter plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/9212-flutter
• Dart plugin can be installed from:
🔨 https://plugins.jetbrains.com/plugin/6351-dart

[✓] VS Code (version 1.52.1)
• VS Code at /usr/share/code
• Flutter extension can be installed from:
🔨 https://marketplace.visualstudio.com/items?itemName=Dart-Code.flutter

[✓] Connected device (2 available)
• SM G970F (mobile) • RF8M20WXQAA • android-arm64 • Android 10 (API 29)
• SM G973F (mobile) • RF8N40DYR5Y • android-arm64 • Android 11 (API 30)

! Doctor found issues in 3 categories.

The issue #58163 seems to be related as well.

Greetings,
Joshua

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Important issues not at the top of the work listfound in release: 3.3Found to occur in 3.3found in release: 3.4Found to occur in 3.4has reproducible stepsThe issue has been confirmed reproducible and is ready to work onp: cameraThe camera pluginpackageflutter/packages repository. See also p: labels.team-ecosystemOwned by Ecosystem teamtriaged-ecosystemTriaged by Ecosystem team

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions