Skip to content

Conversation

@camsim99
Copy link
Contributor

@camsim99 camsim99 commented Sep 16, 2025

When NV21 image format is requested for streaming images, this PR ensures that:

  1. The NV21-compatible (but actually YUV_420_888) three planes per image are converted to a single NV21 plane (Y + interleaved VU planes).
  2. The single CameraImagePlane created has overridden raw and camera image format NV21.*

This should make this package compatible with google_ml_kit_flutter 🤞 I tested this change with the Barcode scanner example. Fixes flutter/flutter#174923.

*The conversion will fail if the image is actually not NV21 compatible for some reason, so this should never be a false positive.

Note: Uses code inspired by googlesamples/mlkit.

Pre-Review Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

Footnotes

  1. Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling. 2 3

Copy link
Contributor Author

@camsim99 camsim99 Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is directly inspired by this camera_android code that came from Google MLKit. I made some changes the methods to (1) take in the objects I want and (2) for planesToNV21, I worked off of yuv420ThreePlanesToNV21 but deleted lots of logic and added an exception. So, I'm not sure what to do here....

Should I also link to https://github.com/googlesamples/mlkit/blob/master/android/vision-quickstart/app/src/main/java/com/google/mlkit/vision/demo/BitmapUtils.java and call it a day or are there more license implications?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stuartmorgan-g Do you know what we should do in cases like this by chance?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I also link to https://github.com/googlesamples/mlkit/blob/master/android/vision-quickstart/app/src/main/java/com/google/mlkit/vision/demo/BitmapUtils.java and call it a day or are there more license implications?

Third-party code cannot be handled that way; the camera_android code shouldn't have landed like that (unless it had OSPO approval), so it should definitely not be replicated, and the old code should be fixed.

The standard way to handle this would be:

  • Land it in a third_party directory, with the correct license.
  • Include that license in the package LICENSE file.

However, because it is specifically Google that has the copyright on that code, it may be that there are simpler options. I would reach out to OSPO and ask how this should be handled.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the guidance! Reached out to OSPO and they let me know that if our code is derived from code authored by Googlers or contributed to a Google-managed open source project under the terms of Google's contributor license agreement, then I can treat the code as though we wrote it ourselves.

Because the BitmapUtils.java code comes from mlkit/android/vision-quickstart that is licensed under the aforementioned agreement, I think I am okay to use it without reference to the original code. I will go ahead and make edits to accommodate this fact and also make a follow-up PR to remove the reference from camera_android.

Copy link
Contributor Author

@camsim99 camsim99 Sep 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just FYI: I did send an email back with my intentions just to sanity check my decision here!

Edit: They confirmed this is the right move 👍 They also explicitly stated not to cite where the code comes from in the file.

@camsim99 camsim99 marked this pull request as ready for review September 17, 2025 21:06
@camsim99 camsim99 requested review from a team and jesswrd September 17, 2025 21:07
@camsim99 camsim99 added the triage-android Should be looked at in Android triage label Sep 22, 2025
auto-submit bot pushed a commit that referenced this pull request Sep 24, 2025
…t` code (#10056)

Remove references to [mlkit/android/vision-quickstart](https://github.com/googlesamples/mlkit/tree/master/android/vision-quickstart) as per my findings in #10022 (comment).

## Pre-Review Checklist

**Note**: The Flutter team is currently trialing the use of [Gemini Code Assist for GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code). Comments from the `gemini-code-assist` bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

[^1]: Regular contributors who have demonstrated familiarity with the repository guidelines only need to comment if the PR is not auto-exempted by repo tooling.
@stuartmorgan-g
Copy link
Collaborator

Updating brach to pick up the license format check change, since this adds new files. The new files will need to be updated to match the new header boilerplate.

@camsim99
Copy link
Contributor Author

@jesswrd would you mind taking a look at this PR this week? Thanks in advance!

@camsim99 camsim99 added the autosubmit Merge PR when tree becomes green via auto submit App label Sep 29, 2025
@auto-submit auto-submit bot merged commit f419532 into flutter:main Sep 29, 2025
80 checks passed
engine-flutter-autoroll added a commit to engine-flutter-autoroll/flutter that referenced this pull request Sep 30, 2025
github-merge-queue bot pushed a commit to flutter/flutter that referenced this pull request Sep 30, 2025
flutter/packages@34eec78...287739d

2025-09-29 [email protected] Roll Flutter (stable) from
d693b4b to ac4e799 (4 revisions) (flutter/packages#10124)
2025-09-29 [email protected] Fix CODEOWNERS error
(flutter/packages#10127)
2025-09-29 [email protected] [file_selector] Update Pigeon in
Android implementation (flutter/packages#10126)
2025-09-29 [email protected]
[In_app_purchase_storekit] Do not throw PigeonError when a transaction
is pending / cancelled / unverified (flutter/packages#9627)
2025-09-29 [email protected] [video_player] Separate texture ID on
Android (flutter/packages#10029)
2025-09-29 [email protected] Roll Flutter from
6cc976e to 96fe3b3 (32 revisions) (flutter/packages#10125)
2025-09-29 [email protected]
[camera_android_camerax] Fix NV21 Format (flutter/packages#10022)
2025-09-29 [email protected] Manual roll Flutter from
b1a28bc to 6cc976e (4 revisions) (flutter/packages#10116)
2025-09-29 [email protected] [camera_avfoundation]
Implementation swift migration - part 13 (flutter/packages#9930)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC [email protected] on the revert to ensure that a
human
is aware of the problem.

To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
okorohelijah pushed a commit to okorohelijah/flutter that referenced this pull request Oct 7, 2025
flutter/packages@34eec78...287739d

2025-09-29 [email protected] Roll Flutter (stable) from
d693b4b to ac4e799 (4 revisions) (flutter/packages#10124)
2025-09-29 [email protected] Fix CODEOWNERS error
(flutter/packages#10127)
2025-09-29 [email protected] [file_selector] Update Pigeon in
Android implementation (flutter/packages#10126)
2025-09-29 [email protected]
[In_app_purchase_storekit] Do not throw PigeonError when a transaction
is pending / cancelled / unverified (flutter/packages#9627)
2025-09-29 [email protected] [video_player] Separate texture ID on
Android (flutter/packages#10029)
2025-09-29 [email protected] Roll Flutter from
6cc976e to 96fe3b3 (32 revisions) (flutter/packages#10125)
2025-09-29 [email protected]
[camera_android_camerax] Fix NV21 Format (flutter/packages#10022)
2025-09-29 [email protected] Manual roll Flutter from
b1a28bc to 6cc976e (4 revisions) (flutter/packages#10116)
2025-09-29 [email protected] [camera_avfoundation]
Implementation swift migration - part 13 (flutter/packages#9930)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC [email protected] on the revert to ensure that a
human
is aware of the problem.

To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
reidbaker pushed a commit to AbdeMohlbi/flutter that referenced this pull request Dec 10, 2025
flutter/packages@34eec78...287739d

2025-09-29 [email protected] Roll Flutter (stable) from
d693b4b to ac4e799 (4 revisions) (flutter/packages#10124)
2025-09-29 [email protected] Fix CODEOWNERS error
(flutter/packages#10127)
2025-09-29 [email protected] [file_selector] Update Pigeon in
Android implementation (flutter/packages#10126)
2025-09-29 [email protected]
[In_app_purchase_storekit] Do not throw PigeonError when a transaction
is pending / cancelled / unverified (flutter/packages#9627)
2025-09-29 [email protected] [video_player] Separate texture ID on
Android (flutter/packages#10029)
2025-09-29 [email protected] Roll Flutter from
6cc976e to 96fe3b3 (32 revisions) (flutter/packages#10125)
2025-09-29 [email protected]
[camera_android_camerax] Fix NV21 Format (flutter/packages#10022)
2025-09-29 [email protected] Manual roll Flutter from
b1a28bc to 6cc976e (4 revisions) (flutter/packages#10116)
2025-09-29 [email protected] [camera_avfoundation]
Implementation swift migration - part 13 (flutter/packages#9930)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages-flutter-autoroll
Please CC [email protected] on the revert to ensure that a
human
is aware of the problem.

To file a bug in Flutter:
https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

autosubmit Merge PR when tree becomes green via auto submit App p: camera platform-android triage-android Should be looked at in Android triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[camera_android_camerax] Wrong ImageFormatGroup for nv21 causes ML Kit incompatibility

3 participants