Would it not make more sense for the image orientation to depend on the orientation of the UI, rather than the device's accelerometer?

Would it not make more sense for the image orientation to depend on the orientation of the UI, rather than the device's accelerometer?

No unfortunately, because the UI orientation can be permanently locked (to portrait) while the user is taking a picture with the phone rotated (to landscape). flutter/flutter#18391 was filed separately on Android for this a little while ago, and you can see some comments mentioning the problem happening on Android while the UI is locked earlier in the linked issue for this PR: 1, 2, 3

well done, many thanks.

Hi @mklim . Thanks so much for working to solve this problem; it is a blocker for the launch of our product so it's very much appreciated. Also, sorry for coming in late to this PR. I've been testing it out with camera lib 0.2.9+1, and it seems like there is a problem with the orientation that is being detected with the CMMotionManager. In general, we do not always get the correct orientations for our photos, especially when the device is pitched at an angle > 45% (I'm using the term pitch as defined in https://developer.apple.com/documentation/coremotion/getting_processed_device-motion_data/understanding_reference_frames_and_device_attitude )

I think I've isolated it to two issues in the getImageRotation method. One is an easy fix, which is that I believe pitch is being used where yaw should instead be used. You can notice this if you take a photo in portrait with top of the phone titled downward, e.g. as though taking a picture of something on a table from 45% above the table. When I switch the calculation from using pitch to using yaw, things generally get more stable, although there is actually still a major problem. Below is the change I made to use yaw instead of pitch.

 -   const bool vertical = fabs(pitch) <= M_PI_4;
 +  const double yaw = atan2(2*(orientation.w * orientation.z + orientation.x * orientation.y), 1 - 2 * 
 (orientation.y * orientation.y + orientation.z * orientation.z));
 +  const bool vertical = fabs(yaw) <= M_PI_4;

The second issue seems harder to overcome though, which is that the yaw value returned by the motion manager seems to be relative to the position of the device when the application is started. What this means is that everything works as expected if you start the application while the phone is in portrait mode, but if you start the application while it is in landscape, or at some other odd angle, orientations start coming back wrong.

Incidentally, most of my testing was with iPhoneX, although I also tested some of the potential fixes on an iPad.

I was searching for alternative libraries to use to compute orientation, and came across the same as you: https://developer.apple.com/documentation/uikit/uidevice/1620053-orientation?language=objc. However, on my devices, it does not seem to suffer from the problem you mentioned in your 3rd list item -- of not working when the interface is locked in portrait mode. The documentation also seems to suggest that it should work correctly even when UI is locked: "This value represents the physical orientation of the device and may be different from the current orientation of your application’s user interface."

Here is a definition of getImageRotation that seems to work for my device, even when the UI is locked in portrait (although the begin and end notification calls may belong outside of this method ultimately):

- (UIImageOrientation)getImageRotation {
    [[UIDevice currentDevice] beginGeneratingDeviceOrientationNotifications];
    const UIDeviceOrientation currentDeviceOrientation = [[UIDevice currentDevice] orientation];
    [[UIDevice currentDevice] endGeneratingDeviceOrientationNotifications];
    if (currentDeviceOrientation == UIDeviceOrientationLandscapeLeft) {
        return UIImageOrientationUp;
    }
    else if (currentDeviceOrientation == UIDeviceOrientationLandscapeRight) {
        return UIImageOrientationDown;
    }
    else if (currentDeviceOrientation == UIDeviceOrientationPortraitUpsideDown){
        return UIImageOrientationLeft;
    }
    else if (currentDeviceOrientation == UIDeviceOrientationPortrait) {
        return UIImageOrientationRight;
    }
    else {
        // Otherwise we just don't know (face down, face up, or unknown)
        // TODO: Maybe use the UIInterfaceOrientation when in these scenarios
        return UIImageOrientationUp;
    }
}

Let me know what you think.

Thanks!

Thanks for the detailed look and debugging info @idealopamp! I'm still digging into this in more detail, but in general I think there's possibly device or OS specific inconsistencies going on here. Opened flutter/flutter#26786 to track this.