Skip to content
This repository was archived by the owner on Dec 3, 2025. It is now read-only.

Conversation

@GeertJohan
Copy link
Contributor

@GeertJohan GeertJohan commented Dec 28, 2018

Related to #32

This PR adds pixel_ratio calculations to the linux/GLFW embedder.

The pixel_ratio seems to be based on 160 dpi. This is the same conversion rate for the dp in Android. The Flutter documentation mentions the Nexus 6 to have a pixel_ratio of 3.5, which matches with it's pixels/dp scale-factor as well.

GLFW makes use of screen coordinates, which are a virtual unit and do not always map 1:1 to physical pixels on a device. I am not 100% sure if the conversion to actual pixels is needed, or if Flutter should just know about screen coordinates?

@GeertJohan GeertJohan changed the title WIP: Add pixel_ratio to Linux embedder WIP: [linux] Add pixel_ratio to the embedder Dec 28, 2018
@GeertJohan GeertJohan changed the title WIP: [linux] Add pixel_ratio to the embedder [linux] Add pixel_ratio to the embedder Dec 28, 2018
@stuartmorgan-g
Copy link
Collaborator

I don't think we need the DPI calculations here; on macOS we just pass the pixel scale factor (e.g., 2 for Retina displays), so pixelsPerScreenCoordinate is likely the relevant value in your code. However, I'm not sure what, if anything, you need to do in GLFW to actually cause the GLFW window to use a high-resolution buffer though, or if that will be done automatically. Are you seeing a value other than 1 there without any other changes?

One you provide a non-1 value for pixel_ratio you'll also need to adjust any coordinates sent to Flutter, or click handling will break.

@GeertJohan
Copy link
Contributor Author

GeertJohan commented Dec 28, 2018

This was my initial thought as well. But if I used only the pixelsPerScreenCoordinate, the application ofcourse still renders tiny (text barely readable). On my display, 1 pixel = 1 screen coordinate, so nothing changes. I believe that 1:1 ratio is the "normal" case for most linux users; there is no up-scaling performed by the GPU/driver and applications get the full display (in my case 4K) to render. In those cases the ratio would always be 1.0, and the app renders very "small".

After digging into the inner workings of Flutter I found that the logical pixels, as given by the MediaQuery are actually the physical pixels we pass here divided by the ratio we pass here. Logical pixels are used for rendering things equally across devices.

Logical pixels are roughly the same visual size across devices.
https://docs.flutter.io/flutter/widgets/MediaQueryData/size.html

For example:
A Nexus 6 has a pixel_ratio of 3.5 (560 DPI of the device divided by 160 as baseline 1.0 scale).
It's physical resolution is 1440x2560. But the Flutter MediaQuery size gives us only 411.4x683.4 logical pixels.

If I create a Container widget at 400 width (Container.width field takes logical pixels), it is rendered at 400*3.5 = 1400 physical pixels on the Nexus 6. And it is rendered at 400*1.6 = 640 pixels on my laptop. The physical size in inches is nearly equal on both devices, which is the goal of Flutter.

Therefore I think we must take the DPI of the display into account here, so that the logical pixels in Flutter are calculated correctly on any device.

The one thing I'm not sure of is screen coordinates vs pixels in GLFW. If Flutter renders to a canvas of screen coordinates, we should not calculate and apply pixelsPerScreenCoordinates. I would break things (render out of frame, or only use a small part of the frame). If Flutter renders to actual physcial pixels (as defined on the framebuffer), then we should.

One you provide a non-1 value for pixel_ratio you'll also need to adjust any coordinates sent to Flutter, or click handling will break.

On my machine I'm running with 1.6, and click handling works as before. But pixelsPerScreenCoordinates is 1.0, so there is no difference between the width/height we pass to flutter here and the coordinates obtained from glfwGetCursorPos.
Indeed, if pixelsPerScreenCoordinates is actually needed, then we should apply that factor to positions in GLFWmouseButtonCallback.

I will be able to test on multiple monitors soon. I hope that gives me a non-1.0 pixelsPerScreenCoordinates to see how that works out for the rendering.

@stuartmorgan-g
Copy link
Collaborator

I believe that 1:1 ratio is the "normal" case for most linux users; there is no up-scaling performed by the GPU/driver and applications get the full display (in my case 4K) to render.

I see, it sounds like I'm overfitting from the way macOS handles high-resolution displays then.

If Flutter renders to actual physcial pixels (as defined on the framebuffer)

Unless the engine works very differently on Linux than macOS (which seems unlikely for the layer that would be involved), the sizes and coordinates passed through the embedding API should be in framebuffer coordinates. E.g., glfwGetFramebufferSize's output values should be what are passed as the width and height in that struct AFAIK. The current GLFW embedder code relies on the multiplier being 1.

@GeertJohan
Copy link
Contributor Author

I just tried multiple screen configurations. It seems that relying on GLFW alone won't be enough when using scaling in X11.
For example, I'm running my laptop with builtin 4K display and a second (DP-attached) 1680x1050 display. That display is scaled 2x (3360x2100).
selection_006

GLFW claims:

The virtual position of a monitor is in screen coordinates and, together with the current video mode, describes the viewports that the connected monitors provide into the virtual desktop that spans them.

However, GLFW seems to mess up the size of the second monitor. A call to glfwGetVideoMode for the second monitor reports the native resolution (1680x1050) in screen coordinates. But the display actually spans 3360x2100 in the virtual desktop. The wrong size messes up the display detection I built (locally) which should be detecting on which monitor the window currently resides. Because window's position as glfwGetWindowPos is the position in the virtual desktop.

There seems to be no way to obtain the correct viewport size of a scaled monitor without talking to X11. I guess this explains why a lot of applications I'm using only scale to the primary monitor, and ignore other monitors. (VS Code, firefox, etc.)

It seems that some support for scale differences on monitors is added with the glfwGetMonitorContentScale function, which was introduced in GLFW v3.3. That version was released in november 2018 so we probably shouldn't rely on that just yet.

I think the best way moving forward for now is calculating DPI based on the primary monitor, as done in this PR. This works great as long as secondary monitors are scaled to match the primary. It's not perfect, but a step forward anyway.

As far as the pixelPerScreenCoordinate is concerned. I haven't yet been able to cause a situation where this is not 1:1. But indeed adding the multiplier to pointer event x/y's makes sense. I've reworked the code to cache most size values to bring down latency on input handling and resizing.

@stuartmorgan-g
Copy link
Collaborator

Pushed a merge, since my refactoring to combine the Linux and Windows implementation conflicted with this. As a result, this now improves the Windows implementation as well :)

Copy link
Collaborator

@stuartmorgan-g stuartmorgan-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some comments on specifics, but overall this looks good, and definitely improves the experience on high-DPI Linux and Windows displays.

I agree that going with this approach for now despite the multi-monitor issue makes sense, since it'll be a big improvement for the common case. The code was changed in the past to add pre-3.3 support, so breaking that again now just for the multi-display case probably isn't the right tradeoff. (I'll file a bug to capture that though.)

FlutterEngineSendWindowMetricsEvent(state->engine, &event);
}

static void UpdateMonitorStats(GLFWwindow *window) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This needs a declaration comment.

Also, since with the changes above this will only compute one value, it would probably be clearer to just make it something like GetMonitorDPI, and have the caller update state with the return value.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function only has one caller and very low complexity. Will inline it into CreateFlutterWindow.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's enough going on in CreateFlutterWindow that I would prefer monitor detail calculation to stay factored out into a function.

glfwGetWindowSize(window, &width, &height);
GLFWwindowSizeCallback(window, width, height);
glfwSetWindowSizeCallback(window, GLFWwindowSizeCallback);
glfwSetWindowSizeCallback(window, GLFWWindowSizeCallback);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, the lower-case W here was intentional, since the typename of the thing being resized is GLFWwindow rather than GLFWWindow; since it's being removed it doesn't really matter though. The mouse/cursor capitalization looks like a mistake though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the capitalization because it was inconsistent across GLFW* functions. GLFWKeyCallback is already with capital K. Which should it be?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would argue that it should have been capital W though, because that's just the name of the callback in GLFW.
If we wanted to include the typename of the thing that's on which the callback is applied consistently it should imo be GLFWwindowWindowSizeCallback, GLFWwindowFramebufferSizeCallback, GLFWwindowMouseButtonCallback etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could actually still do that right now? Prefixing all the callback functions with the type on which they are called (GLFWwindow) instead of just GLFW.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Standardizing on capitalizing is the right way to go; I just remember that it came up during the initial review (the argument wasn't based on the parameter being passed, but that it was a GLFWwindow size that was changing). In retrospect it clearly has been confusing and should have been capitalized then.

@GeertJohan GeertJohan force-pushed the feature/linux-hidpi branch from 802c39a to 28ebbe5 Compare January 3, 2019 13:12
@GeertJohan
Copy link
Contributor Author

Thanks for the feedback @stuartmorgan. I have pushed the changes. There's one conversation still open/unclear to me, about the GLFWwindow prefix. When that is resolved I hope this should be good to merge.

@GeertJohan GeertJohan changed the title [linux] Add pixel_ratio to the embedder [glfw] Add pixel_ratio to the embedder Jan 3, 2019
std::unique_ptr<flutter_desktop_embedding::KeyEventHandler> key_event_handler;

// Window framebuffer dimensions, in pixels.
int width_px, height_px = 1;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are these statefull instead of passed as parameters? It looks like they are only ever stored just before calling a function and read inside that function.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Check, I have inlined SyncWindowMetrics.

@GeertJohan
Copy link
Contributor Author

The commit history kinda became a mess with all the changes, so I have rebased the changes on the latest master and opened a new PR for it. #204

@GeertJohan GeertJohan closed this Jan 4, 2019
@stuartmorgan-g
Copy link
Collaborator

For future reference we land all PRs with squash-and-merge, so the commit history of PRs doesn't matter.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants