Skip to content

script: Get the window rectangle from the WebViewDelegate instead of via the compositor#37960

Merged
mrobinson merged 8 commits intoservo:mainfrom
yezhizhen:fix-outerwidth
Jul 11, 2025
Merged

script: Get the window rectangle from the WebViewDelegate instead of via the compositor#37960
mrobinson merged 8 commits intoservo:mainfrom
yezhizhen:fix-outerwidth

Conversation

@yezhizhen
Copy link
Copy Markdown
Member

@yezhizhen yezhizhen commented Jul 9, 2025

Previously, screenX, screenY, outerHeight, outerWidth, moveBy, resizeBy ask compositor for window rectangle, which then return "inner" rectangle after consulting Embedder.

This PR

  1. removes GetClientWindowRect from compositor, and directly let script ask embedder.
  2. add window_size to ScreenGeometry
  3. add a lot of docs to ScreenGeometry

Testing: tests\wpt\mozilla\tests\mozilla\window_resizeTo.html can now pass for Headed Window.
Fixes: #37824

@yezhizhen yezhizhen requested review from jdm and xiaochengh July 9, 2025 10:39
@yezhizhen yezhizhen changed the title script: ask embedder for Window Rectangle script: directly ask embedder for Window Rectangle instead of proxy through compositor Jul 9, 2025
@yezhizhen
Copy link
Copy Markdown
Member Author

Ah just found out one more bug.. Fixing in #37961. We would have new wpt passing when combining both!

github-merge-queue bot pushed a commit that referenced this pull request Jul 9, 2025
#37961)

Previously, we only consider OS decoration height. But when testing
#37960, I find that the decoration width is also non-zero.

Testing: Need to wait W3C spec change
web-platform-tests/wpt#53421 related to
webdriver rectangle. When combined with #37960, this can fix at least
`window_resizeTo.html`.

Signed-off-by: Euclid Ye <[email protected]>
@yezhizhen
Copy link
Copy Markdown
Member Author

Ah just found out one more bug.. Fixing in #37961. We would have new wpt passing when combining both!

#37899 (comment)
Just found out that test was passed accidentally a few days ago... But the change here would make it pass for headed window.

Comment on lines +480 to +484
/// Get the window rectangle.
fn window_rect(&self, _webview: WebView) -> DeviceIndependentIntRect {
DeviceIndependentIntRect::default()
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not add a new WebViewDelegate method for this, as now the embedder needs to implement two things in order to properly communicate window sizes. Instead in libservo, somehow access the RenderingContext size and the offset from the existingscreen_geometry() method.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hidpi_scale_factor is stored in WebViewInner as part of the WebView struct, so you should have access to it in libservo.

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. But RenderingContext/WindowRenderingContext etc. all only holds inner_size, same as the previous compositor proxy.

I also checked WebViewInner.rect, which is supposed to be inner_size rect, but currently heavily & wrongly mix-used.. (Will file an issue tomorrow)

TL;DR: It is still difficult to get outer_size without adding method to WebViewDelegate...

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

now the embedder needs to implement two things in order to properly communicate window sizes.

I have a idea: remove fn window_rect from WindowPortsMethods added last week in #37812. Just move logic to our newly created WebViewDelegate::window_rect.

This way, we also supports egl.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. So the issue is that we really need the outer window rect and also the inner window rect? Can you have ScreenGeometry return this information? I think it makes sense that we get all of this information in a similar place (for now) so that the person embedding Servo understands how all the parts fit together.

/// Information about a `WebView`'s screen and containing window geometry. This
/// is used for the [Screen](https://drafts.csswg.org/cssom-view/#the-screen-interface)
/// CSSOM APIs and `window.screenLeft` / `window.screenTop`.
#[derive(Clone, Copy, Debug, Default)]
pub struct ScreenGeometry {
    /// The size of the screen in device pixels. This will be converted to
    /// CSS pixels based on the pixel scaling of the `WebView`.
    pub size: DeviceIntSize,
    /// The available size of the screen in device pixels. This size is the size
    /// available for web content on the screen, and should be `size` minus any system
    /// toolbars, docks, and interface elements. This will be converted to
    /// CSS pixels based on the pixel scaling of the `WebView`.
    pub available_size: DeviceIntSize,
    /// The rectangle in device pixels of the `WebView`'s containing window on the screen.
    pub window_rect: DeviceIntRect,
    /// The offset of the `WebView` in its containing window in device pixels..
    pub offset_in_window: DeviceIntPoint,

Would that work?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works like a charm.

let pos = screen.window_offset.to_f32();
let window_size = screen.window_size.to_f32();

let pos_in_css_pixel = (pos / dpi).round().to_i32();
Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have an important question, given that I encountered several rounding display/layout issues before (e.g. #37243). Should we round or truncate?
They seem to be mix-used in Servo rn. Compositor always truncates. Layout sometimes rounds.
cc @Loirooriol @sagudev

Personally, I tends to keep fractional value as long as possible (like during layout calculation), and round when we have to. Never truncates.

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be left as an open question. I am only certain that WebDriver set window rect should always round.
Can you review this pls? @mrobinson

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good question that I don't know the answer to. :/

@yezhizhen yezhizhen added the T-linux-wpt Do a try run of the WPT label Jul 11, 2025
@github-actions github-actions bot removed the T-linux-wpt Do a try run of the WPT label Jul 11, 2025
@github-actions
Copy link
Copy Markdown

🔨 Triggering try run (#16213003380) for Linux (WPT)

@github-actions
Copy link
Copy Markdown

Test results for linux-wpt from try job (#16213003380):

Flaky unexpected result (18)
  • FAIL [expected PASS] /_mozilla/css/dirty_viewport.html (#13731)
  • TIMEOUT [expected OK] /_mozilla/mozilla/window_resize_event.html (#36741)
    • TIMEOUT [expected PASS] subtest: Popup onresize event fires after resizeTo

      Test timed out
      

  • OK /_webgl/conformance/textures/misc/texture-upload-size.html (#21770)
    • PASS [expected FAIL] subtest: WebGL test #61
    • PASS [expected FAIL] subtest: WebGL test #63
    • PASS [expected FAIL] subtest: WebGL test #65
    • PASS [expected FAIL] subtest: WebGL test #67
    • FAIL [expected PASS] subtest: WebGL test #69

      assert_true: Texture was smaller than the expected size 2x2 expected true got false
      

    • FAIL [expected PASS] subtest: WebGL test #71

      assert_true: getError expected: INVALID_VALUE. Was NO_ERROR : when calling texSubImage2D with the same texture upload with offset 1, 1 expected true got false
      

    • FAIL [expected PASS] subtest: WebGL test #73

      assert_true: Texture was smaller than the expected size 2x2 expected true got false
      

    • FAIL [expected PASS] subtest: WebGL test #75

      assert_true: getError expected: INVALID_VALUE. Was NO_ERROR : when calling texSubImage2D with the same texture upload with offset 1, 1 expected true got false
      

  • TIMEOUT [expected OK] /_webgl/conformance/uniforms/out-of-bounds-uniform-array-access.html (#26225)
    • NOTRUN [expected PASS] subtest: Overall test
  • OK /css/css-cascade/layer-font-face-override.html (#35935)
    • FAIL [expected PASS] subtest: @font-face override update with appended sheet 1

      assert_equals: expected "80px" but got "38.3166666666667px"
      

    • FAIL [expected PASS] subtest: @font-face override update with appended sheet 2

      assert_equals: expected "80px" but got "38.3166666666667px"
      

  • OK /css/css-fonts/generic-family-keywords-001.html (#37467)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted generic(fangsong)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted generic(kai)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted generic(khmer-mul)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted generic(nastaliq)
  • OK /css/css-fonts/variations/at-font-face-font-matching.html (#20684)
    • PASS [expected FAIL] subtest: Matching font-style: 'oblique -21deg' should prefer 'oblique 40deg 50deg' over 'italic'
  • ERROR [expected TIMEOUT] /fetch/fetch-later/quota/same-origin-iframe/max-payload.tentative.https.window.html (#35210)
  • OK /html/browsers/browsing-the-web/navigating-across-documents/009.html (#24456)
    • FAIL [expected PASS] subtest: Link with onclick form submit to javascript url with document.write and href navigation

      assert_array_equals: expected property 1 to be "href" but got "click" (expected array ["write", "href"] got ["write", "click"])
      

  • TIMEOUT [expected OK] /html/browsers/browsing-the-web/navigating-across-documents/anchor-fragment-form-submit.html
  • TIMEOUT [expected OK] /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/iframe-src-aboutblank-navigate-immediately.html (#29048)
    • TIMEOUT [expected FAIL] subtest: Navigating to a different document with form submission

      Test timed out
      

  • OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/load-pageshow-events-iframe-contentWindow.html (#28681)
    • FAIL [expected PASS] subtest: load &amp; pageshow events do not fire on contentWindow of &lt;iframe&gt; element created with src=''

      assert_unreached: load should not be fired Reached unreachable code
      

    • PASS [expected FAIL] subtest: load &amp; pageshow events do not fire on contentWindow of &lt;iframe&gt; element created with src='about:blank'
  • OK /html/browsers/browsing-the-web/navigating-across-documents/initial-empty-document/load-pageshow-events-window-open.html (#28691)
    • FAIL [expected PASS] subtest: load event does not fire on window.open('about:blank')

      assert_unreached: load should not be fired Reached unreachable code
      

  • PASS [expected FAIL] /html/canvas/element/manual/text/canvas.2d.disconnected.html (#30063)
  • OK [expected CRASH] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-3.html (#24066)
  • OK /resize-observer/change-layout-in-error.html (#32629)
    • PASS [expected FAIL] subtest: Changing layout in window error handler should not result in lifecyle loop when resize observer loop limit is reached.
  • OK /resize-observer/eventloop.html (#33599)
    • FAIL [expected PASS] subtest: test0: multiple notifications inside same event loop

      assert_equals: new loop expected 1 but got 0
      

  • OK [expected ERROR] /webxr/render_state_update.https.html (#27535)
Stable unexpected results that are known to be intermittent (17)
  • TIMEOUT /FileAPI/url/url-in-tags-revoke.window.html (#19978)
    • PASS [expected TIMEOUT] subtest: Fetching a blob URL immediately before revoking it works in &lt;script&gt; tags.
  • FAIL [expected PASS] /_mozilla/css/stacked_layers.html (#15988)
  • FAIL [expected PASS] /_mozilla/mozilla/sslfail.html (#10760)
  • FAIL [expected PASS] /css/css-sizing/dynamic-available-size-iframe.html (#31559)
  • OK /fetch/metadata/generated/css-font-face.https.sub.tentative.html (#32732)
    • FAIL [expected PASS] subtest: sec-fetch-dest

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."
      

  • OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-same-origin.window.html (#29049)
    • PASS [expected FAIL] subtest: Same-origin navigation started from unload handler must be ignored
  • OK /html/browsers/history/the-history-interface/traverse_the_history_5.html (#21383)
    • PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
  • TIMEOUT /html/interaction/focus/the-autofocus-attribute/supported-elements.html (#24145)
    • TIMEOUT [expected NOTRUN] subtest: Non-HTMLElement should not support autofocus

      Test timed out
      

  • TIMEOUT [expected OK] /html/interaction/focus/the-autofocus-attribute/update-the-rendering.html (#24145)
    • TIMEOUT [expected FAIL] subtest: "Flush autofocus candidates" should be happen before a scroll event and animation frame callbacks

      Test timed out
      

  • TIMEOUT /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-2.html (#22667)
    • TIMEOUT [expected FAIL] subtest: Check that popups from a sandboxed iframe escape the sandbox if allow-popups-to-escape-sandbox is used

      Test timed out
      

  • CRASH [expected TIMEOUT] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_escaping-3.html (#24057)
  • TIMEOUT [expected OK] /html/semantics/embedded-content/the-iframe-element/iframe_sandbox_popups_nonescaping-2.html (#22154)
    • NOTRUN [expected FAIL] subtest: Check that popups from a sandboxed iframe do not escape the sandbox
  • OK /html/webappapis/user-prompts/print-during-unload.html (#35944)
    • PASS [expected FAIL] subtest: print() during unload
  • OK /preload/preload-error.sub.html (#37177)
    • PASS [expected FAIL] subtest: CORS (style): main
    • PASS [expected FAIL] subtest: 404 (xhr): main
    • PASS [expected FAIL] subtest: CORS (xhr): main
    • FAIL [expected PASS] subtest: MIME-error (script): main

      assert_greater_than: http://web-platform.test:8000/preload/resources/dummy.css?pipe=header%28Content-Type%2Ctext%2Fnotjavascript%29&amp;label=script should be loaded expected a number greater than 0 but got 0
      

  • TIMEOUT [expected OK] /resource-timing/nested-context-navigations-iframe.html (#24311)
    • TIMEOUT [expected PASS] subtest: Test that iframe navigations are not observable by the parent, even after history navigations by the parent

      Test timed out
      

    • NOTRUN [expected PASS] subtest: Test that crossorigin iframe navigations are not observable by the parent, even after history navigations by the parent
    • NOTRUN [expected PASS] subtest: Test that cross-site iframe navigations are not observable by the parent, even after history navigations by the parent
    • NOTRUN [expected PASS] subtest: Test that iframe navigations are not observable by the parent
    • NOTRUN [expected PASS] subtest: Test that crossorigin iframe navigations are not observable by the parent
    • NOTRUN [expected PASS] subtest: Test that cross-site iframe navigations are not observable by the parent
    • NOTRUN [expected PASS] subtest: Test that iframe refreshes are not observable by the parent
    • NOTRUN [expected PASS] subtest: Test that crossorigin iframe refreshes are not observable by the parent
    • NOTRUN [expected PASS] subtest: Test that cross-site iframe refreshes are not observable by the parent
  • CRASH [expected TIMEOUT] /trusted-types/trusted-types-navigation.html?06-10 (#37920)
  • TIMEOUT [expected OK] /webmessaging/with-ports/018.html (#24485)
    • TIMEOUT [expected PASS] subtest: origin of the script that invoked the method, javascript:

      Test timed out
      

@github-actions
Copy link
Copy Markdown

✨ Try run (#16213003380) succeeded.

Copy link
Copy Markdown
Member

@mrobinson mrobinson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking great. This is a nice cleanup. Just one real issue here and bunch of nits. Thanks!

EmbedderMsg::GetWindowRect(webview_id, response_sender) => {
let window_rect = if let Some(webview) = self.get_webview_handle(webview_id) {
let dpi = webview.dpi();
let screen = webview.delegate().screen_geometry(webview).unwrap();
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This unwrap means that by default Servo will panic as the default implementation of fn screen_geometry returns None. Please do this:

Suggested change
let screen = webview.delegate().screen_geometry(webview).unwrap();
let Some(screen_geometry) = webview.delegate().screen_geometry(webview) else {
return Default::default();
}

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. I wrote like this because all egl/desktop implementation of screen_geometry returns Some.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but we don't have control over what other embedders do (the embedding API is meant to be generic and public) and the interface allows returning None as a valid response. We need to obey the contract the API gives.

@mrobinson mrobinson changed the title script: directly ask embedder for Window Rectangle instead of proxy through compositor script: Get the window rectangle from the WebViewDelegate instead of via the compositor Jul 11, 2025
@yezhizhen yezhizhen requested a review from mrobinson July 11, 2025 09:40
Signed-off-by: Euclid Ye <[email protected]>
Signed-off-by: Euclid Ye <[email protected]>
@mrobinson mrobinson added the T-android Do a try run on Android label Jul 11, 2025
@github-actions github-actions bot removed the T-android Do a try run on Android label Jul 11, 2025
@github-actions
Copy link
Copy Markdown

🔨 Triggering try run (#16219671511) for Android

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! But please address the concern.

};

(screen_geometry.window_rect.to_f32() / hidpi_scale_factor)
.round_out()
Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should round instead of round_out, because we want "as close as possible" as claimed in #37960 (comment) to be consistent with WebDriver get rect. It is ok if things don't fit in.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I search the WebDriver spec for the text "as close as possible" and I only see that this should apply when trying to resize the window. I don't interpret this as referring to rounding floating point pixels though, but for when the operating system imposes other kinds of sizing constraints on the window. In any case, is this terminology used anywhere with regard to actually getting the window's size.

FWIW, I used round_out here as it seemed to make more sense that the rectangle contained the entire window versus just part of it.

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But if using round_out, this will start to fail some WebDriver tests. Do you suggest that I change other round to round_out in previous merged PR?

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FWIW, I used round_out here as it seemed to make more sense that the rectangle contained the entire window versus just part of it.

But this can also introduce some danger. You may try to paint on larger than available window, leading to some obscured corner. round_in would make more sense to me..

EDIT: Maybe leave as another open question, and use round() for now?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these tests using floating point window sizes?

Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I test on my own high DPR device with headed window. The assertion will be off by 1 with this.

For headless window, this should be fine as DPR is always 1.

size: self.screen_size,
available_size: self.screen_size,
offset: Default::default(),
window_rect: Default::default(),
Copy link
Copy Markdown
Member Author

@yezhizhen yezhizhen Jul 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The window_size here wasn't 0, but self.inner_size.get().
The coordinates has always been 0 so it's ok.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. This should be fixed now.

@github-actions
Copy link
Copy Markdown

✨ Try run (#16219671511) succeeded.

yezhizhen and others added 2 commits July 11, 2025 23:21
Signed-off-by: Euclid Ye <[email protected]>
Co-authored-by: Martin Robinson <[email protected]>
Signed-off-by: Martin Robinson <[email protected]>
@mrobinson
Copy link
Copy Markdown
Member

I am not sure about the rounding solution, as I think it might be covering up some other issue, but it's okay for now. Could it be that when we change the size of the window, we need to round up as well?

@mrobinson mrobinson enabled auto-merge July 11, 2025 17:50
@mrobinson mrobinson added this pull request to the merge queue Jul 11, 2025
Merged via the queue into servo:main with commit c5aeac3 Jul 11, 2025
21 checks passed
@yezhizhen yezhizhen deleted the fix-outerwidth branch July 12, 2025 02:24
@yezhizhen
Copy link
Copy Markdown
Member Author

yezhizhen commented Jul 12, 2025

I am not sure about the rounding solution, as I think it might be covering up some other issue, but it's okay for now. Could it be that when we change the size of the window, we need to round up as well?

I took a closer look.

  1. I believe there is some spec issue.
    [Set Window Rect] Change parameter requirement from Number to Integer and range to "maximum safe integer" w3c/webdriver#1909 (comment)

  2. For example, the test requires response to be integer, which is not mentioned in spec. Nor is it sensible considering DPR.

    def test_response_payload(session):

  3. The set parameter can be floats. But it is not mentioned why we expect rounding in following way.

    def test_width_height_floats(session):
    response = set_window_rect(session, {"width": 550.5, "height": 420})
    value = assert_success(response, session.window.rect)
    assert value["width"] == 550


TL;DR: Will file some issues in w3c/webdriver and discuss before getting to this.

EDIT: Opened in w3c/webdriver#1911

@mrobinson
Copy link
Copy Markdown
Member

Great! Thank you for the investigation.

github-merge-queue bot pushed a commit that referenced this pull request Jul 12, 2025
…via the compositor (#38020)

Similar to #37960, previously, `AvailHeight`, `AvailWidth`, `Height`,
`Width` ask compositor for screen metrics. This PR moves the request to
embedder.

This simplifies code, and reduces workload of compositor, which is
busier most of time.

Testing: No behaviour change. Updated some tests. `Width/Height` matches
other browsers.

---------

Signed-off-by: Euclid Ye <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ScreenY and outerHeight ignores the title bar height

3 participants