Skip to content

Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS by removing .ini#40714

Merged
jschwe merged 1 commit intoservo:mainfrom
WaterWhisperer:tests
Nov 18, 2025
Merged

Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS by removing .ini#40714
jschwe merged 1 commit intoservo:mainfrom
WaterWhisperer:tests

Conversation

@WaterWhisperer
Copy link
Copy Markdown
Contributor

@WaterWhisperer WaterWhisperer commented Nov 18, 2025

Testing: None
Fixes: #25046

@servo-highfive servo-highfive added the S-awaiting-review There is new code that needs to be reviewed. label Nov 18, 2025
yezhizhen
yezhizhen requested changes Nov 18, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@yezhizhen yezhizhen left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do you think it is outdated? The test is still there. And when I run it, still got error as expected most of time.

Also it is still intermittent.

@servo-highfive servo-highfive added S-needs-code-changes Changes have not yet been made that were requested by a reviewer. and removed S-awaiting-review There is new code that needs to be reviewed. labels Nov 18, 2025
@WaterWhisperer
Copy link
Copy Markdown
Contributor Author

WaterWhisperer commented Nov 18, 2025

Why do you think it is outdated? The test is still there. And when I run it, still got error as expected most of time.

Also it is still intermittent.

When I run ./mach test-wpt tests/wpt/tests/html/semantics/embedded-content/media-elements/event_timeupdate_noautoplay.html, it 100/100 pass. Is my method wrong?

@mrobinson
Copy link
Copy Markdown
Member

mrobinson commented Nov 18, 2025

When I run ./mach test-wpt tests/wpt/tests/html/semantics/embedded-content/media-elements/event_timeupdate_noautoplay.html, it 100/100 pass. Is my method wrong?

Tests can fail on CI, but pass locally and vice versa. This requires doing an investigation into why this is happening in order to fix it. It is very valuable work if you are willing to do it!

@yezhizhen
Copy link
Copy Markdown
Member

yezhizhen commented Nov 18, 2025 •
edited
Loading

it 100/100 pass.

This means it matches the expectation, a.k.a. Error.

For me I run until it fails expectation, and get 29/30 pass. But those 29 pass means "ERROR".

And the 1 is actually "OK".

@yezhizhen yezhizhen added the T-linux-wpt Do a try run of the WPT label Nov 18, 2025
@github-actions github-actions bot removed the T-linux-wpt Do a try run of the WPT label Nov 18, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 18, 2025

🔨 Triggering try run (#19464504204) for Linux (WPT)

@yezhizhen
Copy link
Copy Markdown
Member

yezhizhen commented Nov 18, 2025 •
edited
Loading

What it looks like for me in 2028dab. Each repetition has 4 subtests. It is only OK (failing "ERROR" expectation) for one subtest at third repetition.

 0:26.56 INFO Repetition 3
 0:26.56 SUITE_START: web-platform-test - running 1 tests
 0:26.56 INFO Using 1 child processes
 0:26.56 WARNING Browser.__init__ kwargs: {'capabilities': None}
 0:26.56 INFO Starting WebDriver: D:\servo\target\release\servo.exe --webdriver=25451 --hard-fail -u Servo/wptrunner --ignore-certificate-errors --enable-experimental-web-platform-features --window-size 800x600 about:blank --certificate-path D:\servo\tests\wpt\tests\tools\certs\cacert.pem --config-dir C:\Users\Y84399~1\AppData\Local\Temp\servo-t87wdw63 --user-stylesheet D:\servo\tests\wpt\tests\fonts\ahem.css --headless --prefs-file D:\servo\resources\wpt-prefs.json
 0:27.21 INFO Webdriver started successfully.
 0:27.21 INFO Starting runner
 0:28.49 TEST_START: /html/semantics/embedded-content/media-elements/event_timeupdate_noautoplay.html
 0:28.54 pid:26576 Full command: D:\servo\target\release\servo.exe --webdriver=25451 --hard-fail -u Servo/wptrunner --ignore-certificate-errors --enable-experimental-web-platform-features --window-size 800x600 about:blank --certificate-path D:\servo\tests\wpt\tests\tools\certs\cacert.pem --config-dir C:\Users\Y84399~1\AppData\Local\Temp\servo-t87wdw63 --user-stylesheet D:\servo\tests\wpt\tests\fonts\ahem.css --headless --prefs-file D:\servo\resources\wpt-prefs.json
pid:26576
 0:28.54 pid:26576 (servo.exe:26576): GLib-GObject-WARNING **: 19:49:24.675: ../gobject/gsignal.c:2772: instance '0000021EE2505000' has no handler with id '32'
 0:28.54 pid:26576
 0:28.54 pid:26576 (servo.exe:26576): GLib-GObject-WARNING **: 19:49:24.676: ../gobject/gsignal.c:2772: instance '0000021EE2505170' has no handler with id '75'
 0:28.76 TEST_END: Test OK, expected ERROR. Subtests passed 4/4. Unexpected 0
 0:28.76 INFO No more tests
 0:31.80 INFO Trying to shut down gracefully by extension command
 0:28.76 INFO Closing logging queue
 0:28.76 INFO queue closed
 0:37.88 SUITE_END

web-platform-test
~~~~~~~~~~~~~~~~~
Ran 15 checks (12 subtests, 3 tests)
Expected results: 14
Unexpected results: 1
  test: 1 (1 ok)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 18, 2025

Test results for linux-wpt from try job (#19464504204):

Flaky unexpected result (41)
  • OK /FileAPI/url/url-with-fetch.any.worker.html (#21517)
    • FAIL [expected PASS] subtest: Revoke blob URL after calling fetch, fetch should succeed

      promise_test: Unhandled rejection with value: object "TypeError: Network error occurred"

  • OK /IndexedDB/idbfactory_open.any.html
    • FAIL [expected PASS] subtest: Calling open() with version argument 1.5 should not throw.

      assert_equals: version expected 1 but got 9007199254740991

  • OK /_mozilla/mozilla/getBoundingClientRect.html (#39668)
    • FAIL [expected PASS] subtest: getBoundingClientRect 1

      assert_equals: expected 62 but got 60.35

  • OK /_mozilla/webxr/create_session.https.html
    • FAIL [expected PASS] subtest: create_session

      can't access property "simulateDeviceConnection", navigator.xr.test is undefined

  • ERROR [expected TIMEOUT] /_mozilla/webxr/sessionavailable.https.html
  • CRASH [expected OK] /_webgl/conformance/canvas/canvas-zero-size.html
  • CRASH [expected OK] /_webgl/conformance/ogles/GL/pow/pow_001_to_008.html
  • CRASH [expected OK] /_webgl/conformance/ogles/GL/sin/sin_001_to_006.html (#21493)
  • CRASH [expected OK] /_webgl/conformance/rendering/draw-arrays-out-of-bounds.html
  • CRASH [expected OK] /_webgl/conformance/rendering/gl-scissor-canvas-dimensions.html
  • CRASH [expected OK] /_webgl/conformance/state/gl-initial-state.html
  • CRASH [expected OK] /_webgl/conformance/textures/image_bitmap_from_image_data/tex-2d-rgb-rgb-unsigned_short_5_6_5.html
  • CRASH [expected OK] /_webgl/conformance2/extensions/ovr_multiview2_timer_query.html
  • CRASH [expected ERROR] /_webgl/conformance2/textures/image/tex-3d-rgb9_e5-rgb-half_float.html
  • CRASH [expected ERROR] /_webgl/conformance2/textures/misc/tex-new-formats.html
  • CRASH [expected OK] /_webgl/conformance2/wasm/readpixels-2gb-in-4gb-wasm-memory.html
  • OK [expected TIMEOUT] /cookiestore/httponly_cookies.https.window.html (#40697)
    • FAIL [expected TIMEOUT] subtest: HttpOnly cookies are not observed

      assert_equals: HttpOnly cookie we wrote using HTTP in cookie jar is invisible to script expected (undefined) undefined but got (string) "cspViolationReportCookie1=1"

    • FAIL [expected NOTRUN] subtest: HttpOnly cookies can not be set by document.cookie

      assert_equals: Trying to store an HttpOnly cookie with document.cookie fails expected "cookie1=value1; cookie3=value3" but got "HTTPONLY-cookie=value; cookie1=value1; cookie3=value3"

    • FAIL [expected NOTRUN] subtest: HttpOnly cookies can not be set by CookieStore

      assert_equals: httpOnly is not an option for CookieStore.set() expected "cookie1=value1; cookie2=value2; cookie3=value3" but got "HTTPONLY-cookie=value; cookie1=value1; cookie2=value2; cookie3=value3"

    • FAIL [expected NOTRUN] subtest: HttpOnly cookies are not deleted/overwritten

      assert_equals: HttpOnly cookie is not deleted expected (string) "HTTPONLY-cookie=value" but got (undefined) undefined

  • TIMEOUT [expected OK] /credential-management/credentialscontainer-frame-basics.https.html (#39430)
    • TIMEOUT [expected FAIL] subtest: navigator.credentials should be undefined in documents generated from data: URLs.

      Test timed out

  • FAIL [expected PASS] /css/css-backgrounds/background-size-041.html
  • OK /css/css-cascade/layer-font-face-override.html (#35935)
    • PASS [expected FAIL] subtest: @font-face override update with appended sheet 1
    • PASS [expected FAIL] subtest: @font-face override update with appended sheet 2
  • OK /css/css-fonts/generic-family-keywords-003.html (#38994)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted generic(nastaliq) (drawing text in a canvas)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted ui-sans-serif (drawing text in a canvas)
  • OK /custom-elements/form-associated/ElementInternals-setFormValue.html (#29174)
    • PASS [expected FAIL] subtest: Multiple values - name content attribute is ignored
  • CRASH [expected ERROR] /encoding/streams/decode-non-utf8.any.serviceworker.html
  • ERROR /fetch/metadata/generated/serviceworker.https.sub.html (#36247)
    • FAIL [expected PASS] subtest: sec-fetch-site - Same origin, no options - registration

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

  • TIMEOUT [expected CRASH] /fetch/metadata/window-open.https.sub.html (#40339)
  • OK [expected ERROR] /focus/focus-event-after-switching-iframes.sub.html (#40368)
  • ERROR [expected TIMEOUT] /html/browsers/browsing-the-web/history-traversal/pageswap/pageswap-initial-navigation.html (#40387)
  • CRASH [expected OK] /html/browsers/the-window-object/window-reuse-in-nested-browsing-contexts.tentative.html
  • OK /html/semantics/document-metadata/the-meta-element/pragma-directives/attr-meta-http-equiv-refresh/allow-scripts-flag-changing-1.html (#39694)
    • PASS [expected FAIL] subtest: Meta refresh is blocked by the allow-scripts sandbox flag at its creation time, not when refresh comes due
  • OK /html/semantics/document-metadata/the-meta-element/pragma-directives/attr-meta-http-equiv-refresh/allow-scripts-flag-changing-2.html (#39703)
    • FAIL [expected PASS] subtest: Meta refresh of the original iframe is not blocked if moved into a sandboxed iframe

      uncaught exception: Error: assert_unreached: The iframe into which the meta was moved must not refresh Reached unreachable code

  • OK [expected TIMEOUT] /html/semantics/embedded-content/media-elements/src_object_blob.html (#40340)
    • PASS [expected TIMEOUT] subtest: HTMLMediaElement.srcObject blob
  • OK /html/semantics/forms/form-submission-0/text-plain.window.html (#28687)
    • FAIL [expected PASS] subtest: text/plain: Basic test (formdata event)

      assert_equals: expected "basic=test\r\n" but got ""

    • FAIL [expected PASS] subtest: text/plain: Basic File test (normal form)

      assert_equals: expected "basic=file-test.txt\r\n" but got ""

    • PASS [expected FAIL] subtest: text/plain: single quote in filename (formdata event)
  • OK /html/semantics/forms/form-submission-0/urlencoded2.window.html (#28687)
    • FAIL [expected PASS] subtest: application/x-www-form-urlencoded: Basic File test (normal form)

      assert_equals: expected "basic=file-test.txt" but got ""

    • PASS [expected FAIL] subtest: application/x-www-form-urlencoded: 0x00 in name (formdata event)
  • OK [expected ERROR] /html/user-activation/no-activation-thru-escape-key.html (#40343)
  • OK /navigation-timing/test-navigation-type-reload.html (#33334)
    • PASS [expected FAIL] subtest: Reload domComplete > Original domComplete
    • PASS [expected FAIL] subtest: Reload domContentLoadedEventEnd > Original domContentLoadedEventEnd
    • PASS [expected FAIL] subtest: Reload domContentLoadedEventStart > Original domContentLoadedEventStart
    • PASS [expected FAIL] subtest: Reload domInteractive > Original domInteractive
    • PASS [expected FAIL] subtest: Reload fetchStart > Original fetchStart
    • PASS [expected FAIL] subtest: Reload loadEventEnd > Original loadEventEnd
    • PASS [expected FAIL] subtest: Reload loadEventStart > Original loadEventStart
  • CRASH [expected OK] /pointerevents/pointerevent_multiple_pointerover_no_pointer_movement.html
  • OK /service-workers/service-worker/fetch-event.https.html (#36234)
    • PASS [expected FAIL] subtest: Service Worker falls back to network in fetch event with POST form
  • CRASH [expected ERROR] /trusted-types/SharedWorker-importScripts.html
  • CRASH [expected OK] /trusted-types/block-string-assignment-to-DedicatedWorker-setTimeout-setInterval.html
  • TIMEOUT /trusted-types/trusted-types-navigation.html?06-10 (#37920)
    • PASS [expected FAIL] subtest: Navigate a frame via anchor with javascript:-urls in report-only mode.
  • OK [expected TIMEOUT] /trusted-types/trusted-types-navigation.html?31-35 (#38034)
    • PASS [expected TIMEOUT] subtest: Navigate a frame via form-submission with javascript:-urls w/ default policy in report-only mode.
    • FAIL [expected NOTRUN] subtest: Navigate a window via form-submission with javascript:-urls w/ a default policy throwing an exception in enforcing mode.

      promise_test: Unhandled rejection with value: "Unexpected message received: \"No securitypolicyviolation reported!\""

    • FAIL [expected NOTRUN] subtest: Navigate a window via form-submission with javascript:-urls w/ a default policy throwing an exception in report-only mode.

      promise_test: Unhandled rejection with value: "Unexpected message received: \"No securitypolicyviolation reported!\""

    • FAIL [expected NOTRUN] subtest: Navigate a window via form-submission with javascript:-urls w/ a default policy making the URL invalid in enforcing mode.

      promise_test: Unhandled rejection with value: "Unexpected message received: \"No securitypolicyviolation reported!\""

Stable unexpected results that are known to be intermittent (22)
  • OK /IndexedDB/idbobjectstore_getAll.any.html (#39276)
    • PASS [expected FAIL] subtest: Get all values with transaction.commit()
  • OK /IndexedDB/idbobjectstore_getAll.any.worker.html (#39400)
    • PASS [expected FAIL] subtest: Get all values with transaction.commit()
  • FAIL [expected PASS] /_mozilla/mozilla/sslfail.html (#10760)
  • TIMEOUT [expected OK] /_mozilla/mozilla/window_resize_event.html (#36741)
    • TIMEOUT [expected PASS] subtest: Popup onresize event fires after resizeTo

      Test timed out

  • TIMEOUT /content-security-policy/inheritance/location-reload.html (#38983)
    • PASS [expected FAIL] subtest: location.reload() of empty iframe.
  • OK /css/css-fonts/generic-family-keywords-001.html (#37467)
    • FAIL [expected PASS] subtest: @font-face matching for quoted and unquoted generic(khmer-mul)

      assert_equals: quoted generic(khmer-mul) matches  @font-face rule expected 50 but got 30

  • TIMEOUT [expected OK] /fetch/api/redirect/redirect-keepalive.https.any.html (#32153)
    • TIMEOUT [expected PASS] subtest: [keepalive][iframe][load] mixed content redirect; setting up

      Test timed out

  • OK /fetch/fetch-later/permissions-policy/deferred-fetch-allowed-by-permissions-policy.https.window.html (#40478)
    • FAIL [expected PASS] subtest: Permissions policy header: "deferred-fetch=*" allows fetchLater() in the top-level document.

      assert_equals: Number of sent beacons does not match expected count: expected 1 but got 0

  • OK /fetch/metadata/generated/css-font-face.sub.tentative.html (#34624)
    • PASS [expected FAIL] subtest: sec-fetch-storage-access - Not sent to non-trustworthy same-site destination
    • FAIL [expected PASS] subtest: sec-fetch-storage-access - Not sent to non-trustworthy cross-site destination

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

  • OK /fetch/metadata/generated/element-img-environment-change.https.sub.html (#30111)
    • PASS [expected FAIL] subtest: sec-fetch-site - Cross-site, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Same-Origin -> Same-Site -> Same-Origin redirect, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Cross-Site -> Same Origin, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Cross-Site -> Same-Site, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Cross-Site -> Cross-Site, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Same-Origin -> Same Origin, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-mode - no attributes
    • FAIL [expected PASS] subtest: sec-fetch-mode - attributes: crossorigin

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

  • OK /fetch/metadata/generated/element-img-environment-change.sub.html (#30111)
    • PASS [expected FAIL] subtest: sec-fetch-site - Not sent to non-trustworthy cross-site destination, no attributes
    • PASS [expected FAIL] subtest: sec-fetch-storage-access - Not sent to non-trustworthy same-origin destination, no attributes
  • OK /html/browsers/browsing-the-web/navigating-across-documents/005.html (#27062)
    • FAIL [expected PASS] subtest: Link with onclick navigation and href navigation 

      assert_equals: expected "href" but got "click"

  • OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-cross-origin.sub.window.html (#29056)
    • FAIL [expected PASS] subtest: Cross-origin navigation started from unload handler must be ignored

      promise_test: Unhandled rejection with value: object "SecurityError: The operation is insecure."

  • OK /html/browsers/browsing-the-web/navigating-across-documents/replace-before-load/a-click.html (#28697)
    • PASS [expected FAIL] subtest: aElement.click() before the load event must NOT replace
  • OK /html/browsers/windows/embedded-opener-remove-frame.html (#23867)
    • FAIL [expected PASS] subtest: opener of discarded auxiliary browsing context

      assert_object_equals: property "get" expected function "function opener() {
    [native code]
}" got function "function opener() {
    [native code]
}"

  • OK /html/dom/documents/resource-metadata-management/document-cookie.html (#40673)
    • FAIL [expected PASS] subtest: document has no cookie

      assert_equals: expected "" but got "domain-attribute-matches-host=0"

    • FAIL [expected PASS] subtest: document.cookie

      assert_equals: expected "" but got "domain-attribute-matches-host=0"

    • FAIL [expected PASS] subtest: document.cookie 1

      assert_equals: expected "a=b" but got "a=b; domain-attribute-matches-host=0"

    • FAIL [expected PASS] subtest: document.cookie 2

      assert_equals: expected "" but got "a=b; domain-attribute-matches-host=0"

  • ERROR [expected OK] /html/infrastructure/common-dom-interfaces/collections/domstringlist.html (#40665)
  • OK [expected TIMEOUT] /html/interaction/focus/the-autofocus-attribute/autofocus-dialog.html (#29087)
    • FAIL [expected TIMEOUT] subtest: <dialog>-contained autofocus element gets focused when the dialog is shown

      assert_equals: expected "DIV" but got "BODY"

  • CRASH [expected OK] /pointerevents/compat/pointerevent_touch-action_two-finger_interaction.html (#40418)
  • OK /preload/preload-error.sub.html (#37177)
    • PASS [expected FAIL] subtest: CORS (fetch): main
  • OK /trusted-types/trusted-types-navigation.html?01-05 (#38975)
    • FAIL [expected PASS] subtest: Navigate a window via anchor with javascript:-urls in report-only mode.

      promise_test: Unhandled rejection with value: "Unexpected message received: \"No securitypolicyviolation reported!\""

  • TIMEOUT [expected OK] /trusted-types/trusted-types-navigation.html?26-30 (#38807)
    • PASS [expected FAIL] subtest: Navigate a window via form-submission with javascript:-urls in report-only mode.
    • PASS [expected FAIL] subtest: Navigate a frame via form-submission with javascript:-urls in enforcing mode.
    • TIMEOUT [expected PASS] subtest: Navigate a frame via form-submission with javascript:-urls w/ default policy in enforcing mode.

      Test timed out

@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 18, 2025

✨ Try run (#19464504204) succeeded.

@yezhizhen
Copy link
Copy Markdown
Member

yezhizhen commented Nov 18, 2025

Oh you may have a point here. It seems to pass in CI.

@yezhizhen yezhizhen added the T-linux-wpt Do a try run of the WPT label Nov 18, 2025
@github-actions github-actions bot removed the T-linux-wpt Do a try run of the WPT label Nov 18, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 18, 2025

🔨 Triggering try run (#19465423278) for Linux (WPT)

@WaterWhisperer
Copy link
Copy Markdown
Contributor Author

WaterWhisperer commented Nov 18, 2025 •
edited
Loading

@yezhizhen Thank you for your detailed explanation! However, I still have some doubts because this result was obtained after I removed event_timeupdate_noautoplay.html.ini. If this means it meets expectations, doesn't it mean that all 100 times were "OK"?
Thanks again for the diligent follow-up.

@yezhizhen yezhizhen changed the title tests: Remove outdated test expectation file for event_timeupdate_noautoplay.html tests: Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS Nov 18, 2025
@yezhizhen yezhizhen changed the title tests: Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS Nov 18, 2025
@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 18, 2025

Test results for linux-wpt from try job (#19465423278):

Flaky unexpected result (43)
  • OK /IndexedDB/idbfactory_open.any.html
    • FAIL [expected PASS] subtest: Calling open() with version argument 1.5 should not throw.

      assert_equals: version expected 1 but got 9007199254740991

  • OK /_mozilla/mozilla/getBoundingClientRect.html (#39668)
    • FAIL [expected PASS] subtest: getBoundingClientRect 1

      assert_equals: expected 62 but got 60.35

  • OK /_mozilla/webxr/create_session.https.html
    • FAIL [expected PASS] subtest: create_session

      can't access property "simulateDeviceConnection", navigator.xr.test is undefined

  • ERROR [expected TIMEOUT] /_mozilla/webxr/sessionavailable.https.html
  • CRASH [expected OK] /_webgl/conformance/ogles/GL/swizzlers/swizzlers_057_to_064.html
  • CRASH [expected ERROR] /_webgl/conformance/textures/canvas/tex-2d-rgb-rgb-unsigned_short_5_6_5.html
  • CRASH [expected ERROR] /_webgl/conformance/textures/canvas/tex-2d-rgba-rgba-unsigned_byte.html
  • CRASH [expected OK] /_webgl/conformance/textures/image_bitmap_from_blob/tex-2d-alpha-alpha-unsigned_byte.html
  • CRASH [expected OK] /_webgl/conformance/textures/image_bitmap_from_image_data/tex-2d-alpha-alpha-unsigned_byte.html
  • CRASH [expected OK] /_webgl/conformance/textures/image_bitmap_from_image_data/tex-2d-rgba-rgba-unsigned_byte.html
  • CRASH [expected OK] /_webgl/conformance/textures/misc/origin-clean-conformance.html (#24529)
  • OK /_webgl/conformance/textures/misc/texture-upload-size.html (#21770)
    • PASS [expected FAIL] subtest: WebGL test #45
    • PASS [expected FAIL] subtest: WebGL test #47
    • PASS [expected FAIL] subtest: WebGL test #49
    • PASS [expected FAIL] subtest: WebGL test #51
    • FAIL [expected PASS] subtest: WebGL test #53

      assert_true: Texture was smaller than the expected size 2x2 expected true got false

    • FAIL [expected PASS] subtest: WebGL test #55

      assert_true: getError expected: INVALID_VALUE. Was NO_ERROR : when calling texSubImage2D with the same texture upload with offset 1, 1 expected true got false

    • FAIL [expected PASS] subtest: WebGL test #57

      assert_true: Texture was smaller than the expected size 2x2 expected true got false

    • FAIL [expected PASS] subtest: WebGL test #59

      assert_true: getError expected: INVALID_VALUE. Was NO_ERROR : when calling texSubImage2D with the same texture upload with offset 1, 1 expected true got false

    • PASS [expected FAIL] subtest: WebGL test #61
    • PASS [expected FAIL] subtest: WebGL test #63
    • And 26 more unexpected results...
  • CRASH [expected ERROR] /_webgl/conformance2/textures/misc/tex-input-validation.html (#38890)
  • CRASH [expected OK] /_webgl/conformance2/wasm/buffersubdata-16gb-wasm-memory.html
  • OK [expected TIMEOUT] /cookiestore/httponly_cookies.https.window.html (#40697)
    • FAIL [expected TIMEOUT] subtest: HttpOnly cookies are not observed

      assert_equals: HttpOnly cookie we wrote using HTTP in cookie jar is invisible to script expected (undefined) undefined but got (string) "cspViolationReportCookie1=1"

    • FAIL [expected NOTRUN] subtest: HttpOnly cookies can not be set by document.cookie

      assert_equals: Trying to store an HttpOnly cookie with document.cookie fails expected "cookie1=value1; cookie3=value3" but got "HTTPONLY-cookie=value; cookie1=value1; cookie3=value3"

    • FAIL [expected NOTRUN] subtest: HttpOnly cookies can not be set by CookieStore

      assert_equals: httpOnly is not an option for CookieStore.set() expected "cookie1=value1; cookie2=value2; cookie3=value3" but got "HTTPONLY-cookie=value; cookie1=value1; cookie2=value2; cookie3=value3"

    • FAIL [expected NOTRUN] subtest: HttpOnly cookies are not deleted/overwritten

      assert_equals: HttpOnly cookie is not deleted expected (string) "HTTPONLY-cookie=value" but got (undefined) undefined

  • FAIL [expected PASS] /css/css-backgrounds/background-size-042.html
  • OK /css/css-fonts/generic-family-keywords-003.html (#38994)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted generic(nastaliq) (drawing text in a canvas)
    • PASS [expected FAIL] subtest: @font-face matching for quoted and unquoted ui-sans-serif (drawing text in a canvas)
  • OK /custom-elements/form-associated/ElementInternals-setFormValue.html (#29174)
    • PASS [expected FAIL] subtest: Multiple values - name content attribute is ignored
  • CRASH [expected ERROR] /dom/abort/AbortSignal.https.any.shadowrealm-in-audioworklet.html (#38763)
  • CRASH [expected ERROR] /encoding/streams/decode-ignore-bom.any.serviceworker.html
  • TIMEOUT [expected OK] /fetch/api/redirect/redirect-keepalive.https.any.html (#32153)
    • TIMEOUT [expected PASS] subtest: [keepalive][iframe][load] mixed content redirect; setting up

      Test timed out

  • OK /fetch/fetch-later/send-on-discard/not-send-after-abort.https.window.html (#40696)
    • FAIL [expected PASS] subtest: A discarded document does not send an already aborted fetchLater request.

      assert_equals: Number of sent beacons does not match expected count: expected 1 but got 0

  • OK /fetch/metadata/generated/css-font-face.https.sub.tentative.html (#32732)
    • PASS [expected FAIL] subtest: sec-fetch-dest
  • TIMEOUT /fetch/metadata/generated/css-images.sub.tentative.html (#29047)
    • TIMEOUT [expected PASS] subtest: background-image sec-fetch-site - Not sent to non-trustworthy same-origin destination

      Test timed out

  • OK [expected ERROR] /focus/focus-event-after-switching-iframes.sub.html (#40368)
  • OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-same-origin-fragment.html (#20768)
    • FAIL [expected PASS] subtest: Tests that a fragment navigation in the unload handler will not block the initial navigation

      assert_equals: expected "" but got "#fragment"

  • CRASH [expected OK] /html/browsers/the-window-object/open-close/open_initial_size.html
  • OK /html/browsers/windows/embedded-opener-remove-frame.html (#23867)
    • FAIL [expected PASS] subtest: opener of discarded auxiliary browsing context

      assert_object_equals: property "get" expected function "function opener() {
    [native code]
}" got function "function opener() {
    [native code]
}"

  • TIMEOUT [expected OK] /html/interaction/focus/the-autofocus-attribute/update-the-rendering.html (#24145)
    • TIMEOUT [expected FAIL] subtest: "Flush autofocus candidates" should be happen before a scroll event and animation frame callbacks

      Test timed out

  • OK /html/semantics/document-metadata/the-meta-element/pragma-directives/attr-meta-http-equiv-refresh/allow-scripts-flag-changing-2.html (#39703)
    • FAIL [expected PASS] subtest: Meta refresh of the original iframe is not blocked if moved into a sandboxed iframe

      uncaught exception: Error: assert_unreached: The iframe into which the meta was moved must not refresh Reached unreachable code

  • OK /html/semantics/forms/form-submission-0/multipart-formdata.window.html (#28725)
    • PASS [expected FAIL] subtest: multipart/form-data: non-ASCII in name and value (formdata event)
  • OK /html/semantics/forms/form-submission-0/text-plain.window.html (#28687)
    • FAIL [expected PASS] subtest: text/plain: Basic File test (normal form)

      assert_equals: expected "basic=file-test.txt\r\n" but got ""

    • PASS [expected FAIL] subtest: text/plain: Basic File test (formdata event)
    • PASS [expected FAIL] subtest: text/plain: 0x00 in value (formdata event)
  • OK /html/semantics/forms/form-submission-0/urlencoded2.window.html (#28687)
    • FAIL [expected PASS] subtest: application/x-www-form-urlencoded: Basic test (formdata event)

      assert_equals: expected "basic=test" but got ""

  • OK [expected CRASH] /html/semantics/forms/the-fieldset-element/disabled-003.html (#31730, #39631)
  • OK /preload/preload-xhr.html (#39092)
    • FAIL [expected PASS] subtest: Make an XHR request immediately after creating link rel=preload.

      assert_equals: resources/dummy.xml?token=d20281b9-b513-4da8-8944-e0041ff640b7 expected 1 but got 0

  • ERROR [expected OK] /resource-timing/cors-preflight.any.html (#28694)
  • CRASH [expected OK] /trusted-types/Node-multiple-arguments.html
  • CRASH [expected OK] /trusted-types/eval-function-constructor.html
  • TIMEOUT /trusted-types/trusted-types-navigation.html?06-10 (#37920)
    • PASS [expected FAIL] subtest: Navigate a frame via anchor with javascript:-urls in report-only mode.
  • TIMEOUT /trusted-types/trusted-types-navigation.html?31-35 (#38034)
    • TIMEOUT [expected PASS] subtest: Navigate a frame via form-submission with javascript:-urls in report-only mode.

      Test timed out

    • NOTRUN [expected TIMEOUT] subtest: Navigate a frame via form-submission with javascript:-urls w/ default policy in report-only mode.
  • CRASH [expected OK] /uievents/idlharness.window.html
  • CRASH [expected OK] /workers/SharedWorker-script-error.html
  • ERROR [expected OK] /workers/baseurl/alpha/sharedworker-in-worker.html (#21315)
Stable unexpected results that are known to be intermittent (30)
  • OK /IndexedDB/idbobjectstore_getAll.any.html (#39276)
    • PASS [expected FAIL] subtest: Get all values with transaction.commit()
  • OK /IndexedDB/idbobjectstore_getAll.any.worker.html (#39400)
    • PASS [expected FAIL] subtest: Get all values with transaction.commit()
  • FAIL [expected PASS] /_mozilla/mozilla/sslfail.html (#10760)
  • TIMEOUT [expected OK] /_mozilla/mozilla/window_resize_event.html (#36741)
    • TIMEOUT [expected PASS] subtest: Popup onresize event fires after resizeTo

      Test timed out

  • OK /content-security-policy/frame-ancestors/frame-ancestors-path-ignored.window.html (#36468)
    • FAIL [expected PASS] subtest: A 'frame-ancestors' CSP directive with a URL that includes a path should be ignored.

      assert_unreached: The IFrame should have been blocked (or cross-origin). It wasn't. Reached unreachable code

  • TIMEOUT /content-security-policy/inheritance/location-reload.html (#38983)
    • PASS [expected FAIL] subtest: location.reload() of empty iframe.
  • OK /css/css-animations/event-order.tentative.html (#39000)
    • PASS [expected FAIL] subtest: Same events on pseudo-elements follow the prescribed order
  • OK /css/css-cascade/layer-cssom-order-reverse.html (#36094)
    • PASS [expected FAIL] subtest: Delete layer invalidates @font-face
  • OK /css/css-fonts/generic-family-keywords-001.html (#37467)
    • FAIL [expected PASS] subtest: @font-face matching for quoted and unquoted generic(khmer-mul)

      assert_equals: quoted generic(khmer-mul) matches  @font-face rule expected 50 but got 30

    • FAIL [expected PASS] subtest: @font-face matching for quoted and unquoted generic(nastaliq)

      assert_equals: quoted generic(nastaliq) matches  @font-face rule expected 50 but got 30

  • OK /fetch/fetch-later/permissions-policy/deferred-fetch-allowed-by-permissions-policy.https.window.html (#40478)
    • FAIL [expected PASS] subtest: Permissions policy header: "deferred-fetch=*" allows fetchLater() in the top-level document.

      assert_equals: Number of sent beacons does not match expected count: expected 1 but got 0

  • OK /fetch/metadata/generated/css-font-face.sub.tentative.html (#34624)
    • PASS [expected FAIL] subtest: sec-fetch-storage-access - Not sent to non-trustworthy same-site destination
    • FAIL [expected PASS] subtest: sec-fetch-storage-access - Not sent to non-trustworthy cross-site destination

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

  • OK /fetch/metadata/generated/element-img-environment-change.https.sub.html (#30111)
    • FAIL [expected PASS] subtest: sec-fetch-site - Same-Origin -> Same-Site -> Same-Origin redirect, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Cross-Site -> Same Origin, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Cross-Site -> Same-Site, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Cross-Site -> Cross-Site, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Same-Origin -> Same Origin, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Same-Origin -> Same-Site, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Same-Origin -> Cross-Site, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Same-Site -> Same Origin, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-site - Same-Site -> Same-Site, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-site - Same-Site -> Cross-Site, no attributes
    • And 2 more unexpected results...
  • OK /fetch/metadata/generated/element-img-environment-change.sub.html (#30111)
    • PASS [expected FAIL] subtest: sec-fetch-site - Not sent to non-trustworthy cross-site destination, no attributes
    • PASS [expected FAIL] subtest: sec-fetch-mode - Not sent to non-trustworthy same-origin destination, no attributes
    • PASS [expected FAIL] subtest: sec-fetch-mode - Not sent to non-trustworthy cross-site destination, no attributes
    • PASS [expected FAIL] subtest: sec-fetch-dest - Not sent to non-trustworthy cross-site destination, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-user - Not sent to non-trustworthy same-origin destination, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

    • PASS [expected FAIL] subtest: sec-fetch-user - Not sent to non-trustworthy same-site destination, no attributes
    • PASS [expected FAIL] subtest: sec-fetch-user - Not sent to non-trustworthy cross-site destination, no attributes
    • PASS [expected FAIL] subtest: sec-fetch-storage-access - Not sent to non-trustworthy same-origin destination, no attributes
    • FAIL [expected PASS] subtest: sec-fetch-storage-access - Not sent to non-trustworthy cross-site destination, no attributes

      promise_test: Unhandled rejection with value: object "Error: Failed to query for recorded headers."

  • OK /html/browsers/browsing-the-web/navigating-across-documents/005.html (#27062)
    • FAIL [expected PASS] subtest: Link with onclick navigation and href navigation 

      assert_equals: expected "href" but got "click"

  • OK /html/browsers/browsing-the-web/navigating-across-documents/navigation-unload-cross-origin.sub.window.html (#29056)
    • FAIL [expected PASS] subtest: Cross-origin navigation started from unload handler must be ignored

      promise_test: Unhandled rejection with value: object "SecurityError: The operation is insecure."

  • OK /html/browsers/history/the-history-interface/traverse_the_history_3.html (#21383)
    • PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
  • OK /html/browsers/history/the-history-interface/traverse_the_history_4.html (#21383)
    • PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
  • OK /html/browsers/history/the-history-interface/traverse_the_history_5.html (#21383)
    • PASS [expected FAIL] subtest: Multiple history traversals, last would be aborted
  • ERROR [expected OK] /html/infrastructure/common-dom-interfaces/collections/domstringlist.html (#40665)
  • OK [expected TIMEOUT] /html/interaction/focus/the-autofocus-attribute/autofocus-dialog.html (#29087)
    • FAIL [expected TIMEOUT] subtest: <dialog>-contained autofocus element gets focused when the dialog is shown

      assert_equals: expected "DIV" but got "BODY"

  • ERROR [expected OK] /html/semantics/embedded-content/media-elements/event_timeupdate_noautoplay.html (#25046)
  • OK /html/semantics/embedded-content/the-iframe-element/iframe-loading-lazy-nav-window-open.html (#32596)
    • FAIL [expected PASS] subtest: Navigating iframe loading='lazy' before it is loaded: location.replace

      uncaught exception: Error: assert_equals: expected "http://web-platform.test:8000/html/semantics/embedded-content/the-iframe-element/support/blank.htm?nav" but got "http://web-platform.test:8000/html/semantics/embedded-content/the-iframe-element/support/blank.htm?src"

  • OK [expected ERROR] /html/semantics/forms/the-input-element/click-user-gesture.html (#40512)
  • OK /navigation-timing/test-navigation-type-reload.html (#33334)
    • PASS [expected FAIL] subtest: Reload domContentLoadedEventEnd > Original domContentLoadedEventEnd
    • PASS [expected FAIL] subtest: Reload domInteractive > Original domInteractive
  • CRASH [expected OK] /pointerevents/compat/pointerevent_touch-action_two-finger_interaction.html (#40418)
  • OK /preload/preload-error.sub.html (#37177)
    • PASS [expected FAIL] subtest: 404 (fetch): main
    • PASS [expected FAIL] subtest: CORS (fetch): main
  • OK /reporting/same-origin-same-site-credentials.https.sub.html (#40479)
    • FAIL [expected PASS] subtest: Reporting endpoints received credentials.

      assert_equals: No additional cookies were received expected 4 but got 5

  • TIMEOUT [expected OK] /trusted-types/trusted-types-navigation.html?01-05 (#38975)
    • TIMEOUT [expected PASS] subtest: Navigate a window via anchor with javascript:-urls in report-only mode.

      Test timed out

    • NOTRUN [expected PASS] subtest: Navigate a window via anchor with javascript:-urls w/ default policy in report-only mode.
    • NOTRUN [expected PASS] subtest: Navigate a frame via anchor with javascript:-urls in enforcing mode.
  • OK /trusted-types/trusted-types-navigation.html?26-30 (#38807)
    • PASS [expected FAIL] subtest: Navigate a window via form-submission with javascript:-urls in report-only mode.
    • PASS [expected FAIL] subtest: Navigate a frame via form-submission with javascript:-urls in enforcing mode.
  • OK [expected ERROR] /webxr/render_state_update.https.html (#27535)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Nov 18, 2025

✨ Try run (#19465423278) succeeded.

@yezhizhen yezhizhen changed the title Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS Update expectation for media-elements/event_timeupdate_noautoplay.html to PASS by removing .ini Nov 18, 2025
yezhizhen
yezhizhen approved these changes Nov 18, 2025
View reviewed changes
Copy link
Copy Markdown
Member

@yezhizhen yezhizhen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@yezhizhen yezhizhen added this pull request to the merge queue Nov 18, 2025
@servo-highfive servo-highfive added S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. and removed S-needs-code-changes Changes have not yet been made that were requested by a reviewer. labels Nov 18, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 18, 2025
@servo-highfive servo-highfive removed the S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. label Nov 18, 2025
@servo-highfive servo-highfive added the S-tests-failed The changes caused existing tests to fail. label Nov 18, 2025
@yezhizhen yezhizhen added this pull request to the merge queue Nov 18, 2025
@servo-highfive servo-highfive added S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. and removed S-tests-failed The changes caused existing tests to fail. labels Nov 18, 2025
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 18, 2025
@servo-highfive servo-highfive added S-tests-failed The changes caused existing tests to fail. and removed S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. labels Nov 18, 2025
@jschwe jschwe added this pull request to the merge queue Nov 18, 2025
@servo-highfive servo-highfive added S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. and removed S-tests-failed The changes caused existing tests to fail. labels Nov 18, 2025
Merged via the queue into servo:main with commit 645947b Nov 18, 2025
129 checks passed
@servo-highfive servo-highfive removed the S-awaiting-merge The PR is in the process of compiling and running tests on the automated CI. label Nov 18, 2025
@WaterWhisperer WaterWhisperer deleted the tests branch November 19, 2025 04:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yezhizhen yezhizhen yezhizhen approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Intermittent fail in /html/semantics/embedded-content/media-elements/event_timeupdate_noautoplay.html

5 participants

@WaterWhisperer @mrobinson @yezhizhen @jschwe @servo-highfive