Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[e2e tests] Fixes to support WP core 6.8 #56121

Merged
merged 19 commits into from
Mar 6, 2025

Conversation

adimoldovan
Copy link
Contributor

@adimoldovan adimoldovan commented Mar 5, 2025

Changes proposed in this Pull Request:

Updates to e2e checks to support WordPress 6.8

This should fix the following tests failing with 6.8 (28 tests in total):

- cart/cart.spec.js
- checkout/checkout.spec.js
- coupons/cart-block-coupons.spec.js
- coupons/checkout-block-coupons.spec.js
- editor/create-woocommerce-blocks.spec.js
- editor/create-woocommerce-patterns.spec.js
- customize-store/assembler/footer.spec.js Assembler -> Footers › Picking a footer should trigger an update on the site preview
- customize-store/assembler/header.spec.js › Assembler -> headers › Picking a header should trigger an update on the site preview
- customize-store/assembler/logo-picker/logo-picker.spec.js > Assembler -> Logo Picker › Selecting an image should update the site preview
- product/product-tags-attributes.spec.js › Browse product tags and attributes from the product page › can see products showcase
- user/users-create.spec.js
- wp-core/create-page.spec.js

How to test the changes in this Pull Request:

Build the utils package:

pnpm --filter="@woocommerce/e2e-utils-playwright" build

Run the tests:

pnpm test:e2e cart.spec checkout.spec cart-block-coupons.spec checkout-block-coupons.spec create-woocommerce-blocks create-woocommerce-patterns footer header logo-picker product-tags-attributes users-create create-page

All tests need to pass with:

  • WP 6.7.2
  • WP 6.7.2 + Gutenberg stable
  • WP 6.7.2 + Gutenberg nightly
  • WP 6.8-beta1
  • WP 6.8-beta1 + Gutenberg stable
  • WP 6.8-beta1 + Gutenberg nightly

Testing that has already taken place:

Tested locally:

  • ✅ WP 6.7.2
  • ✅ WP 6.7.2 + Gutenberg stable
  • ✅ WP 6.7.2 + Gutenberg nightly
  • ✅ WP 6.8-beta1
  • ✅ WP 6.8-beta1 + Gutenberg stable
  • ✅ WP 6.8-beta1 + Gutenberg nightly

In CI:

  • ✅ WP 6.7.2
  • ✅ WP 6.7.2 + Gutenberg stable
  • ✅ WP 6.7.2 + Gutenberg nightly
  • ✅ WP 6.8-beta1
  • ✅ WP 6.6.2

@github-actions github-actions bot added focus: e2e tests Issues related to e2e tests package: @woocommerce/e2e-utils-playwright Issues related to @woocommerce/e2e-utils-playwright package plugin: woocommerce Issues related to the WooCommerce Core plugin. labels Mar 5, 2025
@adimoldovan adimoldovan changed the title [e2e tests] Fixes for WP core 6.8 [e2e tests] Fixes to support WP core 6.8 Mar 5, 2025
@adimoldovan adimoldovan self-assigned this Mar 5, 2025
@adimoldovan adimoldovan marked this pull request as ready for review March 6, 2025 09:01
@adimoldovan adimoldovan requested a review from a team as a code owner March 6, 2025 09:01
Copy link
Contributor

github-actions bot commented Mar 6, 2025

Testing Guidelines

Hi @woocommerce/solaris,

Apart from reviewing the code changes, please make sure to review the testing instructions (Guide) and verify that relevant tests (E2E, Unit, Integration, etc.) have been added or updated as needed.

Reminder: PR reviewers are required to document testing performed. This includes:

  • 🖼️ Screenshots or screen recordings.
  • 📝 List of functionality tested / steps followed.
  • 🌐 Site details (environment attributes such as hosting type, plugins, theme, store size, store age, and relevant settings).
  • 🔍 Any analysis performed, such as assessing potential impacts on environment attributes and other plugins, conducting performance profiling, or using LLM/AI-based analysis.

⚠️ Within the testing details you provide, please ensure that no sensitive information (such as API keys, passwords, user data, etc.) is included in this public issue.

Copy link
Contributor

github-actions bot commented Mar 6, 2025

Test using WordPress Playground

The changes in this pull request can be previewed and tested using a WordPress Playground instance.
WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser.

Test this pull request with WordPress Playground.

Note that this URL is valid for 30 days from when this comment was last updated. You can update it by closing/reopening the PR or pushing a new commit.

@rodelgc
Copy link
Contributor

rodelgc commented Mar 6, 2025

Thank you so much for working on this @adimoldovan. The tests passed on my end as well with the following WP and Gutenberg combinations:

WP 6.8 beta ✅
WP 6.8 beta & GB stable ✅
WP 6.8 beta & GB nightly ✅

Tested it locally by running:

export WP_ENV_CORE="https://wordpress.org/wordpress-6.8-beta1.zip"

# commands to build and launch env

pnpm test:e2e cart/cart.spec.js \
 checkout/checkout.spec.js \
 coupons/cart-block-coupons.spec.js \
 coupons/checkout-block-coupons.spec.js \
 editor/create-woocommerce-blocks.spec.js \
 editor/create-woocommerce-patterns.spec.js \
 user/users-create.spec.js \
 wp-core/create-page.spec.js

pnpm test:e2e -g "Picking a footer should trigger an update on the site preview"
pnpm test:e2e -g "Picking a header should trigger an update on the site preview"
pnpm test:e2e -g "Selecting an image should update the site preview"
pnpm test:e2e -g "can see products showcase"
Screenshot 2025-03-06 at 7 01 42 PM Screenshot 2025-03-06 at 7 06 00 PM Screenshot 2025-03-06 at 7 05 05 PM Screenshot 2025-03-06 at 7 04 19 PM Screenshot 2025-03-06 at 7 03 34 PM

I noticed that the test guest can checkout paying with cash on delivery on classic checkout always fails on the first run. It only passes on a retry. I noticed it happening both in CI and locally.

Screenshot 2025-03-06 at 6 56 55 PM

After looking at other CI runs from trunk it seems like this behavior is also happening there so I don't think this minor flakiness was introduced in this PR. Approving it.

@adimoldovan
Copy link
Contributor Author

I noticed that the test guest can checkout paying with cash on delivery on classic checkout always fails on the first run. It only passes on a retry. I noticed it happening both in CI and locally.

@rodelgc yes, I noticed that too. It's on my radar.
Thanks for reviewing.

@adimoldovan adimoldovan merged commit d0e9ef8 into trunk Mar 6, 2025
78 checks passed
@adimoldovan adimoldovan deleted the e2e/fix-tests-with-wpcore-6.8 branch March 6, 2025 11:39
@github-actions github-actions bot added this to the 9.9.0 milestone Mar 6, 2025
@github-actions github-actions bot added the needs: analysis Indicates if the PR requires a PR testing scrub session. label Mar 6, 2025
adimoldovan added a commit that referenced this pull request Mar 7, 2025
* [e2e-utils] Update editor-canvas locator to support updates in Gutenberg nightly (#56083)

* [e2e tests] Fixes to support WP core 6.8 (#56121)
jorgeatorres added a commit to jorgeatorres/woocommerce that referenced this pull request Mar 7, 2025
* Prep release/9.8 for 9.8 cycle (woocommerce#56031)

Prep release/9.8 for 9.8 cycle with version bump to 9.8.0-beta.1

Co-authored-by: WooCommerce Bot <no-reply@woocommerce.com>

* Release: Prepare the changelog for 9.8 (woocommerce#56035)

* Delete changelog files from 9.8 release

* Update the readme files for the 9.8 release

* Update release date

---------

Co-authored-by: WooCommerce Bot <no-reply@woocommerce.com>
Co-authored-by: Jorge Torres <jorge.torres@automattic.com>

* Cherry pick 56047 into frozen release: release/9.8 (woocommerce#56057)

Fix combine 415 403 (woocommerce#56047)

* Secure “Generate coupon code” button

* Secure “Export products” button

* Secure “Import products” button

* Secure inline error messages

* Secure link inside inline error messages

* Remove “onChangeShippingMethodSelector”

* Add changelog message

* Secure “Read more” button

* Added changelog.

* Added element creation to avoid appending text to jQuery Object.

---------

Co-authored-by: Alba Rincón <albarin@users.noreply.github.com>
Co-authored-by: Niels Lange <info@nielslange.de>
Co-authored-by: Paulo Arromba <17236129+wavvves@users.noreply.github.com>

* Cherry pick 56048 into frozen release: release/9.8 (woocommerce#56060)

Remove decodeEntities from the product name used on cart & checkout (woocommerce#56048)

revert using RawHTML because it renders an extra div inside the <a> tag

Co-authored-by: Alba Rincón <albarin@users.noreply.github.com>
Co-authored-by: Alex Florisca <alex.florisca@automattic.com>

* Prevent Fatal Error on malformed pagination query (woocommerce#56049)

* Avoid fatal by casting page number to integer

Cast page number to integer when generating aria-label to ensure proper number formatting

* Add changefile(s) from automation for the following project(s): woocommerce

---------

Co-authored-by: github-actions <github-actions@github.com>

* Fix email accent color causing invisible text in emails (woocommerce#56061)

* Use theme button background color as accent color in emails

* Add changelog

* Update tests

* Fix textdomain loading warning in woocommerce 9 80 beta 1 (woocommerce#56133)

* Refactor Email Editor Integration Initialization from using FeaturesUtil to `get_option`

Using FeaturesUtil::feature_is_enabled here triggers a Translation loading early warning message on WordPress 6.8 Beta

* Add change log file

* Delete Transients: avoid running statements without options_to_clear. (woocommerce#56136)

* Delete Transients: avoid running statements without options_to_clear.

* Add changefile(s) from automation for the following project(s): woocommerce

* Delete Transients: adds test case.

* Fixes spelling.

* Fixes _wc_delete_transients call.

* Linting.

* Fix linter white space

* Returns early.

---------

Co-authored-by: cpap <papazoglou.charalampos@gmail.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Tom Cafferkey <tjcafferkey@gmail.com>

* Cherry pick e2e test fixes 56083 and 56121 to 9.8 (woocommerce#56163)

* [e2e-utils] Update editor-canvas locator to support updates in Gutenberg nightly (woocommerce#56083)

* [e2e tests] Fixes to support WP core 6.8 (woocommerce#56121)

---------

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: WooCommerce Bot <no-reply@woocommerce.com>
Co-authored-by: Alba Rincón <albarin@users.noreply.github.com>
Co-authored-by: Niels Lange <info@nielslange.de>
Co-authored-by: Paulo Arromba <17236129+wavvves@users.noreply.github.com>
Co-authored-by: Alex Florisca <alex.florisca@automattic.com>
Co-authored-by: Luigi Teschio <gigitux@gmail.com>
Co-authored-by: github-actions <github-actions@github.com>
Co-authored-by: Ján Mikláš <neosinner@gmail.com>
Co-authored-by: Oluwaseun Olorunsola <30554163+triple0t@users.noreply.github.com>
Co-authored-by: Harris Papazoglou <hrrsppzgl@gmail.com>
Co-authored-by: cpap <papazoglou.charalampos@gmail.com>
Co-authored-by: Tom Cafferkey <tjcafferkey@gmail.com>
Co-authored-by: Adrian Moldovan <3854374+adimoldovan@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
focus: e2e tests Issues related to e2e tests needs: analysis Indicates if the PR requires a PR testing scrub session. package: @woocommerce/e2e-utils-playwright Issues related to @woocommerce/e2e-utils-playwright package plugin: woocommerce Issues related to the WooCommerce Core plugin.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants