Skip to content

Fix payments settings banner experiment logic #33549

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ilyasfoo merged 2 commits into from
Jun 22, 2022
Merged

Fix payments settings banner experiment logic #33549

ilyasfoo merged 2 commits into from
Jun 22, 2022

Conversation

@ilyasfoo
Copy link
Contributor

@ilyasfoo ilyasfoo commented Jun 22, 2022

Changes proposed in this Pull Request:

Previously, the payments settings banner experiment was called before the logic for eligibility is executed. This means that all users who visited the payments settings screen were participants of the experiment, which will skew the experiment results since most of them would be in treatment but they don't experience the banner.

This fix ensures only eligible users can participate the experiment.

How to test the changes in this Pull Request:

  1. Create a site where WooCommerce Payments isn't installed but is in a supported country (such as US)
  2. Use the treatment bookmarklet obtained from Abacus for woocommerce_payments_settings_banner_2022_06 experiment
  3. Open browser's console and go to application cache
  4. Delete woocommerce_payments_settings_banner_2022_06 experiment from the cache
  5. Open the Networks tab and filter for the keyword woocommerce_payments_settings_banner_2022_06
  6. Navigate to WooCommerce > Settings > Payment
  7. Make sure no experiment is requested in the networks tab
  8. Install & enable WooCommerce Payments plugin
  9. Navigate to WooCommerce > Settings > Payment
  10. Observe the experiment is requested and the banner is shown

Other information:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you created a changelog file for each project being changed, ie pnpm changelog add --filter=<project>?

FOR PR REVIEWER ONLY:

  • I have reviewed that everything is sanitized/escaped appropriately for any SQL or XSS injection possibilities. I made sure Linting is not ignored or disabled.

@github-actions github-actions bot added focus: react admin plugin: woocommerce Issues related to the WooCommerce Core plugin. labels Jun 22, 2022
@botwoo
Copy link
Collaborator

botwoo commented Jun 22, 2022

📊 Test reports for this pull request have been published and are accessible through the following links:

Latest commit referenced in the reports: Changelog f6b6ca9
This comment will automatically be updated with the latest referenced commit when you push new changes to this pull request.

Visit the WooCommerce Test Reports homepage to view all published reports. See the FAQs page if you're having problems accessing them.

rjchow
rjchow approved these changes Jun 22, 2022
View reviewed changes
Copy link
Contributor

@rjchow rjchow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Sharp eyes catching this!

There's a bit of a catch for testers who will test this with test helper plugin - the experiment won't show up in the experiments list until all the other conditions are fulfilled and the settings page has been visited once

@ilyasfoo ilyasfoo added this to the 6.7.0 milestone Jun 22, 2022
@ilyasfoo ilyasfoo added the release: cherry-pick Commits from this PR also needs to be added to current release branch. label Jun 22, 2022
@ilyasfoo ilyasfoo merged commit 9c4c383 into trunk Jun 22, 2022
@ilyasfoo ilyasfoo deleted the fix/payments-settings-banner-experiment-logic branch June 22, 2022 08:45
@github-actions
Copy link
Contributor

github-actions bot commented Jun 22, 2022

Hi @ilyasfoo, thanks for merging this pull request. Please take a look at these follow-up tasks you may need to perform:

  • Add the release: add testing instructions label

@rjchow rjchow mentioned this pull request Jun 24, 2022
Merged
8 tasks
ObliviousHarmony added a commit that referenced this pull request Jun 28, 2022
@ObliviousHarmony
Fix payments settings banner experiment logic (#33549)
78b9298 
* Delegate payment experiment logic into its own hook and rearrange the checking logic to be before the experiment is called

* Changelog
@ObliviousHarmony ObliviousHarmony mentioned this pull request Jun 28, 2022
Merged
@ObliviousHarmony ObliviousHarmony removed the release: cherry-pick Commits from this PR also needs to be added to current release branch. label Jun 28, 2022
ObliviousHarmony added a commit that referenced this pull request Jun 28, 2022
@ObliviousHarmony @gigitux
@joshuatf @rjchow
Prepare WooCommerce 6.7 Beta.2 (#33642)
05fadd3 
* Update WooCommerce Blocks package to 7.8.3 (#33514)

* Move tracks _ui and _ut properties out of filtered tracks properties (#33621)

* Move tracks _ui and _ut properties out of filtered tracks properties

* Add tests around tracks properties

* Fix broken variable name

* Add changelog entry

* Fix payments settings banner experiment logic (#33549)

* Delegate payment experiment logic into its own hook and rearrange the checking logic to be before the experiment is called

* Changelog

* fix payments banner experiment test (#33589)

test needed to be updated as the component did not call useExperiment directly anymore

* Prepared Changelog

* Removed Changelog Files

* Updated Version

Co-authored-by: Luigi Teschio <[email protected]>
Co-authored-by: Joshua T Flowers <[email protected]>
Co-authored-by: RJ <[email protected]>
@github-actions github-actions bot mentioned this pull request Jul 13, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@rjchow rjchow rjchow approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

plugin: woocommerce Issues related to the WooCommerce Core plugin.

Projects

None yet

Milestone

6.7.0

Development

Successfully merging this pull request may close these issues.

5 participants

@ilyasfoo @botwoo @rjchow @ObliviousHarmony