Skip to content

Conversation

@aprea
Copy link
Contributor

@aprea aprea commented May 11, 2022

All Submissions:

Changes proposed in this Pull Request:

Internal project: pdjTHR-Vi-p2

This PR adds code to:

  • Request assignment for the woocommerce_wcpay_subscriptions_page_202207_v1 experiment.
  • Handle the two experiment treatments, i.e.
    • Treatment A: Redirect the user to the to the “Add new product” page when they click the "Get started" button.
    • Treatment B: Redirect the user to the WCPay onboarding flow when they click the "Get started" button.

Additionally, each treatment has its own set of steps in the "You’re only steps away from selling subscriptions" section. These steps should accurately reflect their respective flow described above.

How to test the changes in this Pull Request:

Note: Because this PR requires interacting with ExPlat/Abacus it's much easier to test this PR in an environment that communicates with the production WPCOM servers. As such, these instructions are written specifically for testing on Jurassic Ninja or another non-local environment.

You can grab a zip of this PR for testing here: woocommerce.zip

  1. Plugin eligibility: Ensure the following plugins are not installed at all:
    • WooCommerce Subscriptions
    • WooCommerce Payments
    • Sumo Subscriptions
    • Yith Subscriptions
    • Subscripto
    • Subscriptions For WooCommerce
  2. Store location eligibility: For the menu item to be displayed your store needs to based in the US.
    • Go to WooCommerce > Settings and make sure your store is set to a US based address.
  3. Store age criteria: To view the subscription menu item your store will need to have been active for more than 6 months - we're testing this experiment on established stores. If your store doesn't meet this criteria, you can use the following code snippet:
    • update_option( 'woocommerce_admin_install_timestamp', strtotime( '-6 month' ) );
  4. Paid order criteria: Only stores with at least 1 order in the last 30 days are eligible. If your store doesn't have an order in the last 30 days. Either:
    1. Set any existing pending or failed order to processing or completed
    2. If you don't have any order at all, you can just manually create one in WooCommerce > Orders > Add new and set it to processing or completed status
    3. ⚠️ Note: this order sale query is cached and so you may need to delete the 'woocommerce-wcpay-subscriptions_recent_sales_eligibility' transient if this criteria failed previously.
  5. If you meet all the criteria above you should see a WooCommerce > Subscriptions (new) menu item.
  6. View the WooCommerce > Subscriptions (new) page.
    • The 'new' badge should disappear on subsequent page loads.

Testing the treatments

Download, install and activate the WooCommerce Admin Test Helper plugin.

Navigate to Tools → WCA Test Helper → Experiments. This page should contain a row for the "woocommerce_wcpay_subscriptions_page_202207_v1" experiment.

Here you can toggle between the control (treatment A) and the treatment (treatment B).

Treatment A

  1. Toggle to the control experience from Tools → WCA Test Helper → Experiments.
  2. Navigate to WooCommerce → Subscriptions.
  3. Observe the following step headers in the "You’re only steps away from selling subscriptions" section.
    1. Create a subscription
    2. Create and connect your account
    3. Provide a few business details
  4. Click the "Get started" button.
  5. The WooCommerce Payments plugin will be installed & activated and you will be redirected to the “Add new product” page.

Treatment B

  1. Toggle to the treatment experience from Tools → WCA Test Helper → Experiments.
  2. Navigate to WooCommerce → Subscriptions.
  3. Observe the following step headers in the "You’re only steps away from selling subscriptions" section.
    1. Create and connect your account
    2. Provide a few business details
    3. Create subscriptions
  4. Click the "Get started" button.
  5. The WooCommerce Payments plugin will be installed & activated and you will be redirected to the WCPay onboarding flow.

Other information:

  • Have you added an explanation of what your changes do and why you'd like us to include them?
  • Have you written new tests for your changes, as applicable?
  • Have you successfully run tests with your changes locally?
  • Have you created a changelog file for each project being changed, ie pnpm nx changelog <project>?

FOR PR REVIEWER ONLY:

  • I have reviewed that everything is sanitized/escaped appropriately for any SQL or XSS injection possibilities. I made sure Linting is not ignored or disabled.

@github-actions github-actions bot added focus: react admin plugin: woocommerce Issues related to the WooCommerce Core plugin. labels May 11, 2022
@aprea aprea changed the title Add basic experiment functionality. Add experiment functionality to the WooCommerce → Subscriptions page May 12, 2022
@aprea aprea requested review from Jinksi and james-allan May 12, 2022 05:44
@aprea aprea marked this pull request as ready for review May 12, 2022 05:45
@james-allan
Copy link
Contributor

james-allan commented May 13, 2022

Hey @aprea. I'm testing the Get started -> Create product flow and after creating a product I'm not getting the modal popup request to complete the KYC. The product is saved as a draft, the wcpay-subscription-saved-as-draft URL arg is present, also going to the WooCommerce > Subscriptions page shows the empty state KYC screen. 🤷

Have you had this occur by any chance?

There don't appear to be any JS errors either.

Btw I'm testing on https://courageous-warbler.jurassic.ninja/, let me know if you want creds.

@james-allan
Copy link
Contributor

I just replicated on a fresh JN site without this experiment flow so it's not likely to be related to this PR in any case.

@aprea
Copy link
Contributor Author

aprea commented May 13, 2022

Thanks @james-allan, I was also able to reproduce this on my JN site 😢 Seems like a regression in WCPay.

I'm looking into it now.

Copy link
Contributor

@Jinksi Jinksi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM @aprea!

I was able to get Abacus bookmarklets working in a local wp-env site and I can confirm that this code is reading the experiment audience cookie correctly.

I haven't tested the complete control flow, as completing onboarding relies on WCPay PR #4255 being released. I think it is safe to merge this PR into the feature branch since the remainder of the flow is handled by WCPay and is not affected by these changes.

Testing steps completed:
  • Create wp-env site on PR branch
  • Make store eligible for experiment
  • Subscriptions menu item appears with ‘new’ badge
  • Clicking Subscriptions initially took me to the Control flow

Control

  • Event wcadmin_wccore_subscriptions_empty_state_view fired.
  • Observe the Treatment A headers.
  • Click Get started.
    • Event wcadmin_wccore_subscriptions_empty_state_get_started_click fired.
    • WCPay installed.
    • Create a new subscription product page.
  • Click publish.
    • It saves a draft and the expected modal doesn’t appear. ⚠️
    • Issue with WCPay, fix in progress → WCPay PR #4255.

Treatment

  • Event wcadmin_wccore_subscriptions_empty_state_view fired.
  • Observe the Treatment B headers.
  • Click Get Started.
    • Event wcadmin_wccore_subscriptions_empty_state_get_started_click fired.
    • WCPay installed.
  • Complete Stripe Onboarding in Test mode.
  • Redirected to WooCommerce → Subscriptions.

@aprea aprea merged commit 1fa3060 into add/experiment-subscriptions-admin-menu-1 May 16, 2022
@aprea aprea deleted the add/experiment-subscriptions-admin-menu-experiment-framework branch May 16, 2022 12:31
@github-actions
Copy link
Contributor

Hi @aprea, thanks for merging this pull request. Please take a look at these follow-up tasks you may need to perform:

  • Add the release: add testing instructions label

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

plugin: woocommerce Issues related to the WooCommerce Core plugin.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants