Page MenuHomePhabricator

[EPIC] Logged-out user A/B test of Donation entry point improvements
Closed, ResolvedPublic

Description

NOTE: It's possible another epic ticket already exists, in which case we can close this one as a duplicate. I wanted something to thread the tickets I'm about to create under, but wasn't able to find a project/etc to put them under. In general, feel free to edit this as you see fit

Beginning in Q4 and carrying over into Q1, we will be conducting an A/B test using the new experimentation platform, as a means of determining the optimal donation entry points on both mobile and desktop, as well as helping to test the experimentation platform as the first "real" internal users

Requirements

  • an A/B test is run on a TBD population of anonymous users
  • on both mobile and desktop
  • with the primary metric measured being actual donations (quantity) and auxiliary metric being clickthrough rate of the entry point
  • the data of which is then analyzed to determine the "winning" groups on both platforms
  • after which the experimental code is removed to make these changes permanent

Design

I'm sure there are more up to date designs somewhere, but the most recent versions I could find are here: T388149

Measurement plan

https://docs.google.com/document/d/1zIVct6RC8hoSCX5yeqsvSI_aFSRARj1JGgwX3pkRC2Q/edit?usp=sharing

Instrumentation spec

https://docs.google.com/spreadsheets/d/1PLbiTF160-jkqqYMN9HBVyXsf5i0FatKyXsFZNO-cSc/edit?usp=sharing

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

@SToyofuku-WMF we tend to put epics in Web-Team-Roadmap these days. This also seems to be missing a project tag e.g. FY2024-25 KR 3.1 Content Discovery

Wasn't able to finish and I'm beginning to identify some pretty critical open questions, but we should at least have some tickets to consider for the sprint Wednesday. Will keep at this and get things estimated/more refined generally tomorrow 🤞

Tickets are created and scoped up to the putting it on beta/rollout steps, which will be dependent on our meeting this Monday - we should have some good work to pull into the sprint, though

ovasileva removed a project: Web-Team.