Skip to content

feat(analytics): restore Umami on canonical www host + end-to-end checkout funnel events (#4931)#4934

Merged
koala73 merged 7 commits into
mainfrom
feat/funnel-instrumentation-4931
Jul 6, 2026
Merged

feat(analytics): restore Umami on canonical www host + end-to-end checkout funnel events (#4931)#4934
koala73 merged 7 commits into
mainfrom
feat/funnel-instrumentation-4931

Conversation

@koala73

@koala73 koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner

What

Closes #4931 — first executable bet (B2) from the 2026-07-06 revenue strategy review: make the conversion funnel measurable end-to-end.

The P1 hiding inside this feature

https://worldmonitor.app/ 301s to https://www.worldmonitor.app/, but UMAMI_DOMAINS listed only the apex. The deployed Umami tracker disables itself on an exact hostname mismatch (O&&!L.includes(h) in the minified tracker) — so every dashboard event on the canonical host has been silently dropped. This PR adds www.worldmonitor.app to the list (tech/finance/commodity stay excluded per the upstream Umami #4183 note in the source comment).

Funnel events (typed catalog)

  • checkout-start — fired centrally in startCheckout on the dashboard (surface:'dashboard', authed flag) and in the /pro checkout service (surface:'pro-page' / 'pro-resume' for the post-sign-in auto-resume). One call site per surface covers every upgrade CTA.
  • checkout-success / checkout-failed — fired from the checkout-return reconciliation in panel-layout.ts (source distinguishes url-return vs the legacy overlay-flag path).

Money pages get a tracker at all

  • pro-test/index.html (+ built public/pro/index.html) and pro-test/welcome.html now load Umami with nonce="wm-static-bootstrap" (satisfies the strict-dynamic CSP — same pattern as the Turnstile tag) and data-domains including www.
  • The 7 primary welcome CTAs carry data-umami-event="welcome-cta" + data-umami-event-target matching their existing ?ref=welcome-* markers.

Guardrails

tests/funnel-analytics-policy.test.mjs pins: www in the domains list, the three catalog events, all call sites, and the tracker tags (nonce + website-id + www) in both pro pages. The existing secondary-startup.test.mts domains pin is updated with a comment explaining why www must stay.

Verification

  • npx tsx --test tests/funnel-analytics-policy.test.mjs tests/secondary-startup.test.mts — 19/19 pass
  • npm run typecheck — clean
  • cd pro-test && npm run build — clean; grep-verified the built public/pro/*.html carry the tracker tag with nonce + www, and the built welcome JS bundle carries data-umami-event-target
  • scripts/generate-product-config.mjs regen — no-op on generated paths (freshness gate mirror)

Funnel after this lands

pageview → welcome-ctagate-hitcheckout-startcheckout-success, segmented by plan via the existing identifyUser. Purchase truth stays server-side (Convex webhooks); these events make the client-side funnel joinable in one tool.

⚠️ Do not auto-merge — Elie reviews manually.

https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

…ckout funnel events (#4931)

The conversion funnel was unmeasurable and partially dark:

- UMAMI_DOMAINS listed only the apex, but the apex 301s to
  www.worldmonitor.app and the tracker's data-domains check is an EXACT
  hostname match -- every dashboard event on the canonical host was
  silently dropped. www is now listed (tech/finance/commodity stay
  excluded per the upstream Umami #4183 note).
- New typed funnel events: checkout-start (fired from startCheckout on
  the dashboard and from the /pro checkout service, with surface +
  authed props), checkout-success / checkout-failed (fired from the
  checkout-return reconciliation in panel-layout).
- The /pro SPA and welcome landing now load the tracker (static
  wm-static-bootstrap nonce satisfies the strict-dynamic CSP) and the
  primary welcome CTAs carry data-umami-event attributes matching their
  existing ?ref= markers.
- public/pro rebuilt; funnel-analytics-policy test guards the domains
  list, catalog entries, call sites, and tracker tags against silent
  removal.

Closes #4931

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
@vercel

vercel Bot commented Jul 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Jul 6, 2026 11:19am

Request Review

@greptile-apps

greptile-apps Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR resolves a silent analytics outage on the canonical www.worldmonitor.app host (where Umami's exact-hostname check was silently self-disabling because only the apex domain was listed) and instruments an end-to-end checkout funnel: checkout-start / checkout-success / checkout-failed events now flow from the dashboard and the /pro SPA into the same Umami workspace so the pageview → upgrade conversion path is fully joinable.

  • Domain fix: www.worldmonitor.app added to UMAMI_DOMAINS in the dashboard analytics facade and to data-domains in both /pro and welcome static HTML tags; a new policy test pins this invariant to prevent silent regression.
  • Funnel wiring: trackCheckoutStart fires from startCheckout on the dashboard (covering all upgrade CTAs in one call site) and from startCheckout/tryResumeCheckoutFromUrl on /pro (via a local trackFunnelEvent wrapper); trackCheckoutSuccess/trackCheckoutFailed fire from panel-layout.ts's checkout-return reconciliation where both the URL-return and legacy overlay-flag paths are distinguished.
  • Static pages instrumented: pro-test/index.html and pro-test/welcome.html now load the Umami script with nonce=\"wm-static-bootstrap\" (satisfying strict-dynamic CSP) and the welcome CTAs carry data-umami-event attributes; built artifacts in public/pro/ are updated correspondingly.

Confidence Score: 4/5

Safe to merge — purely additive analytics instrumentation with no impact on checkout logic, auth flows, or data persistence.

All checkout flows are guarded with try/catch so analytics failures can never break a purchase. The one gap is that the policy test for the /pro checkout paths checks surface labels but not the event name string itself — if 'checkout-start' were later renamed the test would still pass but the funnel join would break silently. This is the only finding and it does not affect current production behavior.

tests/funnel-analytics-policy.test.mjs — the /pro checkout-start assertion should also pin the event name string, not just the surface labels.

Important Files Changed

Filename Overview
src/services/analytics.ts Adds www.worldmonitor.app to UMAMI_DOMAINS, three new typed funnel events to the EVENTS catalog, and the trackCheckoutStart/trackCheckoutSuccess/trackCheckoutFailed helpers; uses the existing queue-and-flush pattern correctly.
src/services/checkout.ts Adds a single trackCheckoutStart call at the top of startCheckout, before the no-user branch, so both authed and signed-out intent clicks are counted; no behavioral changes to checkout logic.
src/app/panel-layout.ts Imports trackCheckoutSuccess and trackCheckoutFailed; calls them immediately after handleCheckoutReturn/consumePostCheckoutFlag reconciliation with correct source labels.
pro-test/src/services/checkout.ts Adds a local trackFunnelEvent helper accessing window.umami directly; fires checkout-start with surface:'pro-page' and surface:'pro-resume'. Event name is an untyped string with no compile-time check against the main app's UmamiEvent catalog.
tests/funnel-analytics-policy.test.mjs New policy test pins UMAMI_DOMAINS, typed event catalog, all call sites, and tracker script tags; the /pro checkout test checks surface labels but not the event name string 'checkout-start' itself.
pro-test/index.html Adds Umami tracker tag with correct website-id, data-domains including www, and wm-static-bootstrap nonce.
pro-test/welcome.html Adds Umami tracker tag with the same website-id/data-domains/nonce as pro-test/index.html.
pro-test/src/welcome/Hero.tsx Adds data-umami-event and data-umami-event-target attributes to the two primary hero CTAs.
pro-test/src/welcome/FinalCta.tsx Adds data-umami-event attributes to the bottom-of-page primary and secondary CTA links.
pro-test/src/welcome/Nav.tsx Adds data-umami-event='welcome-cta' and data-umami-event-target='welcome-nav' to the nav Launch CTA.
pro-test/src/welcome/PricingTeaser.tsx Adds data-umami-event attributes to both the Free and Pro pricing CTAs with descriptive target names.
tests/secondary-startup.test.mts Updated with a comment explaining why www must stay in UMAMI_DOMAINS; no assertion changes.
public/pro/index.html Built artifact carrying the Umami tracker tag with expected nonce and www in data-domains.
public/pro/welcome.html Built artifact carrying the Umami tracker tag with expected nonce and www in data-domains.
public/pro/assets/index-CNvMpSDX.js Rebuilt /pro bundle containing the trackFunnelEvent helper and checkout-start call sites.
public/pro/assets/welcome-C1WIPzsD.js Rebuilt welcome bundle containing the data-umami-event attributes on CTAs.
public/pro/assets/welcome-C4KU6gzc.js Rebuilt welcome CSS/component bundle artifact.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant U as User
    participant W as Welcome Page
    participant P as /pro Page
    participant D as Dashboard
    participant A as Analytics (Umami)

    Note over W: Umami tracker now active (www + nonce)
    U->>W: Clicks CTA
    W->>A: "welcome-cta {target}"

    U->>P: Navigates to /pro
    Note over P: Umami tracker now active (www + nonce)

    U->>P: Clicks pricing CTA (signed-out)
    P->>A: "checkout-start {surface:'pro-page', authed:false}"
    P->>P: openSignIn(afterSignInUrl with intent)

    U->>P: "Signs in, redirected back with ?checkoutProduct="
    P->>A: "checkout-start {surface:'pro-resume', authed:true}"
    P->>P: window.location.assign(hostedCheckoutUrl)

    U->>D: "Returns to /dashboard?wm_checkout=return"
    Note over D: handleCheckoutReturn reconciles
    D->>A: "checkout-success {source:'url-return'}"
    Note over A: Full funnel joinable in one tool

    alt Dashboard upgrade CTA (signed-in)
        U->>D: Clicks upgrade CTA
        D->>A: "checkout-start {surface:'dashboard', authed:true}"
        D->>D: window.location.assign(hostedCheckoutUrl)
        U->>D: "Returns to /dashboard?wm_checkout=return"
        D->>A: checkout-success / checkout-failed
    end
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant U as User
    participant W as Welcome Page
    participant P as /pro Page
    participant D as Dashboard
    participant A as Analytics (Umami)

    Note over W: Umami tracker now active (www + nonce)
    U->>W: Clicks CTA
    W->>A: "welcome-cta {target}"

    U->>P: Navigates to /pro
    Note over P: Umami tracker now active (www + nonce)

    U->>P: Clicks pricing CTA (signed-out)
    P->>A: "checkout-start {surface:'pro-page', authed:false}"
    P->>P: openSignIn(afterSignInUrl with intent)

    U->>P: "Signs in, redirected back with ?checkoutProduct="
    P->>A: "checkout-start {surface:'pro-resume', authed:true}"
    P->>P: window.location.assign(hostedCheckoutUrl)

    U->>D: "Returns to /dashboard?wm_checkout=return"
    Note over D: handleCheckoutReturn reconciles
    D->>A: "checkout-success {source:'url-return'}"
    Note over A: Full funnel joinable in one tool

    alt Dashboard upgrade CTA (signed-in)
        U->>D: Clicks upgrade CTA
        D->>A: "checkout-start {surface:'dashboard', authed:true}"
        D->>D: window.location.assign(hostedCheckoutUrl)
        U->>D: "Returns to /dashboard?wm_checkout=return"
        D->>A: checkout-success / checkout-failed
    end
Loading

Reviews (1): Last reviewed commit: "feat(analytics): restore Umami on canoni..." | Re-trigger Greptile

Comment on lines +73 to +79
test('/pro checkout service fires checkout-start on both paths', () => {
const src = read('pro-test/src/services/checkout.ts');
assert.ok(src.includes("surface: 'pro-page'"),
'pro-page checkout-start missing from startCheckout');
assert.ok(src.includes("surface: 'pro-resume'"),
'pro-resume checkout-start missing from tryResumeCheckoutFromUrl');
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The test title says "fires checkout-start on both paths" but only asserts the surface labels, not the event name itself. If 'checkout-start' were renamed to something else (e.g. 'checkout-initiated'), both assertions would still pass — the funnel join would silently break because the dashboard's typed catalog and the /pro raw event string would diverge.

Suggested change
test('/pro checkout service fires checkout-start on both paths', () => {
const src = read('pro-test/src/services/checkout.ts');
assert.ok(src.includes("surface: 'pro-page'"),
'pro-page checkout-start missing from startCheckout');
assert.ok(src.includes("surface: 'pro-resume'"),
'pro-resume checkout-start missing from tryResumeCheckoutFromUrl');
});
test('/pro checkout service fires checkout-start on both paths', () => {
const src = read('pro-test/src/services/checkout.ts');
assert.ok(src.includes("trackFunnelEvent('checkout-start',"),
"pro checkout service no longer fires 'checkout-start' — funnel join broken");
assert.ok(src.includes("surface: 'pro-page'"),
'pro-page checkout-start missing from startCheckout');
assert.ok(src.includes("surface: 'pro-resume'"),
'pro-resume checkout-start missing from tryResumeCheckoutFromUrl');
});

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

koala73 added 2 commits July 6, 2026 09:13
checkout.ts now statically imports trackCheckoutStart (#4931); the
overlay-lifecycle and pending-dialog harnesses stub ./billing and
./clerk with only the exports checkout.ts itself uses, so the real
analytics module (which imports onSubscriptionChange and
getClerkUserCreatedAt from those modules) failed the bundle. A no-op
./analytics stub keeps the facade out of the bundle; its behavior is
covered by tests/secondary-startup.test.mts.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
…me surface (PR #4934 review)

Review findings from the adversarial pass (verdict: nothing blocking;
two LOWs fixed for completeness):

- happy.worldmonitor.app rewrites / to the welcome landing, but the pro
  pages' data-domains omitted it -- the landing/pro funnel was blind on
  that host while the dashboard tracked. Added to both pro HTML files.
- The post-sign-in auto-resume re-entered startCheckout and fired a
  second checkout-start with the same 'dashboard' surface, reading a
  signed-out conversion as two attempts. The resume now tags
  surface:'dashboard-resume' (mirroring /pro's pro-page/pro-resume
  split) so funnel queries can dedupe cleanly.

public/pro rebuilt.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Adversarial review + fix round (reviewed at 51a3f1d, fixes in e96c228)

An independent adversarial pass returned verdict: nothing merge-blocking. It verified the 8 risk areas directly: no import cycle or bundle-eagerness regression (analytics was already in the eager graph via panel-layout); trackCheckoutStart cannot throw into the checkout path (window/umami guards + bounded queue; /pro helper is try/catch); the strict-dynamic CSP + static nonce admits the tracker on / and /pro (deploy-config CSP subtests pass); committed public/pro is a faithful rebuild (prerender + critical-css tests pass); no double/missed-fire in the checkout-return branch; no PII in event props (return params are stripped before anything fires); prerender.mjs never touches the tracker tag. It also confirmed the mid-review test-harness commit (51a3f1d) was the complete sweep — no other harness bundles checkout.ts or panel-layout.ts.

Two LOW findings, both fixed in e96c228:

  • happy-host blind spot/ on happy.worldmonitor.app rewrites to the welcome landing, but the pro pages' data-domains omitted happy, so the landing funnel was silently disabled there. Added (dashboard already listed it).
  • checkout-start cardinality — the post-sign-in auto-resume re-entered startCheckout and fired a second checkout-start with the same dashboard surface. Now tagged surface:'dashboard-resume', mirroring /pro's pro-page/pro-resume split, so signed-out conversions dedupe cleanly in funnel queries.

Accepted as informational: /pro events before the deferred tracker loads are dropped (correct trade against blocking checkout); checkout-failed.status is a URL-derived low-trust string (own-session analytics only); the static-nonce pattern is the established codebase convention, not a regression.

For whoever builds the funnel query: attempts = checkout-start where authed:true (surfaces dashboard, dashboard-resume, pro-page, pro-resume); intent clicks = authed:false. A signed-out conversion emits one of each by design.

Post-fix: funnel policy + secondary-startup + both checkout harnesses 28/28, typecheck clean, pro bundle rebuilt.

…ss, closed failure buckets (PR #4934 review round 2)

Four P2 findings, all fixed:

- F1: the tracker tags are now async (a plain defer script holds
  DOMContentLoaded hostage to the analytics host), and the /pro SPA tag
  adds data-exclude-search so checkout-intent params (wm_checkout_*
  product/discount/referral) and Clerk handshake params never land in
  analytics. The welcome landing keeps query tracking (utm/?lang=
  segmentation; no sensitive params are ever routed to /).
- F2: checkout-success is the one funnel event that races a reload --
  the entitlement watcher reloads the dashboard the moment Pro lands,
  often before the deferred Umami queue flushes. trackCheckoutSuccess
  now writes a per-tab sessionStorage marker that is cleared only on
  actual delivery (sendUmamiCall); panel-layout replays it on the next
  boot (replayed:true). Behavioral test covers deliver-clears / replay /
  no-double-count.
- F3: checkout-failed forwards a URL-derived status; checkout-return
  deliberately passes unknown statuses through when Dodo ID params are
  present, so a crafted URL could inject unbounded cardinality.
  Normalized to the closed failed/declined/cancelled/canceled/other set.
- F4 (Greptile): the /pro policy test asserted only the surface labels,
  not that trackFunnelEvent('checkout-start') is actually emitted for
  each path -- tightened to extract emissions, plus new policy guards
  for async/exclude-search, the durable marker wiring, and the closed
  failure vocabulary.

public/pro rebuilt; 33/33 across the durable, policy, secondary-startup
and checkout-harness suites; typecheck clean.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Review round 2 → all four P2 findings fixed in 5c1e499

F1 — tracker boot/param hygiene: FIXED. Both tracker tags are now async defer (a plain defer external script joins the in-order queue and delays DOMContentLoaded behind the analytics host — same reason the Turnstile tag is async). The /pro SPA tag additionally sets data-exclude-search="true" so the checkout-intent resume URL (wm_checkout_product/ref/discount) and Clerk handshake params never reach analytics. Deliberate trade: the welcome landing keeps query tracking — that's where utm_*/?lang= campaign segmentation lives and no sensitive params are ever routed to /; say the word if you'd rather exclude there too. The dashboard is safe by ordering (checkout-return strips its params synchronously at boot, the facade's tracker loads after first paint + 3s).

F2 — dropped terminal event: FIXED. trackCheckoutSuccess now writes a per-tab sessionStorage marker that is cleared only when the event actually reaches the tracker (sendUmamiCall); on any non-checkout-return boot, panel-layout replays a surviving marker as checkout-success{replayed:true}. So the entitlement-watcher reload can no longer eat the purchase signal, and a delivered event can't double-count. Covered by a behavioral test (tests/checkout-success-durable.test.mts): deliver-clears, reload-replays, no-double-count.

F3 — unbounded status cardinality: FIXED. trackCheckoutFailed normalizes the URL-derived status to the closed failed/declined/cancelled/canceled/other vocabulary (checkout-return deliberately forwards unknown statuses when Dodo ID params are present, so the open path was real).

F4 (Greptile) — weak /pro assertion: FIXED. The policy test now extracts the actual trackFunnelEvent('checkout-start', …) emissions and asserts each surface (pro-page, pro-resume) is wired to one — deleting a call now fails the test even if the labels linger. Added policy guards for async/exclude-search, the durable-marker wiring, and the closed failure vocabulary.

Post-fix: 33/33 across durable + policy + secondary-startup + both checkout harnesses, typecheck clean, public/pro rebuilt (built tag verified to carry async + data-exclude-search).

…guard (PR #4934 review round 4)

Elie's cross-PR review, three P2s:

- F2 cardinality: checkout-start productId travels through URL
  (wm_checkout_product) and sessionStorage on the resume paths, so a
  crafted value could inject unbounded cardinality into Umami. Both
  surfaces now bucket unknown ids to 'unknown' against generated
  allowlists (dashboard: DODO_PRODUCTS from products.generated.ts;
  /pro: tiers.json product ids -- exactly the purchasable set per
  build). Checkout itself still passes the raw id through; the backend
  validates. Behavioral test asserts crafted-id collapse + known-id
  passthrough.
- F3 dropped resume event: the /pro tracker is async and typically
  loads AFTER the mount-time tryResumeCheckoutFromUrl, so the silent
  no-op trackFunnelEvent dropped the only signed-in pro-resume event.
  /pro funnel events now queue (bounded, 20) and flush via a 500ms
  poll for up to 30s once window.umami appears.
- F4 double-fire: checkoutInFlight is only set inside doCheckout,
  after the awaited ensureClerk() -- rapid double-clicks both passed
  the entry check and fired checkout-start twice (checkout stayed
  single; analytics did not). A synchronous whole-start re-entrancy
  guard now wraps startCheckout.

Also fixed a latent policy-test bug the bucketing exposed: the
emission-extraction regex stopped at the FIRST ')' -- which is now the
inner bucketProductIdForAnalytics() call -- so it truncated before the
surface label; it now spans to the closing '})'.

public/pro rebuilt; policy + durable + secondary-startup + both
checkout harnesses green; main and pro tsc clean.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Review round 4 (Elie's cross-PR pass) → all three #4934 findings fixed in 4e58a38

F2 — product-id cardinality: FIXED. Both surfaces now bucket unknown product ids to 'unknown' before tracking, against generated allowlists (dashboard: DODO_PRODUCTS from products.generated.ts; /pro: tiers.json ids — exactly the purchasable set for that build). The raw id still flows to checkout, where the backend validates. Behavioral test covers crafted-id collapse + known-id passthrough. This also exposed a latent bug in the round-2 policy test: its emission regex stopped at the first ) — now the inner bucket call — so it truncated before the surface label; fixed to span to the closing }).

F3 — dropped pro-resume event: FIXED. /pro funnel events now queue (bounded at 20) when window.umami isn't there yet, flushing via a 500ms poll for up to 30s — the mount-time tryResumeCheckoutFromUrl event survives the async tracker load. Blocked-tracker case gives up cleanly.

F4 — checkout-start double-fire: FIXED. A synchronous whole-start re-entrancy guard (startCheckoutEntryInFlight, set before any await, cleared on settle) closes the window between the entry check and doCheckout setting checkoutInFlight. Checkout was never double-created; the analytics event was.

Policy guards added for all three. public/pro rebuilt; policy + durable + secondary-startup + both checkout harnesses green; main and pro tsc clean.

…(PR #4934 review round 5)

The round-4 in-memory queue never got to flush on the fast path: a
signed-in click (or auto-resume) continues straight into doCheckout's
top-level redirect to Dodo, unloading /pro with the event still queued.

- /pro now mirrors undelivered checkout-start events into sessionStorage
  (per-tab, survives the Dodo round-trip; bounded at 10) and clears the
  mirror the moment a local flush delivers, so nothing double-replays.
- The dashboard return landing replays them unconditionally at boot
  (both checkout-return and ordinary branches -- the buyer can land on
  either). Every replayed field is re-validated against closed
  vocabularies (event name must be checkout-start, productId re-bucketed
  via the generated allowlist, surface coerced to pro-page/pro-resume,
  authed coerced to boolean) because sessionStorage is client-writable
  and replayed junk must not become analytics cardinality; replays carry
  replayed:true.
- Behavioral tests: valid event replays sanitized, crafted fields
  collapse, unknown event names and garbage entries drop, malformed
  storage clears without tracking, ordinary boots are no-ops. Policy
  test pins the handoff key in BOTH builds + the persist/clear/replay
  wiring.

public/pro rebuilt; policy + durable + secondary-startup + both
checkout harnesses green; main and pro tsc clean.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Review round 5 → fixed in d25fe94

P2 — queued /pro checkout-start dies with the Dodo redirect: FIXED. The round-4 queue was memory-only and the fast signed-in/resume path top-level-redirects before the flush poll runs. Undelivered checkout-start events are now mirrored into sessionStorage (per-tab, survives the Dodo round-trip, bounded at 10, cleared on local delivery), and the dashboard replays them at boot on the same-origin return — with every field re-validated against closed vocabularies (event-name allowlist, productId re-bucketed, surface coerced, authed coerced) since sessionStorage is client-writable; replays carry replayed:true so they're distinguishable in queries.

Behavioral tests cover: sanitized replay, crafted-field collapse, unknown-event drop, malformed-storage clear, ordinary-boot no-op. Policy test pins the handoff key in both builds plus the persist/clear/replay wiring. All suites green, both tsc clean, pro rebuilt.

… (PR #4934 review round 6)

The round-5 replay removed wm-pro-funnel-pending at READ time, then the
replayed events sat in the deferred in-memory queue -- the entitlement
watcher reload (the exact race round-2 fixed for checkout-success)
could kill the queue before Umami loaded, permanently dropping the
/pro checkout-start with the durable handoff already gone.

Now mirrors the checkout-success contract exactly:
- replayPendingProFunnelEvents no longer removes the key; it REWRITES
  it with only the sanitized survivors (closed vocabularies, bounded)
  so a pre-delivery reload retries exactly those and junk cannot loop,
  clearing immediately only when nothing is replayable (malformed /
  all-invalid payloads).
- sendUmamiCall clears the marker once a replayed checkout-start
  (data.replayed === true) actually reaches the tracker. Live
  dashboard checkout-starts never clear it. All replays flush in one
  synchronous loop, so first-delivery-clears is safe.
- Tests now encode the delivery contract: marker persists (sanitized)
  through a pre-delivery reload and the retry delivers; clears on
  delivery; unreplayable payloads still clear. The old assertion that
  pinned clear-on-read is gone. Policy test pins the delivery hook and
  the sanitized rewrite.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
@koala73

koala73 commented Jul 6, 2026

Copy link
Copy Markdown
Owner Author

Review round 6 → fixed in 0318e62

P2 — replay marker cleared before delivery: FIXED. replayPendingProFunnelEvents no longer removes the handoff key at read time. It now mirrors the checkout-success contract exactly: the key is rewritten with the sanitized survivors (closed vocabularies, bounded — junk can't loop) and removed only in sendUmamiCall once a replayed event (data.replayed === true) actually reaches the tracker. Live dashboard checkout-starts never clear it, and since all replays flush in one synchronous loop, first-delivery-clears is race-free. Unreplayable payloads (malformed JSON, all-invalid entries) still clear immediately.

The reviewer was also right that the old test encoded the wrong contract — it asserted clear-on-read. Replaced with tests that pin the delivery contract: marker persists (sanitized) through a pre-delivery reload → the retry delivers → then clears; plus delivery-clears and unreplayable-clears cases. Policy test pins the delivery hook and the sanitized rewrite.

All suites green, tsc clean. No /pro sources changed this round (dashboard-side only).

@koala73
koala73 merged commit c621267 into main Jul 6, 2026
25 checks passed
koala73 added a commit that referenced this pull request Jul 6, 2026
…serve (#4945) (#4946)

* feat(pricing): publish the API Business tier — visible, priced, self-serve (#4945)

API Business ($249.99/mo, 300 req/min, 10k req/day, priority support,
XLSX) existed in the billing system with zero customers because it was
invisible on every pricing surface (publicVisible/selfServe/
currentForCheckout all false, priceCents null).

- Catalog: flags flipped, display-fallback price set to 24999 (the live
  Dodo price still wins on the page and at charge time; $249.99 was
  verified against Dodo via previewChangePlan in #4634).
- Generator: api_business -> 'apiBusiness' locale-key mapping; en.json
  gains the pricing.tiers.apiBusiness placeholder (other locales fall
  back to English at runtime, same as welcome.*).
- api/product-catalog.js mirror: TIER_CONFIG entry + PUBLIC_TIER_GROUPS
  so the live endpoint serves 5 tiers with Dodo-fetched pricing.
- Pricing grid: 1/2/3/5 column rhythm (was 1/2/4) for the fifth card.
- tiers.json + fallback prices regenerated; public/pro rebuilt;
  freshness-test tier maps extended.

New-customer checkout uses the existing create-checkout path (the
active-subscription guard routes existing Starter customers to the
portal, where the #4634 collection upgrade already lives). Existing-
customer upgrade CTA stays with open PR #4672; the 429 upgrade_url
deep-link stays with open PR #4684.

Closes #4945

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* fix(pricing): mirror the apiBusiness locale block into fa.json (#4945)

fa.json is deliberately a byte-identical English mirror; the pro locale
registry test requires every non-English locale to match either the
translated (ar.json) schema or the COMPLETE English schema. en.json
gaining pricing.tiers.apiBusiness broke that for fa — mirrored the same
block in. public/pro rebuilt (locales are bundled).

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* fix(pricing): seed api_business from the Railway price seeder + API billing-family checkout guard + docs parity (PR #4946 review)

Adversarial review findings, all addressed:

- P0: scripts/ais-relay.cjs (the Railway "DodoPrices" seeder) owns the
  Redis payload /api/product-catalog serves on cache HITS -- in steady
  state it WINS over the edge fallback. Its hardcoded 4-tier mirror
  (DODO_PRODUCT_IDS/TIER_CONFIG/PRODUCT_META/FALLBACK_PRICES/
  publicGroups) would have collapsed the live /pro page back to 4 cards
  the moment the fetch resolved. All five sites now carry api_business,
  and a new parity test in product-catalog-freshness derives the
  expected set from productCatalog.ts and fails on ANY priced
  publicVisible entry missing from the seeder (plus a mirror-to-mirror
  publicGroups equality check against api/product-catalog.js).
- HIGH: api_starter and api_business are distinct tier groups, so the
  exact-tierGroup duplicate-checkout guard let an active Starter
  customer buy Business as a SECOND concurrent subscription
  ($99.99 + $249.99 double-billing). New checkoutBillingFamily() treats
  api_* as one family in getCheckoutBlockingSubscription AND
  getBlockingPendingPayment: Starter-active (or Starter-pending-3DS)
  now 409s a Business checkout into the duplicate dialog -> billing
  portal, where the #4634 collection upgrade lives. Pro <-> API
  cross-line stays deliberately allowed. One existing test pinned the
  old policy and was inverted with a comment; three new family tests
  added (starter blocks business, business blocks starter, pro does
  not block business).
- MEDIUM: agent-facing pricing surfaces now include API Business --
  public/pricing.md (prose + machine-readable JSON), docs/pricing.mdx
  (table + bullets), public/llms.txt; pricing-docs-drift PLAN_KEYS +
  EXPECT extended so this drift is caught next time.
- LOW (accepted): non-English locales render English API Business copy
  via the defaultValue fallback -- same generator behavior as every
  other tier string.

599/599 convex tests, 14/14 freshness (incl. new parity), 12/12 drift,
typecheck:api + convex tsc clean.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* fix(pricing): correct Business req/day copy, localeKey in live catalog, api-commerce example (PR #4946 review round 2)

Cross-review findings (Elie's ce-code-review at 83c35c4; the ais-relay
seeder and billing-family guard from that review were already fixed in
278fa90):

- P1 copy conflict: the /pro API section advertised Business at
  "50,000 req/day" across 25 locales while the canonical tier is
  10,000/day -- a purchasable overpromise. Number-only fix in every
  locale (unit phrasing preserved: req/Tag, 次/天, طلب/يوم...).
- P2 localeKey: the live catalog payload (both mirrors: edge endpoint +
  Railway seeder) lacked localeKey, so once live data replaces the
  static tiers.json, PricingSection's fallback name.toLowerCase()
  breaks for multi-word names ('API Business' -> 'api business') and
  localized tier copy is lost. All five TIER_CONFIG entries in both
  mirrors now carry the generated localeKey; parity test extended to
  pin them.
- P2 docs: docs/api-commerce.mdx example response still showed the
  Free/Pro/API/Enterprise catalog with LONG-stale prices (Pro $20/$180
  -- flagged in the 2026-07-05 docs audit too). Example updated to the
  5-tier catalog with current prices + localeKey; api-commerce.mdx
  added to the pricing-docs-drift DOCS list (price matcher now accepts
  bare JSON numbers) so it can never go stale silently again.

public/pro rebuilt (locales are bundled). 35/35 across drift +
freshness/parity + locale-registry suites; typecheck:api clean.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* fix(pricing): 5-tier OpenAPI example, JSON-LD API offers, catalog-true Business support copy (PR #4946 review round 3)

- docs/openapi/CommerceService.openapi.yaml: tiers description and the
  response example now show the full Free/Pro/API/API Business/
  Enterprise catalog (example previously stopped at Pro).
- /pro JSON-LD offers now include API Starter Monthly ($99.99), API
  Starter Annual ($999) and API Business ($249.99) so structured/no-JS
  pricing metadata matches the visible pricing grid.
- apiSection.businessWebhooks said "Unlimited webhooks + SLA" -- no
  webhook cap exists in code to back "unlimited", and SLA is an
  Enterprise-only catalog bullet. Replaced with the catalog-true
  "Priority support + XLSX exports" across all 25 locales (English
  string everywhere; accuracy over localization for a support promise,
  translators can re-localize).

public/pro rebuilt; 144/144 across drift + locale-registry +
freshness/parity + deploy-config suites.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* fix(pricing): explicit billed-monthly state for annual toggle + docs contract accuracy (PR #4946 review round 4)

- P2 annual toggle: with the page-level toggle on Annual, API Business
  (monthly-only) silently checked out its monthly product. The CTA now
  routes through a pure resolveCheckoutProduct() helper whose
  billedMonthlyOnly flag renders an explicit "Billed monthly -- no
  annual plan" note on the card in annual mode. Focused regression
  suite pins the resolver contract (annual->annual, monthly-only->
  monthly WITH the flag, monthly->no flag, free/enterprise->null) plus
  the render wiring. This also resolves the open review thread on the
  annual fallback.
- P2 docs contract: api-commerce.mdx claimed all paid tiers expose
  annualPrice/annualProductId -- reworded to "when an annual variant
  exists" with API Business called out as monthly-only (the OpenAPI
  field descriptions already say "absent when"-style wording).
- P3 architecture doc: docs/architecture/pro-monetization.md tier model
  now lists API Business (it stopped at API Annual + Enterprise).

public/pro rebuilt; 40/40 across billing-mode + drift + freshness/
parity + locale-registry; pro tsc clean.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* docs(pricing): epoch timestamps in commerce example, conditional annual fields in OpenAPI prose, verification metadata (PR #4946 review round 5)

- api-commerce.mdx example showed fetchedAt/cachedUntil as ISO strings;
  the endpoint returns epoch milliseconds (the OpenAPI example was
  already numeric) -- clients must not build against the wrong type.
- CommerceService.openapi.yaml price-field description said
  subscription tiers carry monthlyPrice/annualPrice; now states annual
  fields appear only when an annual variant exists (API Business is
  monthly-only).
- pro-monetization.md claimed verification against a 2026-04-21 SHA;
  bumped to 2026-07-06 with the PR reference instead of a stale SHA
  pin.

17/17 pricing-docs-drift; yaml parses.

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

* chore(pro): rebuild public/pro on the post-#4934 merged tree — tracker tag + 5-tier catalog in one consistent bundle

Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(analytics): conversion-funnel instrumentation — Umami dead on canonical www host, no checkout events, /pro + landing dark

1 participant