Skip to content

fix(payments): P0 — webhook drops requires_customer_action/processing → loses pending-payment signal #4436

Description

@koala73

Summary

Our webhook dispatcher has no case for payment.processing or payment.requires_customer_action (3DS/SCA). They fall to the default branch, get a benign console.warn, and are then recorded in webhookEvents as status:"processed" — but the pending-payment state is never persisted. We end up with no signal that a payment is sitting in 3DS limbo, so we can't prevent duplicate retries (#4438) or drive any recovery/reconciliation (#4439). This is the core reason stuck payments are invisible to us and rot forever.

Correction vs. the original framing

This does not "poison idempotency so later transitions are skipped." Dedup is strictly by webhookId (convex/payments/webhookMutations.ts:36-39, by_webhookId index). A later distinct transition (e.g. payment.succeeded) arrives with a new webhookId, so it is not skipped — only a redelivery of the same event id is skipped, which is correct. The harm is narrower: we mark the pending webhook processed and lose the pending-payment signal; later transitions may still arrive, but the app has no pending state for duplicate prevention or recovery.

Evidence

  • convex/payments/webhookMutations.ts:83-127 — the switch only handles subscription.*, payment.succeeded/payment.failed, refund.*, dispute.*. payment.processing/payment.requires_customer_action/payment.cancelled hit default (console.warn, line 125).
  • convex/payments/webhookMutations.ts:132-138 — every event, including the unhandled ones, is then inserted with status:"processed".
  • convex/payments/webhookMutations.ts:36-39 — idempotency is keyed on webhookId only, so distinct later transitions are not blocked.
  • convex/schema.ts:13 — the paymentEventStatus enum (succeeded | failed | dispute_*) cannot represent a pending/requires-action payment (separate field from webhookEvents.status).
  • Adjacent gap: entitlement is granted purely off subscription.active (convex/payments/subscriptionHelpers.ts:554-556) with no cross-check that a payment.succeeded ever arrived.

Fix direction (medium)

  1. Add explicit cases for payment.processing, payment.requires_customer_action, and payment.cancelled that persist a pending payment row keyed by payment_id (extend paymentEventStatus with a pending/requires_action value). This pending row becomes the signal the duplicate-guard (fix(payments): P1 — duplicate stuck payments (no idempotency key + dedup blind to pending 3DS) #4438) and recovery (fix(checkout): P1 — entitlement watchdog gives up silently on requires_customer_action timeout #4439 / reconciliation) read.
  2. Optional cleanup: stop labeling genuinely-unhandled events as status:"processed" in webhookEvents (use a distinct status like ignored) so the audit log doesn't claim work that didn't happen.

Test first

Webhook test feeding a payment.requires_customer_action event: assert a pending payment row is persisted (not silently dropped), and that a subsequent distinct payment.succeeded for the same payment_id still processes.


Part of the DodoPayments 3DS stuck-payments incident.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P0Critical, blocks release or core functionalityarea: APIBackend API, sidecar, keysbugSomething isn't workingcriticalCritical issue requiring immediate attention

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions