Skip to content

Ignore other brands' Dodo webhooks and quarantine unknown products with operation_quarantined #22

Closed

Description

@s3rot0n1n

Problem

SendLit, MediaLit and CourseLit sell through one Dodo Payments business, each under its own Dodo brand. Dodo sends every event of the business to every webhook endpoint, so SendLit's webhook also receives the other products' payment and subscription events.

SendLit's webhook processor (apps/api/src/billing/webhooks/processor.ts) handles an event for a product it does not know differently from the platform standard:

SendLit today Platform standard (@codelitdev/billing)
Error throw new Error("billing_unknown_provider_product") BillingWorkflowError("operation_quarantined")
Retries Retried with backoff until the attempt budget runs out None: the error is final
Quarantine After the last retry On the first attempt, with lastError: "operation_quarantined"

Nothing is applied to any account, but alerts.ts pages webhook_quarantined for each of these events, so every ordinary MediaLit or CourseLit payment would page SendLit's on-call. SendLit's processor predates the package's and has drifted from it.

Proposed fix

  1. Upgrade @codelitdev/billing from 0.1.0-alpha.3 to the release that adds brand filtering (platform ADR 0009). That release also schedules cancellations at the end of the paid period and adds resumeCancellation (ADR 0010).
  2. Filter by Dodo brand: pass brandId to createDodoBillingProvider, for example from a new DODO_BRAND_ID environment variable set to SendLit's brand ID. Events from other brands are then stored as ignored when they arrive. They are never processed, retried, quarantined or paged.
  3. Check product assignment: confirm in Dodo that SendLit's products are assigned to the SendLit brand and not the business's primary brand. Events carry the primary brand's ID when a product has no brand of its own.
  4. Use the platform error for unknown products: in processor.ts, replace throw new Error("billing_unknown_provider_product") with throw new BillingWorkflowError("operation_quarantined") from @codelitdev/billing/core. In the processor's catch, treat operation_quarantined as final, as the package's processWebhookRecord does: set the status to quarantined with lastError: "operation_quarantined" on the first attempt, without scheduling a retry. Keep backoff for transient errors only.
  5. Move to the package's webhook processing (follow-up): replace the custom processor with billing.runWebhookInboxBatch, so SendLit follows the platform rules without its own copy. First check which SendLit-specific steps in processor.ts need a place in the package's lifecycle hooks.
  6. Keep alerting on real problems: after step 2, a quarantined webhook means a real misconfiguration, such as an own-brand product missing from the catalog. Keep paging on it, ideally from billing.health().quarantinedWebhooks.

Acceptance criteria

  • A webhook for another brand's subscription is stored as ignored and does not page.
  • A webhook for a SendLit product still processes as before.
  • A webhook for an unknown product under SendLit's own brand is quarantined on the first attempt with operation_quarantined, is not retried, and pages once.
  • No code path throws billing_unknown_provider_product any more.
  • DODO_BRAND_ID is documented next to the other Dodo settings.

Timing

Steps 1 to 4 should ship before MediaLit takes its first paid checkout in production. Step 5 can follow.

Activity

  1. changed the title [-]Ignore other brands' Dodo webhooks instead of quarantining and paging[/-] [+]Ignore other brands' Dodo webhooks and quarantine unknown products with operation_quarantined[/+] on Oct 7, 2026
  2. moved this from Backlog to Ready in SendLit Roadmapon Oct 10, 2026
  3. s3rot0n1n commented on Oct 27, 2026

    @s3rot0n1n
    ContributorAuthor

    Closing: the fix is in place in code and on prod and QA.

    • @codelitdev/billing is on 0.4.2 and SendLit uses the package's webhook inbox (runWebhookInboxBatch). The custom processor.ts is gone, and billing_unknown_provider_product is no longer thrown anywhere. Unknown products now go through the package's operation_quarantined path.
    • DODO_BRAND_ID is passed to createDodoBillingProvider, documented in both .env.example files, and set in the infra env templates for prod and QA.
    • Prod: confirmed from a real subscription event (billing webhook-inspect) that SendLit's events carry the configured brand ID and are processed. The API restarted cleanly with the ID set.

    Not yet seen live: a foreign brand's event showing ignored. It will appear on MediaLit's first paid checkout.

    Remaining gap, to be tracked separately: alerts.ts still counts quarantined rows straight from billing_webhook_events and pages on every check. It should read billing.health().quarantinedWebhooks and page once per new quarantine. SendLit also has no test of its own for a foreign-brand event being stored as ignored.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions