feat(api): per-tier API rate limits — Phase 1 (per-account burst + usage meter + safety ceiling) by koala73 · Pull Request #4563 · koala73/worldmonitor

koala73 · 2026-06-30T15:51:07Z

Closes #3199 (Phase 1). Phase 2 (overage billing) tracked in #4560.

What

Wires a per-account REST rate-limit layer into the single gateway chokepoint, replacing the "advertised but unenforced" per-tier allowances. Two distinct limit types:

Per-minute burst (infra/abuse) → hard 429, from apiRateLimit (Starter 60 / Business 300 / Enterprise 1,000 per 60s).
Daily usage meter (commercial allowance) → counts, never rejects at the allowance (new apiDailyAllowance: 1,000 / 10,000 / unlimited).
Safety ceiling at 10× allowance → hard 429 (runaway protection; Enterprise none).

Eligible authenticated keys bypass the global per-IP limiter in enforce mode only (shadow keeps per-IP active, so protection never drops below today). Fail-open throughout. Ships shadow-first behind API_RATE_LIMIT_ENFORCE — merge is inert in prod until the flag flips.

This also fixes the latent contradiction where Enterprise's advertised 1,000/60s was unreachable behind the 600/60s per-IP cap, and gives Business real marketed differentiation (its marketingFeatures were empty).

How (implementation units)

U1 convex/config/productCatalog.ts — apiDailyAllowance field + Business marketing copy.
U2 convex/schema.ts, cacheActions.ts, entitlement-check.ts — thread the field (optional; undefined ⇒ fail-open). Required so strict v.object accepts catalog-sourced webhook writes.
U3 server/_shared/api-key-rate-limit.ts (new) — decision-only module: cached Map<number,Ratelimit> burst + INCR-first daily meter w/ DECR rollback (clones api/mcp/quota.ts), UTC-day key, fail-open.
U4 server/gateway.ts — eligibility (isUserApiKey/isEnterpriseAuth — user keys carry no keyCheck.kind), explicit getEntitlements resolution, enforce-gated per-IP bypass, shadow reason threaded onto the single terminal emitRequest (no double-count), 4 new RequestReason values.
U5 docs/usage-rate-limits.mdx — per-plan table + full X-RateLimit-* headers; keeps the contact-support escalation line.

Rollout

Merge → shadow (API_RATE_LIMIT_ENFORCE unset). Per-IP protection fully retained.
Audit rl_*_shadow telemetry; resolve plan OQ1–5 (10× justification, telemetry durability, "included" copy timing).
Flip API_RATE_LIMIT_ENFORCE=true → enforce (one env change, no redeploy).
Phase 2 (epic: usage-based overage billing on Dodo (Phase 2 of #3199) #4560): wire the meter to Dodo usage-based billing.

Verification

157 tests pass: U1 catalog (7), U2 entitlement threading (vitest), U3 module (16 — meter/ceiling/fail-open/UTC-rollover), U4 gateway wiring (6 vitest — enforce/shadow/bypass/downgrade matrix); 58 existing gateway+rate-limit tests still green.
tsc --noEmit, typecheck:api, and biome lint all clean.
Adversarial code review: sound and shippable, no blocking defects. One LOW finding fixed (allowance:0 would have ceiling-429'd request Batch market and RSS fetching with progressive updates #1 → now fails open). Informational notes carried to Phase 2: the meter counts 4xx (404/405/malformed) outcomes too, which the Phase-2 billing basis (plan OQ4) should exclude.

Risk

Shadow-first + fail-open + per-IP retained in shadow = the merge cannot reduce protection or surprise a paying customer. The only behavioral change at merge is telemetry emission.

Plan: docs/plans/2026-06-30-001-feat-per-tier-api-rate-limits-plan.md

https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

…U1, #3199) Per-account daily REST allowance per tier (Starter 1000 / Business 10000 / Enterprise -1 unlimited; Free/Pro 0). Read by the per-account rate-limit layer: the daily meter counts but never rejects at this value; hard ceiling is 10x. Populates the empty Business marketingFeatures and rewords Starter copy to '1,000 requests/day included'. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

…e (U2, #3199) Adds the optional field to the entitlements schema validator, the syncEntitlementCache validator, and the CachedEntitlements type so the gateway can read features.apiDailyAllowance. Required for U1's catalog change: v.object is strict, so catalog-sourced webhook writes (now carrying apiDailyAllowance) would be rejected without the validator update. Deliberately NOT added to the cache-staleness gate — undefined is fail-open (no daily limit), unlike mcpAccess's fail-closed re-fetch. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

…ing (U3, #3199) New server/_shared/api-key-rate-limit.ts (placed in _shared as a .ts for clean gateway interop, vs the plan's tentative api/*.js). Decision-only: the gateway owns enforce/shadow + Response. Burst = cached Map<number,Ratelimit> per limit, fail-open. Daily meter clones api/mcp/quota.ts (INCR-first + DECR rollback), counts every request, flags only when count > 10x allowance; allowance -1 skips Redis. UTC-day key (runRedisPipeline applies the env prefix). 15 unit tests cover meter/ceiling/fail-open/UTC-rollover with injected pipeline+date. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

At server/gateway.ts:1034: eligible authenticated keys (valid user key via isUserApiKey — they carry no keyCheck.kind — or isEnterpriseAuth) run burst -> meter -> 10x ceiling. User keys resolve getEntitlements explicitly (features aren't in scope here; null/downgraded -> fall through to per-IP, never slidingWindow(0)); enterprise uses hardcoded 1000/min + sha256(wmKey), no daily. The per-IP bypass is ENFORCE-only: in shadow per-IP checkRateLimit still runs so protection never drops below today. Shadow reason is threaded onto the single terminal emit (no double-count). Adds 4 RequestReason values. 6 gateway-wiring tests (vitest, module-stubbed) cover the enforce/shadow/ bypass/downgrade matrix the review flagged; typecheck clean; 58 existing gateway+rate-limit tests still green. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

Adds a per-plan table (Starter/Business/Enterprise burst + daily included + UTC reset + 10x safety ceiling), clarifies authenticated keys are limited per account not per IP, and shows the full X-RateLimit-* + Retry-After header set. Keeps the contact-support escalation line (no self-serve overage until Phase 2). Numbers match the catalog (U1) so doc and enforcement can't drift. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

…quest (#3199) Code-review LOW: reserveDailyMeter short-circuited only allowance < 0, so a (currently unreachable) tier with apiRateLimit>0 + apiDailyAllowance:0 would compute ceiling = 0 and 429 on request #1 in enforce. Guard allowance <= 0 → fail open (no daily limit), matching the layer's fail-open philosophy. +1 test. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

The spread of a conditional REDIS_TEST_RETRY_OPTS widened retry to boolean, which @upstash/redis's RetryConfig rejects under the stricter tsconfig.api.json (pre-push gate). Branch the Redis construction so retry:false stays a literal, mirroring api/mcp/auth.ts. Preserves prod-resilient / test-fast behavior. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

vercel · 2026-06-30T15:51:13Z

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ready	Preview, Comment	Jun 30, 2026 4:14pm

mintlify · 2026-06-30T15:55:06Z

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
WorldMonitor	🟢 Ready	View Preview	Jun 30, 2026, 3:56 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

greptile-apps · 2026-06-30T15:59:43Z

Greptile Summary

Wires a per-account REST rate-limit layer into the single gateway chokepoint, replacing the previously unenforced per-tier allowances. Ships fully behind API_RATE_LIMIT_ENFORCE so the merge is inert in production until the flag flips, and retains per-IP protection in shadow mode throughout.

U3 (server/_shared/api-key-rate-limit.ts) implements a decision-only module with INCR-first daily meter + DECR rollback, per-minute burst via Upstash Ratelimit, and fail-open throughout.
U4 (server/gateway.ts) wires the layer at the single chokepoint: eligible user keys are identified via isUserApiKey (not keyCheck.kind), enterprise keys use a hardcoded limit; enforce mode bypasses per-IP while shadow retains it.
U1–U2 extend the product catalog with apiDailyAllowance, thread the field through the schema/cache/entitlement chain, and flesh out Business marketing copy that was previously empty.

Confidence Score: 4/5

Safe to merge. The change is inert in production until API_RATE_LIMIT_ENFORCE=true is set, and shadow mode preserves full per-IP protection throughout, so no customer impact can come from the merge itself.

The core rate-limit logic is solid: INCR-first atomic meter, idempotent DECR rollback, and fail-open at every Redis failure point. The shadow-first rollout removes deployment risk. Two items are worth tracking before flipping the enforce flag: the burst limiter prefix is not env-scoped (daily meter keys are, via runRedisPipeline), which can bleed preview burst counts into production if both share the same Upstash database; and enterprise keys rate-limit per-key rather than per-account on the burst axis, diverging from how user keys work and from what a reader of the docs might expect.

server/_shared/api-key-rate-limit.ts (burst limiter env prefix) and server/gateway.ts (enterprise identity scope at line 1080)

Important Files Changed

Filename	Overview
server/_shared/api-key-rate-limit.ts	New decision-only module with burst limiter, daily INCR/DECR meter, and fail-open throughout. One env-isolation gap: the burst limiter's prefix is not deployment-scoped unlike the daily meter keys.
server/gateway.ts	Shadow/enforce gating, per-IP bypass, and pendingShadowReason threading are correctly implemented. Enterprise burst identity is per-key (hashKeySync(wmKey)) while user-key burst is per-account (sessionUserId) — asymmetry that may surprise enterprise customers with multiple keys.
convex/config/productCatalog.ts	apiDailyAllowance added for all tiers; Free/Pro set to 0, Starter 1000, Business 10000, Enterprise -1. Business marketingFeatures populated. All catalog entries set the field explicitly.
server/_shared/entitlement-check.ts	apiDailyAllowance added to the CachedEntitlements interface, deliberately excluded from the staleness gate so legacy cache rows aren't forced to refetch (fail-open contract preserved).
docs/usage-rate-limits.mdx	Per-plan table and X-RateLimit-* header documentation added. The 'all keys share one allowance' claim is accurate for the daily axis but not for the per-minute burst on enterprise keys.
convex/schema.ts	apiDailyAllowance added as v.optional(v.number()) to the entitlement cache validator so catalog-sourced webhook writes are accepted by the strict v.object schema.
convex/payments/cacheActions.ts	Mirrors the schema change — apiDailyAllowance added as v.optional(v.number()) to syncEntitlementCache validator. Correctly symmetric with schema.ts.
server/_shared/usage.ts	Four new RequestReason values added (rl_min_429, rl_ceiling_429, rl_min_shadow, rl_ceiling_shadow). No other changes.
tests/api-key-rate-limit.test.mts	16 tests covering meter/ceiling/fail-open/UTC-rollover with injected mock pipeline. Good discipline: Upstash sliding-window math is not re-tested, only the module's own logic.
server/tests/gateway-api-rate-limit.test.ts	6 vitest tests covering enforce/shadow/bypass/downgrade matrix with stubbed burst and meter modules. Tests verify the gateway wiring, not the rate-limit math.
tests/product-catalog-api-allowance.test.mts	7 tests asserting per-tier allowance values and Business marketing copy. The 'every catalog entry sets apiDailyAllowance explicitly' test guards against future catalog additions missing the field.
server/tests/entitlement-check.test.ts	Two new tests confirm apiDailyAllowance threads through getEntitlements for fresh rows and that legacy rows missing the field resolve to undefined (fail-open) without throwing.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant C as Client
    participant GW as Gateway
    participant EC as EntitlementCheck
    participant BL as BurstLimiter (Upstash)
    participant DM as DailyMeter (Redis pipeline)
    participant IP as Per-IP Limiter
    participant H as Route Handler

    C->>GW: Request (wm_ key)
    GW->>GW: validateUserApiKey → sessionUserId

    GW->>EC: checkEntitlement(sessionUserId, path)
    EC-->>GW: null (allowed)

    GW->>EC: getEntitlements(sessionUserId)
    EC-->>GW: "{ apiRateLimit, apiDailyAllowance }"

    alt ENFORCE mode
        GW->>BL: checkBurst(perMinute, userId)
        alt Burst exceeded
            BL-->>GW: "{ ok: false }"
            GW-->>C: 429 (rl_min_429)
        else Burst OK
            BL-->>GW: "{ ok: true }"
            GW->>DM: reserveDailyMeter(userId, allowance)
            alt Over 10x ceiling
                DM-->>GW: "{ overCeiling: true }"
                GW->>DM: rollback()
                GW-->>C: 429 (rl_ceiling_429)
            else Within ceiling
                DM-->>GW: "{ overCeiling: false }"
                GW->>H: invoke handler (per-IP bypassed)
                H-->>GW: Response
                GW-->>C: 200 (ok)
            end
        end
    else SHADOW mode
        GW->>BL: checkBurst(perMinute, userId)
        BL-->>GW: result
        Note over GW: Sets pendingShadowReason if would-have-429d
        GW->>IP: checkRateLimit (per-IP always active)
        IP-->>GW: null (or 429)
        GW->>H: invoke handler
        H-->>GW: Response
        GW-->>C: "200 (effectiveReason = shadow reason if status < 400)"
    end

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant C as Client
    participant GW as Gateway
    participant EC as EntitlementCheck
    participant BL as BurstLimiter (Upstash)
    participant DM as DailyMeter (Redis pipeline)
    participant IP as Per-IP Limiter
    participant H as Route Handler

    C->>GW: Request (wm_ key)
    GW->>GW: validateUserApiKey → sessionUserId

    GW->>EC: checkEntitlement(sessionUserId, path)
    EC-->>GW: null (allowed)

    GW->>EC: getEntitlements(sessionUserId)
    EC-->>GW: "{ apiRateLimit, apiDailyAllowance }"

    alt ENFORCE mode
        GW->>BL: checkBurst(perMinute, userId)
        alt Burst exceeded
            BL-->>GW: "{ ok: false }"
            GW-->>C: 429 (rl_min_429)
        else Burst OK
            BL-->>GW: "{ ok: true }"
            GW->>DM: reserveDailyMeter(userId, allowance)
            alt Over 10x ceiling
                DM-->>GW: "{ overCeiling: true }"
                GW->>DM: rollback()
                GW-->>C: 429 (rl_ceiling_429)
            else Within ceiling
                DM-->>GW: "{ overCeiling: false }"
                GW->>H: invoke handler (per-IP bypassed)
                H-->>GW: Response
                GW-->>C: 200 (ok)
            end
        end
    else SHADOW mode
        GW->>BL: checkBurst(perMinute, userId)
        BL-->>GW: result
        Note over GW: Sets pendingShadowReason if would-have-429d
        GW->>IP: checkRateLimit (per-IP always active)
        IP-->>GW: null (or 429)
        GW->>H: invoke handler
        H-->>GW: Response
        GW-->>C: "200 (effectiveReason = shadow reason if status < 400)"
    end

_{Reviews (1): Last reviewed commit: "fix(api): keep retry:false a literal for..." | Re-trigger Greptile}

…4563 review) greptile P2 #1: the burst limiter prefix bypassed runRedisPipeline's env namespacing, so a preview deploy sharing one Upstash DB would collide with prod's burst buckets (daily keys were already isolated). Export getKeyPrefix and prepend it to the burst prefix — consistent with every other Redis key, empty in production. greptile P2 #2: enterprise burst is keyed per-key, not per-account. This is by design — enterprise keys are operator-issued (no shared userId) and have unlimited daily, so there's no quota to multiply; forcing one shared bucket would throttle legitimate multi-key operator use. Clarified in the gateway comment and the docs (the per-account guarantee is scoped to wm_ user keys). Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

…iers + locales + rebuild bundle) (#4574) * fix(pro): regenerate pro-test tiers.json after #4563 catalog copy change (#4563 follow-up) #4563 changed Starter/Business marketingFeatures in the product catalog but didn't re-run scripts/generate-product-config.mjs, leaving the generated /pro view model stale ('1,000 requests/day' instead of '...included' + '60 requests/minute'). The path-filtered pro-test freshness gate didn't fire because #4563 touched only convex/, not pro-test/. Rebuild of public/pro/ follows in this PR. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1 * fix(pro): sync /pro pricing copy + rebuild bundle after #4563 (#4563 follow-up) The /pro PricingSection renders features from the i18n locales (tArray, with generated/tiers.json only as ?? fallback), so the catalog copy change in #4563 didn't reach the page — en.json/fa.json still said '1,000 requests/day'. Sync both locale overrides to the catalog ('60 requests/minute' + '1,000 requests/day included') and rebuild the committed public/pro bundle (Vercel serves it as-is, never rebuilds it). Only en/fa carry this override; other locales inherit. Old copy is fully gone from public/pro. Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1 * fix pro pricing copy drift

…ss (#4974) (#4975) * fix(pricing): remove the phantom "XLSX exports" claim from API Business (#4974) Elie's catch: the tier advertises XLSX exports, but no XLSX exporter, endpoint, or library exists anywhere in the product — the only xlsx mentions are the marketing string itself and the unconsumed exportFormats metadata. The bullet shipped with the original hidden catalog entry (#4563) and #4946 propagated it to every public surface. Removed from: catalog marketingFeatures, both live-catalog mirrors (api/product-catalog.js + ais-relay.cjs), apiSection.businessWebhooks copy (25 locales -> "Priority support"), pricing.md prose + JSON summary, pricing.mdx table + bullets, /pro JSON-LD offer description; tiers.json + en/fa placeholders via generator regen. exportFormats metadata left untouched (entitlement-row shape, no consumer). The tier's true differentiators stand: 300 req/min, 10,000 req/day, priority support. public/pro rebuilt; 54/54 across freshness/parity + drift + locale-registry + billing-mode + funnel-policy suites. Closes #4974 Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s * fix(pricing): drop phantom xlsx from features metadata + mirror cross-links + feature-array parity (PR #4975 review) Greptile's two findings on the removal PR, both addressed: - exportFormats metadata still listed "xlsx" for api_business — inert today (nothing consumes the array) but primed to mislead the next developer who adds export-gating. Removed, and the PlanFeatures field now carries an explicit do-not-gate-on-this warning citing #4974. Enterprise's array left as-is (sales-led; the field-level warning covers it). - The seeder's DODO_TIER_CONFIG is a hand-maintained copy of the edge's TIER_CONFIG with nothing linking them. Both now carry cross-reference comments naming all three copies and the parity test — and the parity test itself now compares the FEATURE ARRAYS (mirror-to-mirror, and against generated tiers.json), which is exactly the check whose absence let the XLSX string survive in copy. Mutation-verified: re-adding the phantom to either mirror fails 4 assertions. 619/619 convex suite, 14/14 freshness/parity, typecheck:api clean, generator regen no-op. Claude-Session: https://claude.ai/code/session_0161Cso3K8pbtf276Q9WPM3s

koala73 added 7 commits June 30, 2026 19:19

koala73 added P1 High priority, fix soon area: API Backend API, sidecar, keys High Value Meaningful contribution to the project labels Jun 30, 2026

vercel Bot deployed to Preview June 30, 2026 15:54 View deployment

mintlify Bot deployed to staging - docs June 30, 2026 15:56 View deployment

greptile-apps Bot reviewed Jun 30, 2026

View reviewed changes

Comment thread server/_shared/api-key-rate-limit.ts

Comment thread server/gateway.ts

mintlify Bot deployed to staging - docs June 30, 2026 16:12 View deployment

vercel Bot deployed to Preview June 30, 2026 16:14 View deployment

koala73 merged commit ee1c7da into main Jun 30, 2026
24 checks passed

koala73 deleted the feat/per-tier-api-rate-limits branch June 30, 2026 17:22

This was referenced Jul 6, 2026

fix(pricing): 'XLSX exports' is advertised on API Business but no XLSX export exists anywhere in the product #4974

Closed

fix(pricing): remove the phantom "XLSX exports" claim from API Business (#4974) #4975

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(api): per-tier API rate limits — Phase 1 (per-account burst + usage meter + safety ceiling)#4563

feat(api): per-tier API rate limits — Phase 1 (per-account burst + usage meter + safety ceiling)#4563
koala73 merged 8 commits into
mainfrom
feat/per-tier-api-rate-limits

koala73 commented Jun 30, 2026

Uh oh!

vercel Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

mintlify Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jun 30, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

koala73 commented Jun 30, 2026

What

How (implementation units)

Rollout

Verification

Risk

Uh oh!

vercel Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mintlify Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Jun 30, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Sequence Diagram

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vercel Bot commented Jun 30, 2026 •

edited

Loading

mintlify Bot commented Jun 30, 2026 •

edited

Loading