Skip to content

fix(telemetry): attribute API-key usage events to tier + plan_key (#4572)#4573

Merged
koala73 merged 2 commits into
mainfrom
feat/api-usage-tier-plan-attribution
Jul 1, 2026
Merged

fix(telemetry): attribute API-key usage events to tier + plan_key (#4572)#4573
koala73 merged 2 commits into
mainfrom
feat/api-usage-tier-plan-attribution

Conversation

@koala73

@koala73 koala73 commented Jun 30, 2026

Copy link
Copy Markdown
Owner

Closes #4572. Follow-up to #3199 / #4563, surfaced while running the shadow-mode limit-abuse audit.

Problem

The audit against Axiom wm_api_usage couldn't attribute an API-key request to its plan/caps:

  • usage.tier was only set on tier-gated routes, so API-key traffic on public routes emitted tier: 0 (56,988 user_api_key events at tier 0 vs 9,452 at tier 2 — the field was unreliable, even though every user_api_key holder is paid tier 2+).
  • Starter and Business are both tier: 2, so even with tier fixed they're indistinguishable — the audit needed an external Convex/Dodo lookup per customer.

Fix

In the #3199 gateway rate-limit block (where getEntitlements(sessionUserId) is already resolved):

  • Set usage.tier = ent.features.tier + usage.planKey = ent.planKey (recorded even for downgraded keys); enterprise env keys → tier 3 / plan_key 'enterprise'.
  • Thread a new plan_key field through UsageIdentity → RequestEvent → buildRequestEvent, mirroring auth_kind/customer_id.

Additive optional field — no Axiom schema migration. The audit can now group by customer_id + plan_key and compare each customer's peak rpm/day to their cap with zero external lookups.

Verification

  • 39 vitest tests pass (3 new buildUsageIdentity plan_key cases + existing gateway-wiring + entitlement suites unchanged).
  • tsc --noEmit, typecheck:api, biome lint all clean.
  • Diff is 50 insertions, purely additive — gateway edits only set telemetry fields; no change to rate-limiting decisions, eligibility, or responses.

Risk

Telemetry-only. Worst case a field is null on some path (handled — plan_key is nullable). No request-flow impact.

https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1

@koala73
fix(telemetry): attribute API-key usage events to tier + plan_key (#4572
2c78b19 
)

The limit-abuse audit (wm_api_usage) couldn't attribute API-key requests to a
plan: usage.tier was unset on non-tier-gated routes (emitting tier:0 for paid
keys), and Starter/Business share tier 2 so couldn't be told apart. Set
usage.tier + usage.planKey from the entitlement the #3199 rate-limit block
already resolves (enterprise -> tier 3 / plan_key 'enterprise'), and thread a
plan_key field through UsageIdentity -> RequestEvent so the audit can compare
each customer's rate to their actual cap with no external Convex lookup.
Additive (optional field, no Axiom migration). +3 buildUsageIdentity tests.

Claude-Session: https://claude.ai/code/session_01SjVX3GyMBmC2XyFWCHogN1
@koala73 koala73 added P2 Medium priority, schedule when capacity allows area: API Backend API, sidecar, keys labels Jun 30, 2026
@vercel

vercel Bot commented Jun 30, 2026 •
edited
Loading

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Jul 1, 2026 9:16am

Request Review

@greptile-apps

greptile-apps Bot commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes plan_key attribution for API-key telemetry events by threading a new nullable plan_key field from entitlement resolution through UsageIdentityInput → UsageIdentity → RequestEvent. Previously, all API-key traffic on public routes emitted tier: 0, making the limit-abuse audit unable to match requests to per-customer caps without external lookups.

  • server/gateway.ts: In the per-account rate-limit block (post-entitlement resolution), usage.tier and usage.planKey are now populated from ent.features.tier / ent.planKey for user API keys, and usage.tier = 3 is set for enterprise keys. Enterprise plan_key falls back to the literal 'enterprise' string via null-coalescing in buildUsageIdentity.
  • server/_shared/usage-identity.ts + usage.ts: Both the identity resolver and the event builder receive planKey as a new optional parameter, matching the existing pattern of auth_kind / customer_id.
  • Tests: Three new cases cover user_api_key plan propagation, enterprise default, and the anon null path.

Confidence Score: 5/5

Purely additive telemetry change; no rate-limiting decisions, eligibility logic, or response paths are touched. Worst case is a null field in Axiom.

The diff is 50 lines, all additive. The new plan_key field is nullable throughout, so missing entitlements degrade gracefully to null. The gateway enrichment happens after the existing getEntitlements call, so no new async work is introduced. Enterprise tier is hardcoded to 3 (consistent with the existing ENTERPRISE_API_RATE_LIMIT pattern), and the enterprise plan string fallback is covered by a dedicated test. TypeScript strict mode and three targeted vitest cases validate the wiring end-to-end.

No files require special attention.

Important Files Changed

Filename Overview
server/_shared/usage-identity.ts Adds optional planKey to UsageIdentityInput and plan_key to UsageIdentity; each auth branch wires it through correctly, with the enterprise fallback defaulting to 'enterprise' via null-coalescing.
server/_shared/usage.ts Adds plan_key: string
server/gateway.ts Enriches the usage object with tier and planKey inside the per-account rate-limit block (after entitlement resolution), and passes planKey through to buildRequestEvent. Enterprise branch explicitly sets tier=3; user-API-key branch reads both fields from the resolved entitlement.
server/tests/usage-identity.test.ts Adds 3 focused tests covering user_api_key plan propagation, enterprise default, and anon null case; base fixture updated with planKey: null to match the new interface field.

Reviews (1): Last reviewed commit: "fix(telemetry): attribute API-key usage ..." | Re-trigger Greptile

@koala73
koala73 merged commit 9de27d1 into main Jul 1, 2026
24 checks passed
@koala73
koala73 deleted the feat/api-usage-tier-plan-attribution branch July 1, 2026 09:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area: API Backend API, sidecar, keys P2 Medium priority, schedule when capacity allows

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

fix(telemetry): attribute API-key usage events to tier + plan so the limit-abuse audit is self-serve (#3199 follow-up)

1 participant

@koala73