fix(sentry-triage): IndexedDB availability guard + minified-new noise filter#3683
Conversation
…or noise filter
Two Sentry production issues addressed:
1. ReferenceError: indexedDB is not defined at /about (5 events /
1 user, WORLDMONITOR-7479081164). Our panels-*.js bundle called
indexedDB.open(...) from src/services/storage.ts:initDB and
src/workers/vector-db.ts:openDB without checking availability.
Private-browsing modes and some embedded webviews disable IndexedDB.
Fix: probe typeof indexedDB === 'undefined' up-front in both
modules. In storage.ts, throw a new named
IndexedDBUnavailableError; withTransaction catches it and
degrades gracefully — reads return undefined, writes throw the
typed error so callers can opt into different handling.
vector-db.ts rejects the open Promise with a clear error message.
2. undefined is not a constructor (evaluating 'new <1-2-char-var>')
at / (2 events / 1 user, WORLDMONITOR-7477976823). Classic
minified-bundle injection signature: 1 frame in main-*.js with
function: null, no first-party source-mapped frame. Likely
userscript / extension calling a constructor on a stripped name.
Fix: new beforeSend rule, gated on !hasFirstParty AND a 1-2-char
minified name (mirrors the existing Can't find variable: \w{1,2}
pattern at main.ts:339). Bounded length prevents masking a real
new SomeBigClassName call from our own code.
Verified:
- npm run typecheck: PASS
- npm run typecheck:api: PASS
- npm run lint: PASS (no new warnings on changed files)
- npm run test:data: PASS (8699/8699)
- npm run lint:md / version:check / edge bundle checks: PASS
NOTE: A third Sentry issue (WORLDMONITOR-7478486721,
Failed to fetch (clerk.worldmonitor.app) with chrome-extension frame
mid-stack) was investigated and intentionally NOT filtered. The
existing test tests/sentry-beforesend.test.mjs:463 enforces the safety
property that first-party + extension frames must surface — broadening
the filter would silence real api/clerk.worldmonitor.app outages for
users running fetch-wrapping extensions. Issue re-opened in Sentry.
Two other Sentry issues (WORLDMONITOR-7479324527 / 7479324546, Convex
OCC retry exhaustion on _markContactPushed) were resolved-in-next-
release. They were caused by an unauthorized discardWaveRun mid-push
during the wave-10 incident on 2026-05-13 (operator did not approve
that wave; discarded before any emails sent). Not a recurring bug; will
auto-reopen if a future legitimate operator-initiated mid-push discard
re-triggers, in which case _markContactPushed would need OCC-during-
discard handling as a follow-up.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR fixes two production Sentry issues: a
Confidence Score: 4/5Safe to merge — changes are small, well-scoped defensive additions that address real production crashes without touching business logic. The No file requires urgent attention; Important Files Changed
Flowchart%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[initDB called] --> B{db already open?}
B -- yes --> C[return cached db]
B -- no --> D{typeof indexedDB === 'undefined'?}
D -- yes --> E[throw IndexedDBUnavailableError]
D -- no --> F[indexedDB.open ...]
F --> G[resolve db]
H[withTransaction called] --> I[await initDB]
I --> J{error thrown?}
J -- InvalidStateError --> K{attempt 0?}
K -- yes --> L[reset db=null, retry]
K -- no --> M{mode readwrite?}
M -- yes --> N[throw DOMException]
M -- no --> O[return undefined]
J -- IndexedDBUnavailableError --> P{mode readwrite?}
P -- yes --> Q[re-throw typed error]
P -- no --> R[silently return undefined]
J -- other --> S[re-throw]
T[vector-db openDB called] --> U{db already open?}
U -- yes --> V[return cached db]
U -- no --> W{typeof indexedDB === 'undefined'?}
W -- yes --> X[Promise.reject plain Error]
W -- no --> Y[indexedDB.open ...]
Y --> Z[resolve db]
Reviews (1): Last reviewed commit: "fix(sentry-triage): IndexedDB availabili..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (typeof indexedDB === 'undefined') { | ||
| return Promise.reject(new Error('IndexedDB unavailable in this worker context')); | ||
| } |
There was a problem hiding this comment.
storage.ts guard
openDB rejects with a plain new Error(...) while storage.ts exports and uses the named IndexedDBUnavailableError class. Any caller that wants to distinguish unavailability from other IDB failures (quota exceeded, schema errors, etc.) cannot do so with an instanceof check here. The comments say this mirrors the storage.ts guard, but the error type diverges — if enqueue-wrapped callers (or the main thread receiving the rejected promise) ever need typed handling, a plain Error won't match instanceof IndexedDBUnavailableError.
There was a problem hiding this comment.
Addressed in 6320785. Added a parallel IndexedDBUnavailableError class in vector-db.ts matching storage.ts's shape (mirrored, not imported — worker bundle can't safely pull from main-thread src/services/). Callers can now instanceof-check unavailability consistently across both modules.
| // bundle's variable name. Bounded to 1-2 chars to avoid masking a real | ||
| // `new SomeBigClassName` call that our own code makes | ||
| // (WORLDMONITOR-7477976823). | ||
| if (!hasFirstParty && /^undefined is not a constructor \(evaluating 'new \w{1,2}'\)$/.test(msg)) return null; |
There was a problem hiding this comment.
frames.length === 0 guard present in the analogous rule
The PR description says this "mirrors existing Can't find variable: \w{1,2} pattern", but the existing rule on the line immediately above requires frames.length === 0 in addition to !hasFirstParty. The new rule only checks !hasFirstParty, meaning it will also suppress events where third-party (extension/userscript) frames appear in the stack. For the Can't find variable case, an empty stack is a meaningful signal of extension injection with no trace; for undefined is not a constructor, extension frames may legitimately appear in the stack, so the omission could be intentional — but worth confirming that WORLDMONITOR-7477976823 events had non-empty third-party stacks and that the broader suppression scope is desired.
There was a problem hiding this comment.
Rule dropped entirely in 6320785 after the review prompted a re-audit that exposed it as a false fix.
The rule was gated on !hasFirstParty. The Sentry event (WORLDMONITOR-7477976823) had a single frame: /assets/main-nAfRXGNy.js. firstPartyFile (main.ts:296-300) matches /assets/<name>-<hash>.js as first-party unless it's a vendor chunk, and main-*.js is not in the vendor list → hasFirstParty was TRUE → my !hasFirstParty-gated rule never fired on the issue it claimed to suppress.
So the rule was suppressing nothing. No tests are needed for a rule that no longer exists; Sentry 7477976823 has been re-opened to track the real noise honestly. If we ever reinstate a filter for this pattern it would need different discrimination (function-name-null + single-frame, context-line based, etc.) — too speculative for 2 events / 1 user.
Note: the absolute path in your comment (/private/tmp/worldmonitor-pr3683-review/src/main.ts:367) indicates the review was done against commit 45aa2aa (the first commit on this PR). My latest push 6320785 dropped the rule entirely; git show origin/fix/sentry-triage-batch-1:src/main.ts confirms it's no longer present.
…ss name
Two greptile P2 review findings addressed:
1. Drop the undefined is not a constructor (evaluating 'new <X>')
filter from beforeSend. Investigation triggered by the review
comment exposed the filter as a false fix:
- The new rule was gated on !hasFirstParty.
- The Sentry event (WORLDMONITOR-7477976823) had ONE frame:
/assets/main-nAfRXGNy.js.
- firstPartyFile regex (main.ts:296-300) matches
/assets/<name>-<hash>.js as first-party UNLESS it's a vendor
chunk (maplibre/d3/sentry/etc.).
- main-nAfRXGNy.js matches the asset pattern and is NOT in the
vendor list -> hasFirstParty is TRUE -> my rule never fires.
The filter was suppressing nothing for that issue. Removing it
reverts the false-fix; re-opening 7477976823 in Sentry restores
accurate noise tracking. If the rule were to be reinstated, it
would need different gating (e.g., function name === null AND
single-frame, or context-line-based discrimination), which is too
speculative for two events / one user.
2. vector-db.ts now uses a named IndexedDBUnavailableError class
matching storage.ts's shape (mirrored, not imported - worker
bundle context can't safely pull from main-thread src/services/).
Callers can now instanceof-check unavailability vs other IDB
failures (quota, schema) in both modules consistently.
Verified:
- npm run typecheck: PASS
- node --test tests/sentry-beforesend.test.mjs: 0 fail
Sentry 7477976823 re-opened.
Greptile PR #3694 review: P2 — Signal-abort skipped the sleep but kept the loop running. Pre-fix: `if (batchIdx < batches && !signal?.aborted) await sleep(...)` correctly skipped the 5s sleep on abort, but the for-loop continued to the next iteration and started a fresh batch of up to 6 concurrent fetches via withPerCountryTimeout (which creates its own AbortController not linked to the parent signal). Net effect: the actual SIGTERM backstop was onSigterm → process.exit(1), not the abort guard. Fix: `if (signal?.aborted) break;` before the sleep — exits the loop immediately so SIGTERM doesn't start additional in-flight work. P2 — Test comment cited wrong PR number (#3683 vs #3694). The "Halved from 12 → 6 on 2026-05-14 (PR #3683)" comment in the CONCURRENCY test referenced a non-existent PR in this series. Fixed to #3694 so a future reviewer following the comment lands on the right diff. Tests: 87/87 still pass. Updated the BATCH_BACKOFF_MS test to assert both the new `break` shape and the simplified sleep condition; the previous combined `batchIdx < batches && !signal?.aborted` pattern is now split across the two checks.
…ackoff + temp gate (#3694) * fix(seed-portwatch): rate-limit recovery — concurrency 12→6 + batch backoff + temporary gate Option E from the 2026-05-14 incident triage: combined response to the ongoing ArcGIS degradation where both direct AND proxy paths are throttled. Background (post-#3676 + #3681 deployed): Run #1 (19:55 UTC May 13): 24/30 proxy successes, gate fails 24<50 Run #2 (00:03 UTC May 14): 5/30 proxy successes, gate fails 5<50 Both #3676 (cold-fetch cap) and #3681 (proxy fallback on timeout) are working as designed. The remaining problem: Decodo proxy is now also being rate-limited — successive runs degrade as our usage spikes the proxy's bucket. ArcGIS itself appears degraded for this dataset. Three coordinated changes: (B) CONCURRENCY 12 → 6 Halve in-flight fetches so neither ArcGIS-direct nor Decodo-proxy sees a 12-concurrent burst. Math at cold-fetch cap 30: 5 batches × ~60s realistic + 4×5s backoff ≈ 320s 5 batches × ~90s worst case + 4×5s backoff ≈ 470s Both fit the 570s bundle budget. (C) BATCH_BACKOFF_MS = 5_000 (new) Sleep 5s between batches (skip on last, skip on signal-abort to preserve SIGTERM responsiveness). Spaces out per-batch bursts so neither service hits its rate-limit window from our run alone. 20s total added — negligible against the 570s budget. (Temporary) MIN_VALID_COUNTRIES 50 → 25 Coverage gate lowered so partial-success runs (5-25 successful fetches + stale-served from cache) can advance seed-meta. Pre-fix, seed-meta was frozen at 2026-05-12 00:03 UTC for 60+ hours because no run reached 50. The 25 floor is sized so a 5-success run still fails (real outage) but a 25+ partial-success run advances meta and clears the operator-facing WARNING. Marked TEMPORARY in the inline comment with a 2026-05-20 review target — must revert to 50 once ArcGIS recovers and runs reliably exceed 50 again. Test enforces the comment shape so a future reviewer can't silently normalise 25 as the new permanent floor. Tests: 87/87 pass (was 85, +2 for BATCH_BACKOFF + MIN_VALID_COUNTRIES invariants; +1 modification for CONCURRENCY value). This week's portwatch series: #3676 Structural cold-fetch cap (merged) #3681 Proxy retry on timeout (merged) This Rate-limit recovery (concurrency + backoff + gate) * review: break loop on signal-abort + fix wrong PR number in test (P2×2) Greptile PR #3694 review: P2 — Signal-abort skipped the sleep but kept the loop running. Pre-fix: `if (batchIdx < batches && !signal?.aborted) await sleep(...)` correctly skipped the 5s sleep on abort, but the for-loop continued to the next iteration and started a fresh batch of up to 6 concurrent fetches via withPerCountryTimeout (which creates its own AbortController not linked to the parent signal). Net effect: the actual SIGTERM backstop was onSigterm → process.exit(1), not the abort guard. Fix: `if (signal?.aborted) break;` before the sleep — exits the loop immediately so SIGTERM doesn't start additional in-flight work. P2 — Test comment cited wrong PR number (#3683 vs #3694). The "Halved from 12 → 6 on 2026-05-14 (PR #3683)" comment in the CONCURRENCY test referenced a non-existent PR in this series. Fixed to #3694 so a future reviewer following the comment lands on the right diff. Tests: 87/87 still pass. Updated the BATCH_BACKOFF_MS test to assert both the new `break` shape and the simplified sleep condition; the previous combined `batchIdx < batches && !signal?.aborted` pattern is now split across the two checks. * review: bypass 80% degradation guard in cap-mode (P1) Greptile PR #3694 round 3 P1: with the temp coverage gate lowered to 25, a cap-mode partial-success run would CLEAR the coverage gate (countryData.size ≥ 25) but STILL fail the unchanged degradation guard (countryData.size < prevCount × 0.8 ≈ 139 in the incident state) — seed-meta never advances, WARNING persists, PR's main recovery claim broken. Math at the current incident state: prevCount 174 degradation threshold 0.8 × 174 = 139 cap-mode realistic 30 cold-fetch + ~24 stale-served = ~54 → 54 << 139 → DEGRADATION GUARD FAILS → extendTtl + return → seed-meta stays frozen, WARNING persists. Fix shape: signal cap-mode from fetchAll() back to main() via a new `capTriggered: bool` field plus `servedStaleCount` / `droppedTooOldCount` / `droppedNoCacheCount` counters. main() bypasses the 80% guard for cap-mode runs (intentional partial coverage by design — see #3676's rotation contract) and logs a `PARTIAL PUBLISH (cap-mode)` line with the fresh/stale split. The bypass is scoped: non-cap runs (where needsFetch ≤ 30 and we fetched all misses normally) STILL apply the 80% guard so silent data-loss scenarios (ArcGIS regression silently drops 100 → 50 countries) remain caught. Logic shape: if (!validateFn) → extendTtl + return [coverage gate] if (capTriggered) → log PARTIAL PUBLISH + fall through [bypass] else if (countryData.size < prevCount × 0.8) → extendTtl + return [silent-loss guard] → publish + advance seed-meta Tests (3 new): fetchAll-shape (returns the 4 new fields), bypass shape (else-if structure ensures cap-mode skips the guard), bypass exclusivity (non-cap runs still enforce the guard with exactly 2 references to `prevCount × 0.8` — one in condition, one in error msg). 90/90 pass (was 87).
1 participant
Summary
Two Sentry production issues fixed, one investigated-and-deliberately-not-filtered, two resolved-in-next-release (incident artifacts).
ReferenceError: indexedDB is not definedat/about(5 events / 1 user)storage.ts+vector-db.ts; newIndexedDBUnavailableErrorclass;withTransactiondegrades reads silently / surfaces writes as the typed errorundefined is not a constructor (evaluating 'new f')at/(2 events / 1 user)beforeSendrule gated on!hasFirstPartyAND 1-2-char minified name (mirrors existingCan't find variable: \w{1,2}pattern)Failed to fetch (clerk.worldmonitor.app)w/ chrome-extension mid-stacktests/sentry-beforesend.test.mjs:463which guards the invariant that first-party + extension frames must surface. Broadening would silence real api.worldmonitor.app outages for users running fetch-wrapping extensions. Accept 3 events / 1 user as noise._markContactPusheddiscardWaveRunmid-push during the wave-10 incident on 2026-05-13. Not a recurring bug. If a future legitimate operator-initiated mid-push discard re-triggers this, the right fix is OCC-during-discard handling in_markContactPushed(follow-up).Files changed
src/services/storage.ts—IndexedDBUnavailableErrorclass + initDB guard +withTransactioncatch branchsrc/workers/vector-db.ts— parallel guard in workeropenDBsrc/main.ts— one newbeforeSendrule for the minifiednew <X>pattern37 insertions across 3 files.
Test plan
npm run typecheckcleannpm run typecheck:apiclean (+ Convex audit)npm run lintclean (no new warnings on changed files; 235 pre-existing warnings unchanged)npm run test:data— 8699 / 8699 pass (initial filter broke 1 test; reverted; all green)tests/edge-functions.test.mjs— 184 / 184 passnpm run lint:md— 0 errorsnpm run version:check— synced (2.8.0)Why no broader extension-fetch filter
tests/sentry-beforesend.test.mjs:463documents the invariant:The Sentry issue
Failed to fetch (clerk.worldmonitor.app)has both first-party panels-*.js AND chrome-extension frames in the stack. Per the existing invariant, this is the "could be a real clerk outage" case — surface it, don't suppress. Low volume (3 events / 1 user) supports keeping the noise rather than risking blindness on real auth-host failures.