fix(seed-imf): auth-optional Ocp-Apim-Subscription-Key + 4xx-non-retryable guard#3635
Conversation
…yable guard
seed-bundle-market-backup SIGTERM'd at 180s on 2026-05-09 because every IMF
IRFCL request 401'd and `withRetry` exponential backoff burned the whole
budget. Verified end-to-end:
* IMF migrated api.imf.org onto Azure APIM (~2025-Q1) and is rolling out
hard 401 enforcement intermittently — captured a multi-hour enforcement
window today, then verified rollback within ~3h. Need code that survives
both states.
* The gateway's `WWW-Authenticate: Bearer` 401 is misleading; actual scheme
is `Ocp-Apim-Subscription-Key` (verified: Bearer header → 401, APIM key
header → 200, query param `?subscription-key=` also → 200).
Changes:
* scripts/_seed-utils.mjs — new `imfAuthHeaders()` returns {} when
IMF_API_KEY unset (preserves today's anonymous-pass-through) and
{Ocp-Apim-Subscription-Key: …} when set (forward-compat with permanent
enforcement). One subscription unlocks both SDMX 2.1 and 3.0.
* scripts/_seed-utils.mjs — `withRetry` short-circuits on
`err.nonRetryable === true`. Without this, the next enforcement window
will SIGTERM the bundle again even with a valid key (a typo or quota
exhaust = 401 = retry storm).
* scripts/_seed-utils.mjs::imfSdmxFetchIndicator — sends auth header,
tags 4xx as nonRetryable. Covers 6 WEO seeders (growth, macro, labor,
external, national-debt, recovery-fiscal-space).
* scripts/seed-gold-cb-reserves.mjs::fetchIrfclMonthlySeries — same
treatment. Covers the seeder whose SIGTERM surfaced this incident.
* .env.example — IMF_API_KEY (optional, recommended) with portal URL.
Verified:
* 37/37 existing tests pass (gold-cb-reserves, seed-imf-extended,
seed-utils-sigterm-cleanup).
* Live fetch w/o env → 200, 111 IRFCL series in 1.9s (today's state).
* Live fetch w/ env → 200, 208 WEO countries in 1.5s (forward-compat).
* Missing-env path returns {} (no nonRetryable throw — correct relaxed
behavior).
Operator action: register at https://portal.api.imf.org/ → Products → IMF
Data SDMX API → Subscribe → Profile → Subscriptions → copy Primary key →
set IMF_API_KEY on every Railway service that runs an IMF-touching bundle
(seed-bundle-market-backup + whichever runs WEO seeders). Free tier, 5 min,
no IMF approval queue. Setting it is forward-compat; not setting it leaves
us exposed when IMF flips enforcement on permanently.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Greptile SummaryThis PR adds forward-compatible IMF APIM authentication (
Confidence Score: 3/5The auth-header and fast-fail logic are sound, but the 4xx nonRetryable guard incorrectly covers 429 in both fetch paths, which would cause the seeders to give up immediately on any rate-limit response rather than retrying with backoff. Both IMF fetch functions mark the entire 4xx band as nonRetryable, including 429 Too Many Requests. A rate-limited response from the APIM gateway would immediately abort all retry attempts — the exact failure mode this PR is trying to solve for auth errors, now re-introduced for rate limits. The fix is a one-liner in each place, but it affects both changed fetch paths and the shared withRetry behaviour that all future callers will inherit. Both scripts/_seed-utils.mjs and scripts/seed-gold-cb-reserves.mjs carry the same r.status >= 400 && r.status < 500 guard that needs the 429 carve-out before merge. Important Files Changed
Reviews (1): Last reviewed commit: "fix(seed-imf): auth-optional Ocp-Apim-Su..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| if (!r.ok) { | ||
| const err = new Error(`IMF SDMX ${indicator}: HTTP ${r.status}`); | ||
| if (r.status >= 400 && r.status < 500) err.nonRetryable = true; | ||
| throw err; | ||
| } |
There was a problem hiding this comment.
>= 400 && < 500 range silently includes 429 Too Many Requests, which is a transient condition — the whole point of withRetry with backoff. If the IMF APIM gateway starts rate-limiting (a realistic scenario as adoption grows), every request would be tagged nonRetryable and fail immediately on the first attempt, discarding the retry budget the same way a 401 would. The fix that matches the PR's stated intent (guard against auth/permission failures) is to carve out 429.
| if (!r.ok) { | |
| const err = new Error(`IMF SDMX ${indicator}: HTTP ${r.status}`); | |
| if (r.status >= 400 && r.status < 500) err.nonRetryable = true; | |
| throw err; | |
| } | |
| if (!r.ok) { | |
| const err = new Error(`IMF SDMX ${indicator}: HTTP ${r.status}`); | |
| if (r.status >= 400 && r.status < 500 && r.status !== 429) err.nonRetryable = true; | |
| throw err; | |
| } |
There was a problem hiding this comment.
Already addressed in this same commit. Line 707 of _seed-utils.mjs (HEAD = d91f790) reads if (PERMANENT_4XX_STATUSES.has(r.status)) err.nonRetryable = true; — the allowlist is {400, 401, 403, 404, 410, 422, 451}, which excludes 408 and 429.
Looks like Greptile reviewed the prior commit (74de321, which had the broad r.status >= 400 && r.status < 500 predicate) and re-anchored to the latest SHA. tests/seed-utils-with-retry.test.mjs includes an explicit regression assertion (PERMANENT_4XX_STATUSES.has(408) === false and PERMANENT_4XX_STATUSES.has(429) === false) citing this review as the guard. 429 + 503 also attach retryAfterMs so withRetry honors upstream Retry-After. Closing as resolved.
| if (!r.ok) { | ||
| const err = new Error(`IMF IRFCL ${indicator}: HTTP ${r.status}`); | ||
| if (r.status >= 400 && r.status < 500) err.nonRetryable = true; | ||
| throw err; | ||
| } |
There was a problem hiding this comment.
imfSdmxFetchIndicator — fetchIrfclMonthlySeries will also immediately bail on a rate-limit response rather than retrying with backoff.
| if (!r.ok) { | |
| const err = new Error(`IMF IRFCL ${indicator}: HTTP ${r.status}`); | |
| if (r.status >= 400 && r.status < 500) err.nonRetryable = true; | |
| throw err; | |
| } | |
| if (!r.ok) { | |
| const err = new Error(`IMF IRFCL ${indicator}: HTTP ${r.status}`); | |
| if (r.status >= 400 && r.status < 500 && r.status !== 429) err.nonRetryable = true; | |
| throw err; | |
| } |
There was a problem hiding this comment.
Same as the _seed-utils.mjs thread — line 81 here already uses PERMANENT_4XX_STATUSES.has(r.status) (commit d91f790), and on 429/503 attaches retryAfterMs from the Retry-After header. Greptile re-anchored a stale review against the post-fix SHA.
PR #3635 review (P1): the blanket `r.status >= 400 && r.status < 500` predicate marked 408 Request Timeout and 429 Too Many Requests as nonRetryable. Both are explicit "back off and retry" signals — for IMF's APIM gateway, 429 is the most likely status under the parallel WEO indicator fetches in seed-imf-{growth,macro,labor,external}. The previous guard would have converted a transient rate-limit into an immediate seed failure. Changes: * scripts/_seed-utils.mjs — new `PERMANENT_4XX_STATUSES` allowlist (400/401/403/404/410/422/451). Only these tag nonRetryable. 408/429 fall through to normal retry. 5xx already retryable (unchanged). * scripts/_seed-utils.mjs — new `parseRetryAfterMs` helper (mirrors the helper already duplicated in _yahoo-fetch / _gdelt-fetch / _open-meteo-archive — those existing copies left untouched per surgical-scope; consolidate-the-3-old-copies is a separate refactor). Caps hint at 60s so a stuck/abusive header can't park the bundle. * scripts/_seed-utils.mjs — `withRetry` honors `err.retryAfterMs` when caller attaches it. Takes max(baseWait, retryAfterMs) so a short hint doesn't undercut backoff and a long hint isn't ignored. * scripts/_seed-utils.mjs::imfSdmxFetchIndicator AND scripts/seed-gold-cb-reserves.mjs::fetchIrfclMonthlySeries — narrow the nonRetryable predicate to PERMANENT_4XX_STATUSES; on 429/503 attach `retryAfterMs = parseRetryAfterMs(r.headers.get('retry-after'))`. * tests/seed-utils-with-retry.test.mjs — new dedicated test file: - PERMANENT_4XX_STATUSES contains expected codes - 408/429 EXCLUDED (explicit regression guard for the review finding) - 5xx EXCLUDED - parseRetryAfterMs: seconds, HTTP-date, null cases, 60s cap, 1000ms floor for "0"/past-date hints (matches yahoo helper) - withRetry: nonRetryable short-circuit, exponential backoff for plain errors, success on first try, retryAfterMs honored Verified: * 49/49 affected tests pass (new file + gold-cb-reserves + seed-imf-extended + seed-utils-sigterm-cleanup). * Live: 208 WEO countries fetched in 1.5s with key set. * The retryAfterMs-honor test takes ~200ms confirming the upstream hint actually overrides the default backoff (would be ~1ms otherwise).
1 participant
Summary
seed-bundle-market-backupSIGTERM'd at 180s on 2026-05-09 because every IMF IRFCL request 401'd andwithRetry's exponential backoff burned the whole bundle budget. Investigation revealed two compounding issues — fixing both:api.imf.orgonto Azure APIM (~2025-Q1) and is rolling out hard 401 enforcement intermittently. We captured a multi-hour enforcement window today (verified across three IPs incl. production), then verified the rollback within ~3 hours. Code needs to survive both states.withRetryhad no concept of permanent failures — every 4xx (auth, permission, missing config) burned the full retry budget exactly like a transient 5xx. Without anonRetryableshort-circuit, the next enforcement window will SIGTERM the bundle again even with a valid key.The gateway's
WWW-Authenticate: Bearer401 is a misdirection — actual auth scheme isOcp-Apim-Subscription-Key(verified: Bearer → 401, APIM key → 200, query param?subscription-key=also → 200). One subscription unlocks both SDMX 2.1 and 3.0.Changes
scripts/_seed-utils.mjsimfAuthHeaders()— returns{}whenIMF_API_KEYunset (preserves today's anonymous-pass-through),{Ocp-Apim-Subscription-Key: …}when set (forward-compat with permanent enforcement).withRetryshort-circuits onerr.nonRetryable === truebefore sleeping. Generalises across all seeders.imfSdmxFetchIndicatorsends auth header + tags 4xx asnonRetryable. Covers 6 WEO seeders (growth, macro, labor, external, national-debt, recovery-fiscal-space).scripts/seed-gold-cb-reserves.mjsfetchIrfclMonthlySeriessends auth header + tags 4xx. Covers the seeder whose SIGTERM surfaced this incident.// public, no authheader comment fixed..env.example—IMF_API_KEY(optional, recommended) with portal URL.Why auth-optional rather than required
IMF currently allows anonymous requests. Hard-throwing on missing
IMF_API_KEYwould instantly break every IMF seeder right now (they're working). Sending the header opportunistically when the env is set is forward-compatible without regressing today's state. ThenonRetryable4xx guard is the safety net for whenever IMF flips enforcement on permanently.Operator action required (separate from merge)
Set
IMF_API_KEYon every Railway service that runs an IMF-touching bundle:seed-bundle-market-backup(gold-cb-reserves)seed-bundle-macroand siblings —seed-imf-{growth,macro,labor,external},seed-national-debt,seed-recovery-fiscal-space)Get the key at https://portal.api.imf.org/ → Sign in → Products → IMF Data SDMX API → Subscribe → Profile → Subscriptions → Primary key. Free tier, 5 min, no IMF approval queue.
Test plan
gold-cb-reserves,seed-imf-extended,seed-utils-sigterm-cleanup)npm run test:data){}(nononRetryablethrow — correct relaxed behavior)nonRetryable, fails in <1s instead of burning the retry budgetnpm run typecheck,typecheck:api,lint,lint:md,version:check, edge bundle, edge-functions test — all passIMF_API_KEYon Railway services listed above, then watch one cycle ofseed-bundle-market-backupto confirm the section completes in <10s instead of the prior 180s SIGTERM