Summary
seed-bigmac fires Railway "Deploy Crashed!" alerts on a recurring basis. The crash is classified GRACEFUL (exit 75, last-good TTL extended, no data lost) — but it is not a random upstream blip. It is a structural fragility: the per-country Exa search loop is sequential and mathematically guaranteed to breach the runner's fetch-phase deadline whenever Exa latency is elevated.
Evidence (live, 2026-07-07, deployment 6fc174d4-d440-408e-889f-bb552acca84e)
FETCH FAILED: economic:bigmac fetch phase exceeded 240000ms deadline (likely a non-settling await — see issue #4786)
Extended TTL on 3 key(s) (864000s)
=== Failed gracefully (240503ms) ===
Processing 🇿🇦 South Africa (ZAR)...
Big Mac: 54.9 ZAR = \$3.3763
Processing 🇳🇬 Nigeria (NGN)...
The run burned the full 240.5s and was killed mid-loop (still processing countries).
Root cause
scripts/seed-bigmac.mjs:
fetchBigMacPrices() (line 132) loops sequentially over COUNTRIES — 50 countries (line 136: for (const country of COUNTRIES)), awaiting searchExa() each iteration. No Promise.all, no concurrency pool.
searchExa() (line 105) does one fetch('https://api.exa.ai/search', { signal: AbortSignal.timeout(15_000) }) — a 15s per-call ceiling.
- The whole phase runs under
runSeed(...)'s 240 000 ms deadline (line 272).
The math: 50 sequential calls under a 240s ceiling means the average per-country latency only has to exceed 240s ÷ 50 = 4.8s to breach. A partial Exa slowdown (calls drifting toward their 15s timeout) breaches it deterministically. This is the same non-settling/slow-fetch class flagged in #4786, but here the outer deadline guard converts the hang into a whole-run crash + operator alert instead of exit-13.
Why "graceful" is the wrong disposition
Even though no data is lost (TTL extended), each occurrence:
- Fires a Railway "Deploy Crashed!" notification → operator alert fatigue.
- Burns up to 240s of compute doing nothing useful.
- Is invisible in
/api/health (seed-meta stays fresh via the preserve-last-good path), so the only signal is the crash badge the operator is told to ignore.
Proposed fix
Convert the whole-run crash into localized partial-data so the run exits 0:
- Run the 50-country loop with bounded concurrency (e.g. pool of 5–8) + a per-country time budget, so total wall-clock stays well under 240s even under Exa latency.
- Treat an individual country's Exa failure/timeout as a per-country skip (keep prior snapshot value for that country), not a whole-run abort.
- Optionally lower the outer 240s guard once the loop is bounded, so a genuine hang is caught faster.
Result: partial refreshes succeed and exit 0 (no crash alert); only a total-Exa-outage would trip the graceful path.
Meta
Filed off the /diagnose-railway-seeders review — the skill classified this GRACEFUL / "action: None", which is the disposition being re-examined. A recurring graceful crash that alerts the operator is a bug to fix, not noise to ignore.
Summary
seed-bigmacfires Railway "Deploy Crashed!" alerts on a recurring basis. The crash is classified GRACEFUL (exit 75, last-good TTL extended, no data lost) — but it is not a random upstream blip. It is a structural fragility: the per-country Exa search loop is sequential and mathematically guaranteed to breach the runner's fetch-phase deadline whenever Exa latency is elevated.Evidence (live, 2026-07-07, deployment
6fc174d4-d440-408e-889f-bb552acca84e)The run burned the full 240.5s and was killed mid-loop (still processing countries).
Root cause
scripts/seed-bigmac.mjs:fetchBigMacPrices()(line 132) loops sequentially overCOUNTRIES— 50 countries (line 136:for (const country of COUNTRIES)), awaitingsearchExa()each iteration. NoPromise.all, no concurrency pool.searchExa()(line 105) does onefetch('https://api.exa.ai/search', { signal: AbortSignal.timeout(15_000) })— a 15s per-call ceiling.runSeed(...)'s 240 000 ms deadline (line 272).The math: 50 sequential calls under a 240s ceiling means the average per-country latency only has to exceed 240s ÷ 50 = 4.8s to breach. A partial Exa slowdown (calls drifting toward their 15s timeout) breaches it deterministically. This is the same non-settling/slow-fetch class flagged in #4786, but here the outer deadline guard converts the hang into a whole-run crash + operator alert instead of exit-13.
Why "graceful" is the wrong disposition
Even though no data is lost (TTL extended), each occurrence:
/api/health(seed-meta stays fresh via the preserve-last-good path), so the only signal is the crash badge the operator is told to ignore.Proposed fix
Convert the whole-run crash into localized partial-data so the run exits 0:
Result: partial refreshes succeed and exit 0 (no crash alert); only a total-Exa-outage would trip the graceful path.
Meta
Filed off the
/diagnose-railway-seedersreview — the skill classified this GRACEFUL / "action: None", which is the disposition being re-examined. A recurring graceful crash that alerts the operator is a bug to fix, not noise to ignore.