Batch market and RSS fetching with progressive updates#1
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
koala73
added a commit
that referenced
this pull request
Jan 18, 2026
… freshness Assessment & Documentation: - Add docs/GEOPOLITICAL_ASSESSMENT.md with full platform analysis - Strategic improvement roadmap with prioritized recommendations Quick Win #1 - Data Freshness (intelligence gaps): - Add getIntelligenceGaps() and getIntelligenceGapSummary() to data-freshness.ts - Add human-readable messages explaining what analysts CAN'T see - Add hasCriticalGaps() for alert integration Quick Win #2 - Escalation Scores: - Add escalationScore (1-5), escalationTrend, escalationIndicators to Hotspot type - Update 11 major hotspots with scores (Sahel, Haiti, Horn of Africa, Moscow, Beijing, Kyiv, Taipei, Tehran, Tel Aviv, Pyongyang, Sana'a) Quick Win #3 - Signal Context ("Why It Matters"): - Add SIGNAL_CONTEXT with whyItMatters, actionableInsight, confidenceNote - Add getSignalContext() helper for all 10 signal types - Explains analytical significance of each signal type Quick Win #4 - Historical Context: - Add HistoricalContext interface with lastMajorEvent, precedentCount, cyclicalRisk fields - Add whyItMatters field to Hotspot type - Update major hotspots with historical precedents and geopolitical significance Quick Win #5 - Propaganda Risk Flags: - Add PropagandaRisk type and SourceRiskProfile interface - Add SOURCE_PROPAGANDA_RISK mapping for state media (Xinhua, TASS, RT, CGTN) - Add getSourcePropagandaRisk() and isStateAffiliatedSource() helpers - Flag medium-risk state-affiliated sources (Al Jazeera, France 24, DW, etc.)
This was referenced Feb 26, 2026
facusturla
pushed a commit
to facusturla/worldmonitor
that referenced
this pull request
Feb 27, 2026
…i-fetching-for-stocks Batch market and RSS fetching with progressive updates
facusturla
pushed a commit
to facusturla/worldmonitor
that referenced
this pull request
Feb 27, 2026
… freshness Assessment & Documentation: - Add docs/GEOPOLITICAL_ASSESSMENT.md with full platform analysis - Strategic improvement roadmap with prioritized recommendations Quick Win koala73#1 - Data Freshness (intelligence gaps): - Add getIntelligenceGaps() and getIntelligenceGapSummary() to data-freshness.ts - Add human-readable messages explaining what analysts CAN'T see - Add hasCriticalGaps() for alert integration Quick Win koala73#2 - Escalation Scores: - Add escalationScore (1-5), escalationTrend, escalationIndicators to Hotspot type - Update 11 major hotspots with scores (Sahel, Haiti, Horn of Africa, Moscow, Beijing, Kyiv, Taipei, Tehran, Tel Aviv, Pyongyang, Sana'a) Quick Win koala73#3 - Signal Context ("Why It Matters"): - Add SIGNAL_CONTEXT with whyItMatters, actionableInsight, confidenceNote - Add getSignalContext() helper for all 10 signal types - Explains analytical significance of each signal type Quick Win koala73#4 - Historical Context: - Add HistoricalContext interface with lastMajorEvent, precedentCount, cyclicalRisk fields - Add whyItMatters field to Hotspot type - Update major hotspots with historical precedents and geopolitical significance Quick Win koala73#5 - Propaganda Risk Flags: - Add PropagandaRisk type and SourceRiskProfile interface - Add SOURCE_PROPAGANDA_RISK mapping for state media (Xinhua, TASS, RT, CGTN) - Add getSourcePropagandaRisk() and isStateAffiliatedSource() helpers - Flag medium-risk state-affiliated sources (Al Jazeera, France 24, DW, etc.)
20 tasks
This was referenced Mar 10, 2026
10 tasks
4 tasks
3 tasks
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
Addresses 2 P1 review findings on PR #2942. P1 #1: Region pills keyed to labels that do not match persisted Forecast.region. The seed-forecasts writes Forecast.region with country names (Israel, Iran, Taiwan, United States...) and theater/market values (Red Sea, Black Sea, Baltic Sea, South China Sea, Eastern Mediterranean, Strait of Hormuz, Western Pacific...). The server RPC only does a substring .includes() match, so pills like 'Middle East' and 'Europe' miss most of the data. The seed already computes macroRegion via MACRO_REGION_MAP but that field is not in the Forecast proto, so the client never sees it. Fix: mirror MACRO_REGION_MAP to new shared/forecast-macro-regions.js and filter client-side by macro region. Pills now use MENA, EAST_ASIA, EUROPE, AMERICAS, SOUTH_ASIA, AFRICA as the six macro regions that match the publisher's taxonomy. P1 #2: Region filter state lost on refresh. selectedRegion was only consumed by refetchForRegion(). When the data loader or scheduler called updateForecasts(fullForecasts), this.forecasts was overwritten and the region filter disappeared. render() did not know about the filter. Fix: apply region filter inside the render pipeline (matching the existing activeDomain pattern). this.forecasts stays as the full unfiltered set. Clicking a pill no longer issues an RPC; it just updates selectedRegion and re-renders. Removed the regionFetchSeq counter since no network call fires. Side effect: improved perf (no redundant RPC on pill clicks) and instant response. The getForecasts RPC is called once on initial load, same as before.
This was referenced Apr 11, 2026
koala73
added a commit
that referenced
this pull request
Apr 11, 2026
P2 #1 — Mexico bbox mismatch between airport and flight classifiers airportToSnapshotRegion() routes Mexico to north-america by country, but latLonToSnapshotRegion()'s NA bbox started at lat 20°N. Mexico City (19.43°N), Guadalajara (20.67°N), and most Mexican airspace fell into latam, so NA's reroute_intensity understated its actual military pressure and the two classifiers disagreed on the same country. Fix: lower the NA bbox southern edge to lat 16.0°N. That captures every major Mexican city and state capital (Tuxtla Gutiérrez at 16.75°N is the southernmost) while still routing Guatemala City (14.6°N), San Salvador, Belize, and Honduras to latam. Tiny airports at lat ~14.9°N (Tapachula in southern Chiapas) route to latam — acceptable because they're not in the monitored set and don't carry meaningful military traffic. Added 8 regression tests asserting classifier parity across major Mexican cities and correct routing of Central American capitals. P2 #2 — seed-meta freshness path for undated payloads classifyInputs() was treating undated payloads as "fresh" via its fallback branch, so FAA/NOTAM/AviationStack/GPS-jam stalls would never drag down snapshot_confidence. None of those four payloads carry a top-level fetchedAt field; the seeders track freshness in companion seed-meta:* keys only. Fix: added an optional `metaKey` field to FreshnessRegistry entries. classifyInputs() now prefers the metaKey companion's fetchedAt when the spec declares one, falling back to the primary payload's top-level timestamp (existing behavior) and finally to "fresh" (existing fallback for legacy undated payloads). Wired through the full pipeline: - freshness.mjs: added metaKey type, ALL_META_KEYS export, and metaPayloads parameter to classifyInputs() - snapshot-meta.mjs: buildPreMeta() forwards metaSources - seed-regional-snapshots.mjs: readAllInputs() now fetches both ALL_INPUT_KEYS and ALL_META_KEYS in a single pipeline and returns { sources, metaSources }; main() destructures and passes metaSources through computeSnapshot → buildPreMeta - computeSnapshot() signature gained an optional metaSources arg (defaults to {} for back-compat with existing tests) Registered metaKey for each mobility input: aviation:delays:faa:v1 → seed-meta:aviation:faa aviation:delays:intl:v3 → seed-meta:aviation:intl aviation:notam:closures:v2 → seed-meta:aviation:notam intelligence:gpsjam:v2 → seed-meta:intelligence:gpsjam military:flights:v1 does NOT declare a metaKey because its payload already carries top-level fetchedAt — this is asserted in a test. Added 11 regression tests covering: - every undated mobility input has a metaKey - military:flights:v1 does NOT need a metaKey - ALL_META_KEYS aggregation - undated payload + fresh meta → fresh - undated payload + STALE meta → stale (the core bug fix) - undated payload + missing meta → falls back to fresh - missing payload → missing regardless of meta - legacy payloads with top-level timestamp still work - meta without fetchedAt falls through to payload timestamp ## Verification - node --test tests/regional-snapshot-mobility.test.mjs: 54/54 pass (35 original + 19 new regression tests) - npm run test:data: 4556/4556 pass - npm run typecheck: clean - biome lint on touched files: clean
This was referenced Jun 30, 2026
Merged
feat(billing): dunning + winback email lifecycle for failed payments and cancellations (#4932)
#4935
Merged
This was referenced Jul 8, 2026
This was referenced Jul 15, 2026
8 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Description
chunkArrayhelper insrc/utils/index.tsto split lists into batches.fetchMultipleStocks(src/services/markets.ts) to run batches viaPromise.all, add optionsbatchSize,delayMs, andonBatchcallback, and keep a default delay of2500msto respect rate limits.fetchCategoryFeeds(src/services/rss.ts) to fetch feeds in batches and expose anonBatchcallback that provides progressively merged and sorted news items.onBatchcallbacks insrc/App.tsto render partial market and news results for stocks, sectors, commodities, and news categories as batches complete.Testing
npm run dev, which launched Vite successfully and served the app (succeeded).artifacts/worldmonitor-progressive.png) (succeeded).ENETUNREACH) while fetching external APIs in the current environment, which prevented some live API responses (failed requests due to network environment).Codex Task