Skip to content

feat(feeds): add IRNA, Mehr, Jerusalem Post, Ynetnews to middleeast#3236

Merged
koala73 merged 3 commits into
mainfrom
feat/me-feeds-irna-mehr-jpost-ynet
Apr 20, 2026
Merged

feat(feeds): add IRNA, Mehr, Jerusalem Post, Ynetnews to middleeast#3236
koala73 merged 3 commits into
mainfrom
feat/me-feeds-irna-mehr-jpost-ynet

Conversation

@koala73

@koala73 koala73 commented Apr 20, 2026

Copy link
Copy Markdown
Owner

Summary

Add four direct-RSS Middle East sources absent everywhere in the repo today. Highest-ROI slice of the ME source audit (#3226) shippable as a drop-in: no Decodo routing, no custom fetcher, no new Redis keys, no infra.

Source URL Role
IRNA `https://en.irna.ir/rss\` Iranian state wire (English)
Mehr News `https://en.mehrnews.com/rss\` Iranian state-affiliated
Jerusalem Post `https://www.jpost.com/rss/rssfeedsheadlines.aspx\` Israeli English daily
Ynetnews `https://www.ynetnews.com/Integration/StoryRss3089.xml\` Yedioth Ahronoth EN

All four verified to return real RSS XML on `curl` from a clean IP, and grep-verified absent across `src/config/feeds.ts`, `scripts/seed-*.mjs`, `scripts/ais-relay.cjs`, `api/`, and the RSS allowlist.

Changes

`src/config/feeds.ts`

  • `middleeast` block: 4 new entries placed next to their peers (IRNA/Mehr after `Fars News`; JPost/Ynetnews after `Haaretz`).
  • `SOURCE_PROPAGANDA_RISK`: IRNA + Mehr tagged `high / Iran state-affiliated` (joining Press TV). JPost + Ynetnews tagged `low` with `knownBiases` for transparency.

RSS allowlist (byte-identical mirrors enforced by `tests/edge-functions.test.mjs`):

  • `shared/rss-allowed-domains.json`
  • `scripts/shared/rss-allowed-domains.json`
  • `api/_rss-allowed-domains.js`

Added: `en.irna.ir`, `en.mehrnews.com`, `www.jpost.com`, `www.ynetnews.com`.

Deferred (separate PRs)

  • Times of Israel — already in the allowlist; was removed from feeds for cloud-IP 403. Needs Decodo routing.
  • IDF Spokesperson — `idf.il` has no direct RSS endpoint; would need a scraper.
  • Tasnim / Israel Hayom / Press TV RSS — known cloud-IP blocks.
  • WAM / SPA / KUNA / QNA / BNA — public RSS endpoints are dead (sites migrated to SPAs or gate with 403). Audit plan listed these as `Effort: S` but the endpoints don't exist as claimed.

Notes on the parent audit (#3226)

While verifying I hit two factual errors in that plan worth correcting before downstream PRs kick off:

  1. Travel Advisories aren't missing. `scripts/seed-security-advisories.mjs:14` already pulls `travel.state.gov/_res/rss/TAsTWs.xml` along with UK FCDO, 13 US Embassy country-alert feeds, CDC, 5 ECDC feeds, and WHO. The plan only audited `feeds.ts`.
  2. NOTAM source is ICAO, not FAA. `scripts/seed-airport-delays.mjs:12` + `scripts/ais-relay.cjs:2750,9053` use ICAO's global real-time NOTAM API (`dataservices.icao.int/api/notams-realtime-list`), which covers all ICAO-member states including Iran, Iraq, Syria, Lebanon, Yemen. The P8 Notamify spike in the plan is based on a false premise.

Test plan

  • `npm run typecheck`
  • `npm run typecheck:api`
  • `node --test tests/edge-functions.test.mjs` (171/171, enforces the shared/scripts/shared/api mirror contract)
  • `npm run test:data` (5891/5891)
  • 7-day soak after merge: watch for sustained 403s from IRNA/Mehr cloud-IP blocks. Fallback is a Google News wrapper if needed.
  • Manual spot-check: confirm the four new entries appear in `/api/rss-proxy` allowlist and render in the news panel.

Risks

  • Iranian state media occasionally 403s cloud IPs. No evidence of that on IRNA/Mehr as of today, but worth soaking. Fallback is cheap.
  • No new Redis keys, no health.js registration, no cron provisioning needed.

Four direct-RSS sources verified from a clean IP and absent everywhere
in the repo (src/config/feeds.ts, scripts/seed-*, ais-relay.cjs, RSS
allowlist). Closes the highest-ROI Iran / Israel domestic-press gap
from the ME source audit (PR #3226) with zero infra changes.

- IRNA        https://en.irna.ir/rss
- Mehr News   https://en.mehrnews.com/rss
- Jerusalem Post https://www.jpost.com/rss/rssfeedsheadlines.aspx
- Ynetnews    https://www.ynetnews.com/Integration/StoryRss3089.xml

Propaganda-risk metadata:
- IRNA + Mehr tagged high / Iran state-affiliated (join Press TV).
- JPost + Ynetnews tagged low with knownBiases for transparency.

RSS allowlist updated in all three mirrors (shared/, scripts/shared/,
api/_rss-allowed-domains.js) per the byte-identical mirror contract
enforced by tests/edge-functions.test.mjs.

Deferred (separate PRs):
- Times of Israel: already in allowlist; was removed from feeds for
  cloud-IP 403. Needs Decodo routing.
- IDF Spokesperson: idf.il has no direct RSS endpoint; needs scraper.
- Tasnim / Press TV RSS / Israel Hayom: known cloud-IP blocks.
- WAM / SPA / KUNA / QNA / BNA: public RSS endpoints are dead; sites
  migrated to SPAs or gate with 403.

Plan doc (PR #3226) overstated the gap: it audited only feeds.ts and
missed that travel advisories + US Embassy alerts are already covered
by scripts/seed-security-advisories.mjs. NOTAM claim in that doc is
also wrong: we use ICAO's global NOTAM API, not FAA.
@vercel

vercel Bot commented Apr 20, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Apr 20, 2026 3:07pm

Request Review

@greptile-apps

greptile-apps Bot commented Apr 20, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Adds four direct-RSS Middle East sources (IRNA, Mehr News, Jerusalem Post, Ynetnews) to src/config/feeds.ts and their domains to the three allowlist mirrors. The SOURCE_PROPAGANDA_RISK map is updated correctly — IRNA and Mehr tagged high/Iran-state, JPost and Ynetnews tagged low with explicit knownBiases.

Confidence Score: 5/5

Safe to merge; all remaining findings are cosmetic P2 style suggestions

All four allowlist mirrors are updated consistently, risk profiles are correctly set, no new infrastructure is needed, and the only finding is a section-ordering cosmetic issue in SOURCE_PROPAGANDA_RISK

No files require special attention

Important Files Changed

Filename Overview
src/config/feeds.ts Adds IRNA, Mehr News, Jerusalem Post, Ynetnews to middleeast feed list and SOURCE_PROPAGANDA_RISK; minor cosmetic placement issue for low-risk entries inside medium-risk comment section
api/_rss-allowed-domains.js Allowlist updated with all four new domains; insertion order differs from feeds.ts (JPost/Ynet before IRNA/Mehr) but is cosmetic and consistent with the other two mirror files
scripts/shared/rss-allowed-domains.json Mirror of shared/rss-allowed-domains.json updated identically; all four new domains present
shared/rss-allowed-domains.json Canonical shared allowlist updated with the four new domains, consistent with api and scripts mirrors

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[RSS Proxy Request] --> B{Domain in allowlist?}
    B -- No --> C[Reject 403]
    B -- Yes --> D[Fetch RSS feed]
    D --> E{Feed source}
    E --> F["en.irna.ir (IRNA · high risk)"]
    E --> G["en.mehrnews.com (Mehr News · high risk)"]
    E --> H["www.jpost.com (Jerusalem Post · low risk)"]
    E --> I["www.ynetnews.com (Ynetnews · low risk)"]
    F --> J[Apply SOURCE_PROPAGANDA_RISK label]
    G --> J
    H --> J
    I --> J
    J --> K[Return RSS to client]
Loading

Reviews (1): Last reviewed commit: "feat(feeds): add IRNA, Mehr, Jerusalem P..." | Re-trigger Greptile

Comment thread src/config/feeds.ts Outdated
Comment on lines +136 to +137
'Jerusalem Post': { risk: 'low', knownBiases: ['Israeli centre-right'], note: 'English-language Israeli daily of record' },
'Ynetnews': { risk: 'low', knownBiases: ['Israeli mainstream'], note: 'Yedioth Ahronoth English edition' },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Low-risk entries placed inside medium-risk comment block

Jerusalem Post and Ynetnews are inserted between the last medium-risk entry (Moscow Times) and the // Low risk section header. Both have risk: 'low', so they belong below the // Low risk - Independent with editorial standards (explicit) comment rather than above it. Not a runtime bug, but it makes the section boundary misleading for future contributors scanning for risk tiers.

Suggested change
'Jerusalem Post': { risk: 'low', knownBiases: ['Israeli centre-right'], note: 'English-language Israeli daily of record' },
'Ynetnews': { risk: 'low', knownBiases: ['Israeli mainstream'], note: 'Yedioth Ahronoth English edition' },
'Moscow Times': { risk: 'medium', knownBiases: ['Anti-Kremlin'], note: 'Independent, critical of Russian government' },
// Low risk - Independent with editorial standards (explicit)
'Jerusalem Post': { risk: 'low', knownBiases: ['Israeli centre-right'], note: 'English-language Israeli daily of record' },
'Ynetnews': { risk: 'low', knownBiases: ['Israeli mainstream'], note: 'Yedioth Ahronoth English edition' },

Reviewer on #3236 flagged that adding the four new ME feeds to
FULL_FEEDS.middleeast alone leaves them disabled on first run, because
App.ts:661 persists computeDefaultDisabledSources() output derived from
DEFAULT_ENABLED_SOURCES. Users would have to manually re-enable via
Settings > Sources, defeating the purpose of broadening the default
ME mix.

Add the four new sources to DEFAULT_ENABLED_SOURCES.middleeast so they
ship on by default. Placement keeps them adjacent to their peers
(IRNA / Mehr with the other Iran sources, JPost / Ynetnews after
Haaretz). Risk/slant tags already in SOURCE_PROPAGANDA_RISK ensure
downstream digest dedup + summarization weights them correctly.
Greptile on #3236 flagged that both entries are risk: 'low' but were
inserted above the `// Low risk - Independent with editorial standards`
comment header, making the section boundary misleading for future
contributors. Shift them under the header where they belong.

No runtime change; cosmetic ordering only.
@koala73
koala73 merged commit ecd56d4 into main Apr 20, 2026
9 checks passed
@koala73
koala73 deleted the feat/me-feeds-irna-mehr-jpost-ynet branch April 20, 2026 15:07
koala73 added a commit that referenced this pull request Apr 20, 2026
Feeds.ts is at 523 entries after PR #3236 landed. The "435+" figure
has been baked into marketing copy, docs, press kit, and localized
strings for a long time and is now noticeably understated. Bump to
500+ as the new canonical figure.

Also aligned three stale claims in less-visited docs:
  docs/getting-started.mdx        70+ RSS feeds        => 500+ RSS feeds
  docs/ai-intelligence.mdx        344 sources          => 500+ sources
  docs/COMMUNITY-PROMOTION-GUIDE  170+ news feeds      => 500+ news feeds
                                  170+ news sources    => 500+ news sources

And bumped the digest-dedup copy 400+ to 500+ (English + French
locales + pro-test/index.html prerendered body) for consistency with
the pricing and GDELT panels.

Left alone on purpose (different metric):
  22 services / 22 service domains
  24 feeds (security-advisory seeder specifically)
  31 sources (freshness tracker)
  45 map layers

Rebuilt /pro bundle so the per-locale chunks + prerendered index.html
under public/pro/assets ship the new copy. 20 locales updated.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant