Skip to content

data quality: CII risk scores apply undisclosed country-specific event multipliers up to 3×, making scores non-reproducible and politically opaque #3722

Description

@tg12

Location

server/worldmonitor/intelligence/v1/get-risk-scores.ts

const BASELINE_RISK: Record<string, number> = {
  US: 5, RU: 35, CN: 25, UA: 50, IR: 40, IL: 45, TW: 30, KP: 45,
  // ...
};

const EVENT_MULTIPLIER: Record<string, number> = {
  US: 0.3, RU: 2.0, CN: 2.5, UA: 0.8, IR: 2.0, IL: 0.7, TW: 1.5, KP: 3.0,
  // ...
};

These tables feed directly into the Combined Intelligence Index formula:

const baseline = BASELINE_RISK[code] || 20;
const multiplier = EVENT_MULTIPLIER[code] || 1.0;
// ...
const acledScore = Math.min(50, Math.round((d.battles * 3 + d.explosions * 4 + d.civilianViolence * 5) * multiplier));

Impact

The scores are not reproducible from the displayed data. A user who sees that North Korea scores 72/100 cannot independently verify it. The BASELINE_RISK: 45 and EVENT_MULTIPLIER: 3.0 for KP are invisible in the UI — they are editorial judgments hardcoded into the server.

Specific problems:

  1. KP scores high with zero ACLED events. BASELINE_RISK: 45 × 0.4 baseline weight = 18 points before any live data is ingested. At full formula weight, KP can score above 50 with no current events.
  2. IL scores are suppressed (0.7×). Events in Israel score 30% lower than identical events in Russia (2.0×). This is an editorial choice with no published rationale.
  3. The multipliers are applied to ACLED fatality counts, which are already estimates from media reports. Multiplying contested fatality data by a hidden country-specific factor compounds the uncertainty without disclosing it.
  4. The methodology note in the UI does not disclose these baselines or multipliers. Users comparing countries are not aware that a baseline political risk judgment is baked in.

Fix

  • Document the baseline and multiplier methodology publicly (methodology page, API response metadata, or tooltip).
  • Or move these values to the database / config layer so they are auditable and not embedded in server code.
  • At minimum, include staticBaseline and the effective multiplier in the API response so third-party callers can reproduce the computation.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions