Skip to content

fix(ais-relay): subscribe to ShipStaticData so tanker layer actually populates#3410

Merged
koala73 merged 4 commits into
mainfrom
fix/relay-tanker-shiptype-from-staticdata
Apr 25, 2026
Merged

fix(ais-relay): subscribe to ShipStaticData so tanker layer actually populates#3410
koala73 merged 4 commits into
mainfrom
fix/relay-tanker-shiptype-from-staticdata

Conversation

@koala73

@koala73 koala73 commented Apr 25, 2026

Copy link
Copy Markdown
Owner

Summary

User-reported on energy.worldmonitor.app: Live Tanker Positions layer renders zero vessels despite PR #3402's wiring being correct end-to-end.

Root cause (full debug trace)

AISStream's PositionReport.MetaData does NOT carry ShipType per their schema — that field only arrives in ShipStaticData (Type 5) frames. PR #3402 shipped tanker capture predicated on meta.ShipType:

const shipType = Number(meta.ShipType);  // always NaN
if (Number.isFinite(shipType) && shipType >= 80 && shipType <= 89) {
  tankerReports.set(mmsi, ...);  // never executes
}

The relay subscribed to AISStream with FilterMessageTypes: ['PositionReport'] only — ShipStaticData never reached the relay, so the predicate always failed (NaN), tankerReports Map stayed permanently empty, and getVesselSnapshot returned tankerReports: [] to every caller. The frontend layer correctly rendered zero vessels.

Military detection (isLikelyMilitaryCandidate) survived because it has fallback paths (NAVAL_PREFIX_RE on ShipName + MMSI-suffix 00/99). Tanker detection had zero fallback because tanker MMSIs and names don't follow a single regex pattern.

Fix

Change Purpose
FilterMessageTypes['PositionReport', 'ShipStaticData'] Receive Type 5 frames
New processShipStaticDataForMeta handler Cache {shipType, shipName, lastSeen} by MMSI in a vesselMeta Map
Tanker-capture predicate in processPositionReportForSnapshot falls back to vesselMeta.get(mmsi).shipType when meta.ShipType is missing The actual classification logic
TTL eviction in cleanupAggregates (24h) for vesselMeta Bound memory growth on long-running relay

Volume impact

ShipStaticData is broadcast every ~6 min per vessel (vs every 2-10s for PositionReport while underway). Negligible volume increase relative to the existing PositionReport firehose.

Test plan

  • node -c scripts/ais-relay.cjs syntax clean
  • tests/relay-tanker-shipstatic.test.mjs (new, 5/5 pass) — static-analysis regression that pins the fix shape so a future change can't flip FilterMessageTypes back to PositionReport-only and silently re-empty the layer
  • Deploy: Railway redeploy of the AIS-relay service required to pick up the new subscription
  • Soak ~5-10 min post-deploy for vesselMeta to populate from ShipStaticData broadcasts
  • Verify on energy.worldmonitor.app: Live Tanker Positions layer should render vessels in the 6 chokepoint bboxes (Hormuz, Suez, Bab el-Mandeb, Malacca, Panama, Turkish Straits)
  • Network tab check: GetVesselSnapshot response should include snapshot.tankerReports[] with entries for the bbox query

…populates

User-reported on energy.worldmonitor.app: Live Tanker Positions layer
renders zero vessels despite PR #3402 wiring being correct end-to-end.

Root cause:
  AISStream's PositionReport.MetaData does NOT carry `ShipType` per their
  schema — that field only arrives in ShipStaticData (Type 5) frames.
  PR #3402 shipped tanker capture predicated on `meta.ShipType`:

      const shipType = Number(meta.ShipType);  // always NaN
      if (Number.isFinite(shipType) && shipType >= 80 && shipType <= 89) {
        tankerReports.set(mmsi, ...);  // never executes
      }

  The relay subscribed to AISStream with `FilterMessageTypes:
  ['PositionReport']` only — ShipStaticData never reached the relay, so
  the predicate always failed (NaN), `tankerReports` stayed permanently
  empty, and getVesselSnapshot returned `tankerReports: []` to every
  caller. The frontend layer correctly rendered zero vessels.

  Military detection (isLikelyMilitaryCandidate) survived because it has
  fallback paths (NAVAL_PREFIX_RE on ShipName + MMSI-suffix 00/99). Tanker
  detection had zero fallback because tanker MMSIs and names don't follow
  a single regex pattern.

Fix:
  - Add 'ShipStaticData' to AISStream's FilterMessageTypes — Type 5
    frames are broadcast every ~6 min per vessel (vs every 2-10s for
    PositionReport while underway), so the volume add is small.
  - Dispatch ShipStaticData → new processShipStaticDataForMeta handler
    that caches { shipType, shipName, lastSeen } by MMSI in a vesselMeta
    Map.
  - Tanker capture in processPositionReportForSnapshot now falls back to
    vesselMeta.get(mmsi).shipType when meta.ShipType is missing.
  - vesselMeta gets TTL eviction (24h) in cleanupAggregates so a long-
    running relay doesn't accumulate metadata for vessels that have left
    all tracked regions. 24h covers vessels with intermittent visibility
    while bounding memory growth.

Tests:
  - tests/relay-tanker-shipstatic.test.mjs (new, 5/5 pass) — pins the
    fix shape so a regression can't flip FilterMessageTypes back to
    PositionReport-only.
  - `node -c` clean.

Deploy:
  Requires a Railway redeploy of the AIS-relay service to pick up the
  new subscription filter. After redeploy, vesselMeta needs ~5-10 min
  to populate from ShipStaticData broadcasts before tanker classification
  reaches steady state for vessels in the 6 chokepoint bboxes.
@vercel

vercel Bot commented Apr 25, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Preview Apr 25, 2026 6:56pm

Request Review

@greptile-apps

greptile-apps Bot commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes the root cause of the live tanker layer rendering zero vessels: FilterMessageTypes was ['PositionReport']-only, so ShipType (which only arrives in ShipStaticData / AIS Type 5 frames) was never received, making the tanker predicate always evaluate NaN. The fix subscribes to both message types, caches {shipType, shipName} per MMSI in a new vesselMeta Map, and falls back to that cache in processPositionReportForSnapshot.

  • P1: vesselMeta has no hard size cap after the TTL eviction loop — every other Map in cleanupAggregates follows the TTL loop with an evictMapByTimestamp hard cap. A global AIS subscription can accumulate 100k+ unique MMSIs in 24 h, leaving memory growth unbounded on a long-running relay.

Confidence Score: 3/5

Safe to deploy for functionality, but vesselMeta memory growth is unbounded on the long-running Railway relay and should be addressed before a sustained production soak.

The core fix is correct and well-reasoned. One P1 defect (missing hard size cap on vesselMeta unlike all peer Maps) leaves memory growth unbounded on a global AIS subscription, directly contradicting the stated design goal. P1 ceiling is 4/5; the fact that this gap exists in the safety property the PR explicitly claims to provide pulls the score to 3.

scripts/ais-relay.cjs — specifically the cleanupAggregates vesselMeta eviction block (missing evictMapByTimestamp hard cap) and the processShipStaticDataForMeta sd.Type null-coercion guard.

Important Files Changed

Filename Overview
scripts/ais-relay.cjs Adds vesselMeta Map + processShipStaticDataForMeta to cache ShipType from AIS Type 5 frames, widens AISStream subscription to include ShipStaticData, and uses the cache as fallback in tanker classification. Core logic is correct; two issues found: no hard size cap on vesselMeta (P1) unlike all peer Maps, and null sd.Type silently coerces to 0 potentially overwriting valid tanker metadata (P2).
tests/relay-tanker-shipstatic.test.mjs New static-analysis regression test suite (5 tests) that pins the fix shape by scanning the relay source text for FilterMessageTypes, vesselMeta usage, TTL eviction, and ShipStaticData dispatch — prevents silent regression to PositionReport-only subscription.

Sequence Diagram

sequenceDiagram
    participant AISStream
    participant Relay as ais-relay.cjs
    participant vesselMeta as vesselMeta Map
    participant tankerReports as tankerReports Map
    participant Client as Frontend

    AISStream->>Relay: ShipStaticData (Type 5, ~every 6 min)
    Relay->>Relay: processShipStaticDataForMeta()
    Relay->>vesselMeta: set(mmsi, {shipType, shipName, lastSeen})

    AISStream->>Relay: PositionReport (every 2-10s)
    Relay->>Relay: processPositionReportForSnapshot()
    Relay->>vesselMeta: get(mmsi) → cachedMeta
    alt meta.ShipType present (rare per AISStream schema)
        Relay->>Relay: use meta.ShipType
    else fallback to vesselMeta (normal path)
        Relay->>Relay: use cachedMeta.shipType
    end
    alt shipType 80-89 (tanker)
        Relay->>tankerReports: set(mmsi, {lat, lon, shipType, ...})
    end

    Client->>Relay: GetVesselSnapshot (bbox query)
    Relay->>Client: snapshot.tankerReports[] filtered by bbox

    Note over Relay,vesselMeta: cleanupAggregates() every N seconds
    Relay->>vesselMeta: delete entries older than 24h (TTL only — no hard cap)
Loading

Reviews (1): Last reviewed commit: "fix(ais-relay): subscribe to ShipStaticD..." | Re-trigger Greptile

Comment thread scripts/ais-relay.cjs
Comment on lines +6787 to +6795
// vesselMeta TTL eviction: drop entries older than VESSEL_META_TTL_MS so
// long-running relays don't accumulate metadata for vessels that have
// sailed out of any tracked region. ShipStaticData is rebroadcast every
// ~6 min, so a 24h TTL covers vessels with intermittent visibility.
for (const [mmsi, entry] of vesselMeta) {
if (entry.lastSeen < now - VESSEL_META_TTL_MS) {
vesselMeta.delete(mmsi);
}
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 vesselMeta lacks a hard size cap unlike every other Map in cleanupAggregates

Every peer Map in this function (candidateReports, tankerReports, densityGrid, etc.) has both a time-based loop and an evictMapByTimestamp hard-cap call after it. vesselMeta only has the 24 h TTL loop. With a global bounding box ([[-90, -180], [90, 180]]), AISStream will deliver ShipStaticData for hundreds of thousands of unique MMSIs per day; all entries younger than 24 h accumulate without bound. A long-running Railway relay could reach 50–100 MB of vesselMeta alone — directly contradicting the PR's stated design goal of bounding memory growth.

Add a hard cap consistent with the rest of the function, e.g.:

// After the TTL loop:
evictMapByTimestamp(vesselMeta, MAX_VESSELS, (entry) => entry.lastSeen || 0);

(MAX_VESSELS is an existing constant used for the vessels Map, making it a natural upper bound.)

Comment thread scripts/ais-relay.cjs Outdated
Comment on lines +6616 to +6618
// ShipType lives in the message body, not MetaData, on Type 5 frames.
const shipType = Number(sd.Type);
if (!Number.isFinite(shipType)) return;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Number(null) passes the finite-check and can silently overwrite a valid tanker entry

Number(null) evaluates to 0, which passes Number.isFinite. If AISStream ever delivers a ShipStaticData frame where sd.Type is null for a vessel that previously had shipType: 83, vesselMeta.set would overwrite it with shipType: 0, and that vessel would stop being classified as a tanker until its next valid static frame arrives (~6 min). Add an explicit null-guard before coercing:

const rawType = sd.Type;
const shipType = rawType != null ? Number(rawType) : NaN;
if (!Number.isFinite(shipType)) return;

Comment thread scripts/ais-relay.cjs
Comment on lines +6619 to +6623
vesselMeta.set(mmsi, {
shipType,
shipName: (sd.Name || meta.ShipName || '').trim(),
lastSeen: Date.now(),
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 vesselMeta.lastSeen never refreshed from PositionReports

lastSeen is only updated when a ShipStaticData frame arrives (~6 min cycle). If an active tanker's static broadcast is missed or delayed near the 24 h boundary, its vesselMeta entry expires while PositionReports are still being received — causing classification to silently drop until the next static frame. Refreshing lastSeen on each PositionReport for known MMSIs keeps the entry alive proportional to actual observed activity:

// in processPositionReportForSnapshot, after resolving mmsi:
const existing = vesselMeta.get(mmsi);
if (existing) existing.lastSeen = Date.now();

koala73 added 3 commits April 25, 2026 21:51
…eld-name pinning (PR3410 review)

Three review findings on PR #3410:

P2 — Static-analysis tests don't pin field names (correctness/testing):
  Pre-fix tests asserted `vesselMeta.set(... { shipType` which only checks
  the property name, not the source field. A typo regression like
  Number(sd.Typ) or Number(sd.shipType) (lowercase) would still pass —
  silently re-emptying the tanker layer.
  Added two new tests:
    - "reads ShipType from sd.Type (NOT meta.ShipType)" — pins the exact
      Number(sd.Type) substring; also pins sd.Name for shipName fallback.
    - "accepts MMSI from meta.MMSI OR sd.UserID" — pins the new fallback
      shape (see #2 below).

P2 — MMSI-extraction lacks UserID fallback (correctness):
  AISStream's ShipStaticData payload sample mirrors MMSI as `UserID` on
  the message body, while PositionReport puts it under MetaData.MMSI.
  If a wrapper-schema variant ever ships a Type 5 frame without
  MetaData.MMSI, processShipStaticDataForMeta early-returned and
  vesselMeta stayed empty — silent re-empty of the tanker layer.
  Defensive: `String(meta.MMSI || sd.UserID || '')`.

P3 — Test #4 order check used file-wide indexOf (testing):
  RELAY.indexOf('vesselMeta.get(mmsi)') finds the FIRST occurrence in
  the file. A future earlier vesselMeta.get(...) elsewhere would let
  the in-tanker-path lookup be removed without the test failing.
  Scoped the substring search to the body of
  processPositionReportForSnapshot (slice from fn declaration to the
  next top-level `function ` line).

Tests: 7/7 pass (was 5). `node -c` clean.
…candidate paths also use vesselMeta

User-reported review finding on PR #3410: the previous commit fixed only
the tanker snapshot path. Two more sites in processPositionReportForSnapshot
read the same broken meta.ShipType field:

  Site 1 (line ~6651, vessels record):
    vessels.set(mmsi, { ..., shipType: meta.ShipType, ... });
  Consumer: classifyVesselType(vessel?.shipType) at the chokepoint
  transit logging site (line ~6545). With shipType permanently undefined,
  classifyVesselType always returned 'other' — silently breaking per-type
  transit counts in /seedChokepointTransits and any downstream consumer
  using transit-by-type breakdowns.

  Site 2 (line ~6692, military candidate record):
    candidateReports.set(mmsi, { ..., shipType: meta.ShipType, ... });
    isLikelyMilitaryCandidate(meta) — read meta.ShipType for the
    35/55/50-59 type-based arms.
  Consequence: the military classifier survived in production only via
  the NAVAL_PREFIX_RE + MMSI-suffix fallbacks. The type-based arm never
  fired. candidateReports also exposed shipType: undefined to callers.

Fix:
  - Compute `effectiveShipType` ONCE at the top of processPositionReportForSnapshot,
    preferring meta.ShipType when present (defense for any future schema
    enrichment) and falling back to vesselMeta cache from ShipStaticData.
  - Pass effectiveShipType to vessels.set, isLikelyMilitaryCandidate, and
    the tanker-capture predicate. All three sites now classify correctly.
  - Refactor isLikelyMilitaryCandidate(meta) → isLikelyMilitaryCandidate(meta, resolvedShipType)
    with backward-compat fallback to meta.ShipType when no override is passed.

Same root cause, same vesselMeta cache solves all three sites.

Tests: 10/10 (was 7) — added 3 new regression tests pinning:
  - vessels.set shipType field comes from `effectiveShipType`
  - isLikelyMilitaryCandidate signature accepts resolvedShipType param
  - candidateReports.set shipType field comes from `effectiveShipType`
node -c clean.
…eptile review)

Two findings on PR #3410:

P1 — vesselMeta has TTL but no hard size cap:
  Every peer Map in cleanupAggregates (tankerReports, candidateReports,
  densityGrid, vesselHistory) follows its TTL loop with evictMapByTimestamp.
  vesselMeta did not — leaving memory growth unbounded against a hostile
  or buggy upstream flooding unique MMSIs faster than TTL drains them.
  Added MAX_VESSEL_META=50000 (covers ~50-70k active global AIS fleet
  with headroom) + evictMapByTimestamp call after the TTL loop.

P2 — Number(null) === 0 could overwrite valid cache entries:
  processShipStaticDataForMeta's `if (!Number.isFinite(shipType)) return`
  guard accepts shipType=0 (AIS code 0 = "Not available" per ITU-R M.1371).
  A vessel that broadcasts {Type: 85} then later {Type: null} would have
  its cached tanker classification overwritten with shipType=0, downgrading
  it to non-tanker on the next PositionReport.
  Tightened to `|| shipType <= 0`.

Tests: 11/11 (was 10) — added 2 new pins:
  - vesselMeta has TTL AND hard size cap (asserts MAX_VESSEL_META +
    evictMapByTimestamp(vesselMeta, ...))
  - processShipStaticDataForMeta rejects shipType <= 0 (asserts the
    `<= 0` guard so a refactor can't silently drop it)
node -c clean.
@koala73
koala73 merged commit c52b664 into main Apr 25, 2026
10 checks passed
@koala73
koala73 deleted the fix/relay-tanker-shiptype-from-staticdata branch April 25, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant