fix(ais-relay): subscribe to ShipStaticData so tanker layer actually populates#3410
Conversation
…populates User-reported on energy.worldmonitor.app: Live Tanker Positions layer renders zero vessels despite PR #3402 wiring being correct end-to-end. Root cause: AISStream's PositionReport.MetaData does NOT carry `ShipType` per their schema — that field only arrives in ShipStaticData (Type 5) frames. PR #3402 shipped tanker capture predicated on `meta.ShipType`: const shipType = Number(meta.ShipType); // always NaN if (Number.isFinite(shipType) && shipType >= 80 && shipType <= 89) { tankerReports.set(mmsi, ...); // never executes } The relay subscribed to AISStream with `FilterMessageTypes: ['PositionReport']` only — ShipStaticData never reached the relay, so the predicate always failed (NaN), `tankerReports` stayed permanently empty, and getVesselSnapshot returned `tankerReports: []` to every caller. The frontend layer correctly rendered zero vessels. Military detection (isLikelyMilitaryCandidate) survived because it has fallback paths (NAVAL_PREFIX_RE on ShipName + MMSI-suffix 00/99). Tanker detection had zero fallback because tanker MMSIs and names don't follow a single regex pattern. Fix: - Add 'ShipStaticData' to AISStream's FilterMessageTypes — Type 5 frames are broadcast every ~6 min per vessel (vs every 2-10s for PositionReport while underway), so the volume add is small. - Dispatch ShipStaticData → new processShipStaticDataForMeta handler that caches { shipType, shipName, lastSeen } by MMSI in a vesselMeta Map. - Tanker capture in processPositionReportForSnapshot now falls back to vesselMeta.get(mmsi).shipType when meta.ShipType is missing. - vesselMeta gets TTL eviction (24h) in cleanupAggregates so a long- running relay doesn't accumulate metadata for vessels that have left all tracked regions. 24h covers vessels with intermittent visibility while bounding memory growth. Tests: - tests/relay-tanker-shipstatic.test.mjs (new, 5/5 pass) — pins the fix shape so a regression can't flip FilterMessageTypes back to PositionReport-only. - `node -c` clean. Deploy: Requires a Railway redeploy of the AIS-relay service to pick up the new subscription filter. After redeploy, vesselMeta needs ~5-10 min to populate from ShipStaticData broadcasts before tanker classification reaches steady state for vessels in the 6 chokepoint bboxes.
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
Greptile SummaryThis PR fixes the root cause of the live tanker layer rendering zero vessels:
Confidence Score: 3/5Safe to deploy for functionality, but The core fix is correct and well-reasoned. One P1 defect (missing hard size cap on scripts/ais-relay.cjs — specifically the Important Files Changed
Sequence DiagramsequenceDiagram
participant AISStream
participant Relay as ais-relay.cjs
participant vesselMeta as vesselMeta Map
participant tankerReports as tankerReports Map
participant Client as Frontend
AISStream->>Relay: ShipStaticData (Type 5, ~every 6 min)
Relay->>Relay: processShipStaticDataForMeta()
Relay->>vesselMeta: set(mmsi, {shipType, shipName, lastSeen})
AISStream->>Relay: PositionReport (every 2-10s)
Relay->>Relay: processPositionReportForSnapshot()
Relay->>vesselMeta: get(mmsi) → cachedMeta
alt meta.ShipType present (rare per AISStream schema)
Relay->>Relay: use meta.ShipType
else fallback to vesselMeta (normal path)
Relay->>Relay: use cachedMeta.shipType
end
alt shipType 80-89 (tanker)
Relay->>tankerReports: set(mmsi, {lat, lon, shipType, ...})
end
Client->>Relay: GetVesselSnapshot (bbox query)
Relay->>Client: snapshot.tankerReports[] filtered by bbox
Note over Relay,vesselMeta: cleanupAggregates() every N seconds
Relay->>vesselMeta: delete entries older than 24h (TTL only — no hard cap)
Reviews (1): Last reviewed commit: "fix(ais-relay): subscribe to ShipStaticD..." | Re-trigger Greptile |
| // vesselMeta TTL eviction: drop entries older than VESSEL_META_TTL_MS so | ||
| // long-running relays don't accumulate metadata for vessels that have | ||
| // sailed out of any tracked region. ShipStaticData is rebroadcast every | ||
| // ~6 min, so a 24h TTL covers vessels with intermittent visibility. | ||
| for (const [mmsi, entry] of vesselMeta) { | ||
| if (entry.lastSeen < now - VESSEL_META_TTL_MS) { | ||
| vesselMeta.delete(mmsi); | ||
| } | ||
| } |
There was a problem hiding this comment.
vesselMeta lacks a hard size cap unlike every other Map in cleanupAggregates
Every peer Map in this function (candidateReports, tankerReports, densityGrid, etc.) has both a time-based loop and an evictMapByTimestamp hard-cap call after it. vesselMeta only has the 24 h TTL loop. With a global bounding box ([[-90, -180], [90, 180]]), AISStream will deliver ShipStaticData for hundreds of thousands of unique MMSIs per day; all entries younger than 24 h accumulate without bound. A long-running Railway relay could reach 50–100 MB of vesselMeta alone — directly contradicting the PR's stated design goal of bounding memory growth.
Add a hard cap consistent with the rest of the function, e.g.:
// After the TTL loop:
evictMapByTimestamp(vesselMeta, MAX_VESSELS, (entry) => entry.lastSeen || 0);(MAX_VESSELS is an existing constant used for the vessels Map, making it a natural upper bound.)
| // ShipType lives in the message body, not MetaData, on Type 5 frames. | ||
| const shipType = Number(sd.Type); | ||
| if (!Number.isFinite(shipType)) return; |
There was a problem hiding this comment.
Number(null) passes the finite-check and can silently overwrite a valid tanker entry
Number(null) evaluates to 0, which passes Number.isFinite. If AISStream ever delivers a ShipStaticData frame where sd.Type is null for a vessel that previously had shipType: 83, vesselMeta.set would overwrite it with shipType: 0, and that vessel would stop being classified as a tanker until its next valid static frame arrives (~6 min). Add an explicit null-guard before coercing:
const rawType = sd.Type;
const shipType = rawType != null ? Number(rawType) : NaN;
if (!Number.isFinite(shipType)) return;| vesselMeta.set(mmsi, { | ||
| shipType, | ||
| shipName: (sd.Name || meta.ShipName || '').trim(), | ||
| lastSeen: Date.now(), | ||
| }); |
There was a problem hiding this comment.
vesselMeta.lastSeen never refreshed from PositionReports
lastSeen is only updated when a ShipStaticData frame arrives (~6 min cycle). If an active tanker's static broadcast is missed or delayed near the 24 h boundary, its vesselMeta entry expires while PositionReports are still being received — causing classification to silently drop until the next static frame. Refreshing lastSeen on each PositionReport for known MMSIs keeps the entry alive proportional to actual observed activity:
// in processPositionReportForSnapshot, after resolving mmsi:
const existing = vesselMeta.get(mmsi);
if (existing) existing.lastSeen = Date.now();…eld-name pinning (PR3410 review) Three review findings on PR #3410: P2 — Static-analysis tests don't pin field names (correctness/testing): Pre-fix tests asserted `vesselMeta.set(... { shipType` which only checks the property name, not the source field. A typo regression like Number(sd.Typ) or Number(sd.shipType) (lowercase) would still pass — silently re-emptying the tanker layer. Added two new tests: - "reads ShipType from sd.Type (NOT meta.ShipType)" — pins the exact Number(sd.Type) substring; also pins sd.Name for shipName fallback. - "accepts MMSI from meta.MMSI OR sd.UserID" — pins the new fallback shape (see #2 below). P2 — MMSI-extraction lacks UserID fallback (correctness): AISStream's ShipStaticData payload sample mirrors MMSI as `UserID` on the message body, while PositionReport puts it under MetaData.MMSI. If a wrapper-schema variant ever ships a Type 5 frame without MetaData.MMSI, processShipStaticDataForMeta early-returned and vesselMeta stayed empty — silent re-empty of the tanker layer. Defensive: `String(meta.MMSI || sd.UserID || '')`. P3 — Test #4 order check used file-wide indexOf (testing): RELAY.indexOf('vesselMeta.get(mmsi)') finds the FIRST occurrence in the file. A future earlier vesselMeta.get(...) elsewhere would let the in-tanker-path lookup be removed without the test failing. Scoped the substring search to the body of processPositionReportForSnapshot (slice from fn declaration to the next top-level `function ` line). Tests: 7/7 pass (was 5). `node -c` clean.
…candidate paths also use vesselMeta User-reported review finding on PR #3410: the previous commit fixed only the tanker snapshot path. Two more sites in processPositionReportForSnapshot read the same broken meta.ShipType field: Site 1 (line ~6651, vessels record): vessels.set(mmsi, { ..., shipType: meta.ShipType, ... }); Consumer: classifyVesselType(vessel?.shipType) at the chokepoint transit logging site (line ~6545). With shipType permanently undefined, classifyVesselType always returned 'other' — silently breaking per-type transit counts in /seedChokepointTransits and any downstream consumer using transit-by-type breakdowns. Site 2 (line ~6692, military candidate record): candidateReports.set(mmsi, { ..., shipType: meta.ShipType, ... }); isLikelyMilitaryCandidate(meta) — read meta.ShipType for the 35/55/50-59 type-based arms. Consequence: the military classifier survived in production only via the NAVAL_PREFIX_RE + MMSI-suffix fallbacks. The type-based arm never fired. candidateReports also exposed shipType: undefined to callers. Fix: - Compute `effectiveShipType` ONCE at the top of processPositionReportForSnapshot, preferring meta.ShipType when present (defense for any future schema enrichment) and falling back to vesselMeta cache from ShipStaticData. - Pass effectiveShipType to vessels.set, isLikelyMilitaryCandidate, and the tanker-capture predicate. All three sites now classify correctly. - Refactor isLikelyMilitaryCandidate(meta) → isLikelyMilitaryCandidate(meta, resolvedShipType) with backward-compat fallback to meta.ShipType when no override is passed. Same root cause, same vesselMeta cache solves all three sites. Tests: 10/10 (was 7) — added 3 new regression tests pinning: - vessels.set shipType field comes from `effectiveShipType` - isLikelyMilitaryCandidate signature accepts resolvedShipType param - candidateReports.set shipType field comes from `effectiveShipType` node -c clean.
…eptile review) Two findings on PR #3410: P1 — vesselMeta has TTL but no hard size cap: Every peer Map in cleanupAggregates (tankerReports, candidateReports, densityGrid, vesselHistory) follows its TTL loop with evictMapByTimestamp. vesselMeta did not — leaving memory growth unbounded against a hostile or buggy upstream flooding unique MMSIs faster than TTL drains them. Added MAX_VESSEL_META=50000 (covers ~50-70k active global AIS fleet with headroom) + evictMapByTimestamp call after the TTL loop. P2 — Number(null) === 0 could overwrite valid cache entries: processShipStaticDataForMeta's `if (!Number.isFinite(shipType)) return` guard accepts shipType=0 (AIS code 0 = "Not available" per ITU-R M.1371). A vessel that broadcasts {Type: 85} then later {Type: null} would have its cached tanker classification overwritten with shipType=0, downgrading it to non-tanker on the next PositionReport. Tightened to `|| shipType <= 0`. Tests: 11/11 (was 10) — added 2 new pins: - vesselMeta has TTL AND hard size cap (asserts MAX_VESSEL_META + evictMapByTimestamp(vesselMeta, ...)) - processShipStaticDataForMeta rejects shipType <= 0 (asserts the `<= 0` guard so a refactor can't silently drop it) node -c clean.
Summary
User-reported on energy.worldmonitor.app: Live Tanker Positions layer renders zero vessels despite PR #3402's wiring being correct end-to-end.
Root cause (full debug trace)
AISStream's
PositionReport.MetaDatadoes NOT carryShipTypeper their schema — that field only arrives inShipStaticData(Type 5) frames. PR #3402 shipped tanker capture predicated onmeta.ShipType:The relay subscribed to AISStream with
FilterMessageTypes: ['PositionReport']only —ShipStaticDatanever reached the relay, so the predicate always failed (NaN),tankerReportsMap stayed permanently empty, andgetVesselSnapshotreturnedtankerReports: []to every caller. The frontend layer correctly rendered zero vessels.Military detection (
isLikelyMilitaryCandidate) survived because it has fallback paths (NAVAL_PREFIX_REon ShipName + MMSI-suffix00/99). Tanker detection had zero fallback because tanker MMSIs and names don't follow a single regex pattern.Fix
FilterMessageTypes←['PositionReport', 'ShipStaticData']processShipStaticDataForMetahandler{shipType, shipName, lastSeen}by MMSI in avesselMetaMapprocessPositionReportForSnapshotfalls back tovesselMeta.get(mmsi).shipTypewhenmeta.ShipTypeis missingcleanupAggregates(24h) forvesselMetaVolume impact
ShipStaticData is broadcast every ~6 min per vessel (vs every 2-10s for PositionReport while underway). Negligible volume increase relative to the existing PositionReport firehose.
Test plan
node -c scripts/ais-relay.cjssyntax cleantests/relay-tanker-shipstatic.test.mjs(new, 5/5 pass) — static-analysis regression that pins the fix shape so a future change can't flipFilterMessageTypesback toPositionReport-only and silently re-empty the layerGetVesselSnapshotresponse should includesnapshot.tankerReports[]with entries for the bbox query