Skip to content

[BUG] Heartbeats received by a follower Admin do not refresh the master instance state #7778

Closed

Description

@Aias00

Problem

In cluster mode, /instance/beat is not included in the shipped cluster forwarding rules. On a non-master Admin, ClusterForwardFilter consequently passes the request to the local InstanceController, which updates that JVM's local heartbeat map. The master checker does not receive this observation.

If the reporting client uses an Admin load balancer or its configured destination is a follower, a successfully accepted heartbeat can coexist with an OFFLINE master view. This is separate from the missing-local-map null dereference in #7697: a read-side null guard alone does not deliver heartbeats to the status authority. It does not affect configurations that explicitly report successfully to every Admin including the master on each cycle.

Source evidence at current master

Baseline: b2bc43376a0146684faf6d45a6832d201fc61f45.

Verified reproduction

Recompiled the current filter, controller and checker sources on Java 17, using the actual forwarding/ignored lists parsed from the shipped application.yml. A controlled election service reports that the receiving node is a follower. A mock servlet POST to /instance/beat passes through the real filter and real controller into the follower checker. The master checker holds the same identity with an expired last observation.

Observed: the follower becomes/remains freshly ONLINE, the master timestamp is unchanged, and executing the master checker changes the instance to OFFLINE. The fixture prevented network calls and used two actual checker instances. It did not run a load balancer, live master election or HTTP authentication; those deployment boundaries still need integration coverage.

Acceptance criteria

  • Choose and document the cluster-wide authority for heartbeat updates; forward to the current master or use a coherent shared observation mechanism rather than isolated uncoordinated maps.
  • A valid report delivered through any supported Admin entry point refreshes the authoritative instance view within a stated bound.
  • Preserve namespace/identity, avoid forwarding loops and define behavior when master discovery is unavailable or changes during a report.
  • Add filter/controller regression coverage with the actual configured route lists, plus a two-Admin integration test using a load-balanced reporting destination.
  • Cover master handover, duplicate reports, failures/retry and legacy reporters; no false delivery acknowledgement when the authoritative update failed.
  • Keep this correction distinct from readiness semantics and from gateway upstream routing; do not add duplicate unbounded reporting loops.

Related: #7697 (status reads), #7718 (readiness), #7719 (configuration-application feedback), #7743 (slow destination isolation). No production source was modified for this reproduction.

Activity

  1. konlue commented on Oct 17, 2026

    @konlue
    Contributor

    /assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions