Skip to content

fix(admin): ignore heartbeats with an incomplete instance identity (#7764) - #7776

Merged
Aias00 merged 2 commits into
apache:masterfrom
konlue:fix/blank-heartbeat-identity
Oct 27, 2026
Merged

Aias00 merged 2 commits into
apache:masterfrom
konlue:fix/blank-heartbeat-identity

Conversation

@konlue

@konlue konlue commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #7764.

Summary

InstanceBeatInfoDTO carries no identity constraints and InstanceCheckService#handleBeatInfo accepts reports as-is. A beat with a blank IP reaches createOrUpdate, where selectOneByQuery omits the IP predicate for empty values — so an existing valid row can be overwritten with the blank identity.

Modifications

  • Guard the shared entry point handleBeatInfo: a report whose instanceIp, instancePort, instanceType or namespaceId is blank is dropped with a warn log naming the identity.
  • The guard sits in the service (not on the DTO) because shenyu-register-protocol deliberately keeps its wire contract JDK-only, and because handleBeatInfo is the common entry for the controller, WebSocket, HTTP long polling and internal event paths — one check covers them all.

Verifying this change

  • New InstanceCheckServiceTest#testHandleBeatInfoIgnoresIncompleteIdentity: blank ip/port/type/namespace and whitespace variants create no live entry, and a following syncDB() issues zero createOrUpdate calls. Fails on the current code (an entry keyed like :8080@grpc#ns is created).

  • Targeted tests pass:

    ./mvnw -B -ntp -pl shenyu-admin test -Dtest=InstanceCheckServiceTest

    13/13 green.

  • Checkstyle passes:

    ./mvnw -B -ntp -pl shenyu-admin checkstyle:check

    clean.

Notes

With incomplete identities rejected at the entry, the optional-predicate identity lookup can no longer be hit by a broadened match from this path. The /instance/beat response stays protocol-compatible; returning an explicit client error for malformed reports can be layered at the controller later if maintainers prefer. No valid report is affected — all four components are required identity parts.

Review follow-up

  • A blank namespaceId is defaulted to SYS_DEFAULT_NAMESPACE_ID (all in-repo producers already default it), so only instanceIp / instancePort / instanceType are treated as required identity parts; heartbeats missing any of those are dropped with a warn log naming the identity.
  • /instance/beat answers PARAMETER_ERROR for a rejected report (still HTTP 200, wire-compatible with RegisterUtils.doHeartBeat's success-string check), so a misconfigured client no longer beats silently forever.

Aias00
Aias00 previously approved these changes Oct 17, 2026

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — the guard sits at the right level and I traced all in-repo producers before approving.

Evidence

  • Websocket: WebsocketCollector.onOpen (:121-124) throws when the namespace is blank, so an open session always has one. The bootstrap report at :200-211 can only produce null, and only through the "session is closed" branch of getNamespaceId — exactly the case that used to create the :8080@grpc#ns style entry.
  • Long polling: HttpLongPollingDataChangedListener:145,155 forwards a namespace that defaults to SYS_DEFAULT_NAMESPACE_ID (:317-323). Note that this path can send a blank port when the X-Real-Port header is missing (:147-151 publishes StringUtils.EMPTY), so your guard also closes a real gap there.
  • Register flow: AbstractShenyuClientRegisterServiceImpl:265-271 forwards uriRegisterDTO.getNamespaceId(), which the admin fills with SYS_DEFAULT_NAMESPACE_ID (see RegistrationProtocolCompatibilityTest:120), and the client reporter defaults shenyuConfig.getNamespace() to the same constant (ShenyuConfig:79).

Two asks (non-blocking, but please answer in the PR body)

  1. /instance/beat (InstanceController:115-119) still returns SUCCESS for a dropped report, so a misconfigured client keeps beating forever with no signal on its side. You already validate the identity in the service; surfacing it as a 4xx from the controller (or at least documenting the silent drop) would save the next operator an afternoon.
  2. Consider defaulting only the namespace — StringUtils.defaultIfBlank(ns, SYS_DEFAULT_NAMESPACE_ID) — and keeping the hard drop for ip/port/type. An operator who sets shenyu.namespace= to blank currently gets a working (if unsightly) instance today and would get nothing after this change. Dropping is defensible; dropping silently is the part I would avoid either way.

}

public void handleBeatInfo(final InstanceBeatInfoDTO instanceBeatInfoDTO) {
if (isBlankIdentity(instanceBeatInfoDTO)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking: dropping the whole report also drops a client whose only problem is a blank namespace. Since every in-repo producer defaults it (ShenyuConfig:79, HttpLongPollingDataChangedListener:317-323, register flow via SYS_DEFAULT_NAMESPACE_ID), consider StringUtils.defaultIfBlank(ns, SYS_DEFAULT_NAMESPACE_ID) here and keep the hard drop for ip/port/type only. Also note /instance/beat still answers SUCCESS for a dropped report, so the client gets no signal at all (details in the review body).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in — the namespace now defaults via StringUtils.defaultIfBlank(ns, SYS_DEFAULT_NAMESPACE_ID) before the guard (hard drop kept for ip/port/type only), and /instance/beat answers PARAMETER_ERROR for a rejected report. Details in the review follow-up comment below.

@konlue

konlue commented Oct 17, 2026

Copy link
Copy Markdown
Contributor Author

PTAL — both points are addressed in a follow-up commit (pushed above):

  1. Namespace defaulting — done as you suggested.
    handleBeatInfo now applies StringUtils.defaultIfBlank(namespaceId, SYS_DEFAULT_NAMESPACE_ID) before the guard, and the hard drop covers ip / port / type only. A blank shenyu.namespace therefore registers a working instance under the default namespace instead of being dropped.

  2. Signal for rejected reports — done, with a wire-compatible choice.
    /instance/beat keeps the 200 status but answers ShenyuResultMessage.PARAMETER_ERROR when the report is rejected — RegisterUtils.doHeartBeat treats any non-"success" body as a failure and logs/retries, so existing clients get a visible signal without a contract change. Happy to switch to a proper 4xx if you'd rather have the status-code signal.

Tests updated for both (namespace defaulting, rejected-report signal).

Apologies for the re-review round-trip — it felt better than merging the silent drop you flagged.

Aias00
Aias00 previously approved these changes Oct 18, 2026

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-approving after the follow-up. Both points from my previous review are addressed, and I checked the two claims that carry the fix.

Namespace defaulting is the correct value

Constants.SYS_DEFAULT_NAMESPACE_ID (649330b6-...) is the same value ClientRegisterConfig falls back to when shenyu.namespace is blank, so a blank beat now lands on the key that an already-registered instance has — fetchInstanceData loads stored rows under that id, which is why blank beats used to compute ip:port@type# and never match. Verified on the producer side too: HeartbeatListener:102-121 builds the DTO from shenyuConfig.getNamespace(), which is blank when the operator does not configure a namespace, so the in-code comment is accurate rather than an assumption.

The rejected-beat signal is contained

RegisterUtils.doHeartBeat:111-116 throws IOException when the body is not success, so the choice of PARAMETER_ERROR over a 4xx only matters through that path. Both callers contain it: HeartbeatListener#sendHeartbeat:122-126 catches per server and logs "will retry", and the HTTP long-polling listener does the same. That means a permanently incomplete identity yields one ERROR per beat interval on the client and cannot cancel the scheduleAtFixedRate reporter — a visible signal without a wire-contract change, which is what I asked for. A 4xx would have been contained identically; keeping 200 is the lower-risk choice, and I am fine with it.

Removing the duplicate setLastHeartBeatTime and the now-redundant namespace term in isBlankIdentity are both correct follow-ons.

Non-blocking

  1. handleBeatInfo now mutates the caller's DTO (setNamespaceId). It keeps the key and the VO consistent in one place, but the method has gained an input side effect; a local defaultIfBlank(...) passed to both the key builder and the VO would be purer. getInstanceKey(dto) is used elsewhere, so leaving it as is, is reasonable — just be aware the DTO handed in comes back modified.
  2. isBlankIdentity no longer checks the namespace, which is right after defaulting, but the name still reads like a general identity check. One comment ("namespace is defaulted before this guard") keeps someone from re-adding it.

CI

pr_build (2m59s) and rat_check are green on 0b8551e1. The e2e red is one genuine failure plus cascade: e2e-case (e2e-logging-rocketmq) fails at DividePluginTest.testDivide:146 Failed to consume RocketMQ access log (a 31s wait on the access-log pipeline), and the other three e2e-case jobs are cancelled with "The operation was canceled.". Nothing in this diff touches the logging plugin, the RocketMQ client or the divide plugin, so I read that as the known log-pipeline timing flake — but e2e is a required check, so please re-run it before merge.

@Aias00

Aias00 commented Oct 23, 2026

Copy link
Copy Markdown
Contributor

Ping on state: this PR is approved on the code, but the required checks are red and it cannot be merged yet.

The failing jobs are e2e / e2e-case / list-contract / native-sql, which are currently failing across the repository and are tracked in #7970. The build job in this run does not fail on compilation: it stops at the gate step that consumes the e2e result ([[ "failure" == "success" ]] || exit -1), so there is nothing in this PR to fix for that.

No code change is expected from you. Once the shared e2e problem is resolved, please rerun the failed jobs (or push a trivial rebase) so the required checks go green, and this can be merged.

@Aias00

Aias00 commented Oct 24, 2026

Copy link
Copy Markdown
Contributor

@konlue This PR is approved, but 4 checks are red: e2e, e2e-case (e2e-logging-rocketmq-compose, shenyu-e2e-case-logging-rocketmq, :shenyu-examples-http), e2e-case (e2e-grpc-sync-compose, shenyu-e2e-case-grpc, :shenyu-examples-grpc), e2e-case (e2e-apache-dubbo-sync-compose, shenyu-e2e-case-apache-dubbo, :shenyu-examples-apache-du....

I inspected these lanes today and they fail for infrastructure reasons rather than on the diff — the integrated-test jobs die while starting their containers (Elasticsearch /_cat/health returning 000 until the job is cancelled), and the http lane also hits the known CachePluginTest.testMemoryModle / testRedisModle flake. The same jobs are red on master and on unrelated PRs right now.

Please rebase onto master and push (or ask for a re-run) so the checks execute again. If a case is still failing on a fresh run, paste the test name and I'll look properly.

@konlue

konlue commented Oct 24, 2026

Copy link
Copy Markdown
Contributor Author

Rebased onto the latest master to re-trigger the checks.

Aias00
Aias00 previously approved these changes Oct 25, 2026

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guard is correct, and I checked the internal path as well: onInstanceInfoReport builds the DTO from InstanceInfoReportEvent, whose only producer (AbstractShenyuClientRegisterServiceImpl:265) filters on a non-blank host and a non-null port and passes InstanceTypeConstants.CLIENT_INSTANCE_TYPE, so nothing that reaches handleBeatInfo from the event bus loses its heartbeat because of this change. Defaulting a blank namespace to SYS_DEFAULT_NAMESPACE_ID also matches what every in-repo client sends.

Two notes, neither blocking:

  1. beat() now reports the rejection in the response body (PARAMETER_ERROR) while still returning HTTP 200. A client that only checks the status code cannot tell the difference. Since InstanceBeatInfoDTO has no constraints at all, adding @NotBlank to ip/port/type would let @Valid reject it with a 400 before the controller is entered, and the body would stay a plain success message.
  2. handleBeatInfo mutates the caller's DTO when it defaults the namespace. Harmless for the two current callers, but a String namespaceId = defaultIfBlank(...) local would keep the method side-effect free.

Tests cover the four blank-identity shapes plus the namespace defaulting, which is the right scope.

@Aias00

Aias00 commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

Hi @konlue, this PR is already approved, but it currently cannot be merged. Here is the blocker and what is needed:

Blocker
This PR is unmergeable (conflict) against master.

What we need
Please rebase onto the latest master, resolve conflicts, and re-push.

Once the branch is up to date and CI is green, we will squash-merge it. Thanks!

@Aias00 Aias00 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the incomplete-heartbeat guard and controller result. Invalid identities are rejected before cache mutation, and blank namespaces are normalized to the default namespace for compatibility with existing producers. No blocking finding in this patch.

InstanceControllerTest and InstanceCheckServiceTest passed all 50 tests, zero failures/errors/skips. Validation baseline: PR head plus a clean, uncommitted merge of master 52339b2 in the isolated worktree. No source was manually patched or pushed.

Used isolated worktrees and the Maven wrapper with JDK 21/project compiler settings. Checkstyle passed for the targeted Maven runs; RAT and JaCoCo were skipped. No full reactor/e2e pass is claimed.

@Aias00
Aias00 merged commit 4449fa7 into apache:master Oct 27, 2026
55 checks passed
@Aias00 Aias00 added this to the 2.7.2 milestone Nov 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Blank heartbeat identity can overwrite an existing instance record

2 participants