Repository navigation
fix(admin): ignore heartbeats with an incomplete instance identity (#7764) - #7776
Conversation
Aias00
left a comment
There was a problem hiding this comment.
LGTM — the guard sits at the right level and I traced all in-repo producers before approving.
Evidence
- Websocket:
WebsocketCollector.onOpen(:121-124) throws when the namespace is blank, so an open session always has one. The bootstrap report at:200-211can only producenull, and only through the "session is closed" branch ofgetNamespaceId— exactly the case that used to create the:8080@grpc#nsstyle entry. - Long polling:
HttpLongPollingDataChangedListener:145,155forwards a namespace that defaults toSYS_DEFAULT_NAMESPACE_ID(:317-323). Note that this path can send a blank port when theX-Real-Portheader is missing (:147-151publishesStringUtils.EMPTY), so your guard also closes a real gap there. - Register flow:
AbstractShenyuClientRegisterServiceImpl:265-271forwardsuriRegisterDTO.getNamespaceId(), which the admin fills withSYS_DEFAULT_NAMESPACE_ID(seeRegistrationProtocolCompatibilityTest:120), and the client reporter defaultsshenyuConfig.getNamespace()to the same constant (ShenyuConfig:79).
Two asks (non-blocking, but please answer in the PR body)
/instance/beat(InstanceController:115-119) still returnsSUCCESSfor a dropped report, so a misconfigured client keeps beating forever with no signal on its side. You already validate the identity in the service; surfacing it as a 4xx from the controller (or at least documenting the silent drop) would save the next operator an afternoon.- Consider defaulting only the namespace —
StringUtils.defaultIfBlank(ns, SYS_DEFAULT_NAMESPACE_ID)— and keeping the hard drop for ip/port/type. An operator who setsshenyu.namespace=to blank currently gets a working (if unsightly) instance today and would get nothing after this change. Dropping is defensible; dropping silently is the part I would avoid either way.
| } | ||
|
|
||
| public void handleBeatInfo(final InstanceBeatInfoDTO instanceBeatInfoDTO) { | ||
| if (isBlankIdentity(instanceBeatInfoDTO)) { |
There was a problem hiding this comment.
Non-blocking: dropping the whole report also drops a client whose only problem is a blank namespace. Since every in-repo producer defaults it (ShenyuConfig:79, HttpLongPollingDataChangedListener:317-323, register flow via SYS_DEFAULT_NAMESPACE_ID), consider StringUtils.defaultIfBlank(ns, SYS_DEFAULT_NAMESPACE_ID) here and keep the hard drop for ip/port/type only. Also note /instance/beat still answers SUCCESS for a dropped report, so the client gets no signal at all (details in the review body).
There was a problem hiding this comment.
Done in — the namespace now defaults via StringUtils.defaultIfBlank(ns, SYS_DEFAULT_NAMESPACE_ID) before the guard (hard drop kept for ip/port/type only), and /instance/beat answers PARAMETER_ERROR for a rejected report. Details in the review follow-up comment below.
|
PTAL — both points are addressed in a follow-up commit (pushed above):
Tests updated for both (namespace defaulting, rejected-report signal). Apologies for the re-review round-trip — it felt better than merging the silent drop you flagged. |
Aias00
left a comment
There was a problem hiding this comment.
Re-approving after the follow-up. Both points from my previous review are addressed, and I checked the two claims that carry the fix.
Namespace defaulting is the correct value
Constants.SYS_DEFAULT_NAMESPACE_ID (649330b6-...) is the same value ClientRegisterConfig falls back to when shenyu.namespace is blank, so a blank beat now lands on the key that an already-registered instance has — fetchInstanceData loads stored rows under that id, which is why blank beats used to compute ip:port@type# and never match. Verified on the producer side too: HeartbeatListener:102-121 builds the DTO from shenyuConfig.getNamespace(), which is blank when the operator does not configure a namespace, so the in-code comment is accurate rather than an assumption.
The rejected-beat signal is contained
RegisterUtils.doHeartBeat:111-116 throws IOException when the body is not success, so the choice of PARAMETER_ERROR over a 4xx only matters through that path. Both callers contain it: HeartbeatListener#sendHeartbeat:122-126 catches per server and logs "will retry", and the HTTP long-polling listener does the same. That means a permanently incomplete identity yields one ERROR per beat interval on the client and cannot cancel the scheduleAtFixedRate reporter — a visible signal without a wire-contract change, which is what I asked for. A 4xx would have been contained identically; keeping 200 is the lower-risk choice, and I am fine with it.
Removing the duplicate setLastHeartBeatTime and the now-redundant namespace term in isBlankIdentity are both correct follow-ons.
Non-blocking
handleBeatInfonow mutates the caller's DTO (setNamespaceId). It keeps the key and the VO consistent in one place, but the method has gained an input side effect; a localdefaultIfBlank(...)passed to both the key builder and the VO would be purer.getInstanceKey(dto)is used elsewhere, so leaving it as is, is reasonable — just be aware the DTO handed in comes back modified.isBlankIdentityno longer checks the namespace, which is right after defaulting, but the name still reads like a general identity check. One comment ("namespace is defaulted before this guard") keeps someone from re-adding it.
CI
pr_build (2m59s) and rat_check are green on 0b8551e1. The e2e red is one genuine failure plus cascade: e2e-case (e2e-logging-rocketmq) fails at DividePluginTest.testDivide:146 Failed to consume RocketMQ access log (a 31s wait on the access-log pipeline), and the other three e2e-case jobs are cancelled with "The operation was canceled.". Nothing in this diff touches the logging plugin, the RocketMQ client or the divide plugin, so I read that as the known log-pipeline timing flake — but e2e is a required check, so please re-run it before merge.
|
Ping on state: this PR is approved on the code, but the required checks are red and it cannot be merged yet. The failing jobs are No code change is expected from you. Once the shared e2e problem is resolved, please rerun the failed jobs (or push a trivial rebase) so the required checks go green, and this can be merged. |
|
@konlue This PR is approved, but 4 checks are red: I inspected these lanes today and they fail for infrastructure reasons rather than on the diff — the Please rebase onto |
0b8551e to
d7bc1d7
Compare
|
Rebased onto the latest master to re-trigger the checks. |
Aias00
left a comment
There was a problem hiding this comment.
The guard is correct, and I checked the internal path as well: onInstanceInfoReport builds the DTO from InstanceInfoReportEvent, whose only producer (AbstractShenyuClientRegisterServiceImpl:265) filters on a non-blank host and a non-null port and passes InstanceTypeConstants.CLIENT_INSTANCE_TYPE, so nothing that reaches handleBeatInfo from the event bus loses its heartbeat because of this change. Defaulting a blank namespace to SYS_DEFAULT_NAMESPACE_ID also matches what every in-repo client sends.
Two notes, neither blocking:
beat()now reports the rejection in the response body (PARAMETER_ERROR) while still returning HTTP 200. A client that only checks the status code cannot tell the difference. SinceInstanceBeatInfoDTOhas no constraints at all, adding@NotBlankto ip/port/type would let@Validreject it with a 400 before the controller is entered, and the body would stay a plain success message.handleBeatInfomutates the caller's DTO when it defaults the namespace. Harmless for the two current callers, but aString namespaceId = defaultIfBlank(...)local would keep the method side-effect free.
Tests cover the four blank-identity shapes plus the namespace defaulting, which is the right scope.
|
Hi @konlue, this PR is already approved, but it currently cannot be merged. Here is the blocker and what is needed: Blocker What we need Once the branch is up to date and CI is green, we will squash-merge it. Thanks! |
d7bc1d7 to
a161e01
Compare
Aias00
left a comment
There was a problem hiding this comment.
Reviewed the incomplete-heartbeat guard and controller result. Invalid identities are rejected before cache mutation, and blank namespaces are normalized to the default namespace for compatibility with existing producers. No blocking finding in this patch.
InstanceControllerTest and InstanceCheckServiceTest passed all 50 tests, zero failures/errors/skips. Validation baseline: PR head plus a clean, uncommitted merge of master 52339b2 in the isolated worktree. No source was manually patched or pushed.
Used isolated worktrees and the Maven wrapper with JDK 21/project compiler settings. Checkstyle passed for the targeted Maven runs; RAT and JaCoCo were skipped. No full reactor/e2e pass is claimed.
Fixes #7764.
Summary
InstanceBeatInfoDTOcarries no identity constraints andInstanceCheckService#handleBeatInfoaccepts reports as-is. A beat with a blank IP reachescreateOrUpdate, whereselectOneByQueryomits the IP predicate for empty values — so an existing valid row can be overwritten with the blank identity.Modifications
handleBeatInfo: a report whoseinstanceIp,instancePort,instanceTypeornamespaceIdis blank is dropped with a warn log naming the identity.shenyu-register-protocoldeliberately keeps its wire contract JDK-only, and becausehandleBeatInfois the common entry for the controller, WebSocket, HTTP long polling and internal event paths — one check covers them all.Verifying this change
New
InstanceCheckServiceTest#testHandleBeatInfoIgnoresIncompleteIdentity: blank ip/port/type/namespace and whitespace variants create no live entry, and a followingsyncDB()issues zerocreateOrUpdatecalls. Fails on the current code (an entry keyed like:8080@grpc#nsis created).Targeted tests pass:
./mvnw -B -ntp -pl shenyu-admin test -Dtest=InstanceCheckServiceTest13/13 green.
Checkstyle passes:
clean.
Notes
With incomplete identities rejected at the entry, the optional-predicate identity lookup can no longer be hit by a broadened match from this path. The
/instance/beatresponse stays protocol-compatible; returning an explicit client error for malformed reports can be layered at the controller later if maintainers prefer. No valid report is affected — all four components are required identity parts.Review follow-up
namespaceIdis defaulted toSYS_DEFAULT_NAMESPACE_ID(all in-repo producers already default it), so onlyinstanceIp/instancePort/instanceTypeare treated as required identity parts; heartbeats missing any of those are dropped with a warn log naming the identity./instance/beatanswersPARAMETER_ERRORfor a rejected report (still HTTP 200, wire-compatible withRegisterUtils.doHeartBeat's success-string check), so a misconfigured client no longer beats silently forever.