Repository navigation
[ISSUE #7716] Normalize instance information across reporting paths. - #7785
Conversation
Aias00
left a comment
There was a problem hiding this comment.
Approving. The two defects are real and each is fixed at its own layer; I traced the whole write/read path to check that the string contract still holds.
Evidence for the two symptoms
HTTP path — HttpSyncDataService:219 sends SystemInfoUtils.getSystemInfo() as a raw header, while HttpLongPollingDataChangedListener:146 reads it with defaultString(..., "") and :152 re-encoded it with toJson(...). The dashboard's single parse therefore produced a String, not an object. Passing the header text through unchanged is the correct fix.
WebSocket path — ShenyuWebsocketClient:426-433 wraps the same payload into Map.of(BOOTSTRAP_INSTANCE_INFO, info) and serializes the wrapper, so the admin receives a JSON object whose value is object text. convertToMap then decoded recursively, which is why {"note":"{\"nested\":true}"} came out mutated and why malformed optional metadata could throw. Extracting the element from the outer tree (isJsonPrimitive() && isString() ? getAsString() : toString()) keeps the outer encoding while leaving nested values alone — that is the right read point.
Write / read convergence
onInstanceInfoReport (InstanceCheckService:196+) builds the DTO and calls handleBeatInfo, so the normalization at handleBeatInfo is on the path into instanceHealthBeatInfo, and syncDB -> InstanceInfoServiceImpl#createOrUpdate persists that normalized value. Reading normalizes again in buildInstanceInfoVO:114, which covers findById, list (also used by fetchInstanceData) and the paged query, and it does so without writing back — the mapper test that asserts the stored row stays double-quoted while findById returns the decoded text is exactly the assertion I wanted to see. normalize decoding at most one legacy layer, and re-entering it being idempotent, is what keeps this from becoming an open-ended decode chain.
Non-blocking
normalizereturnselement.toString(), so a stored/reported value with whitespace or unusual key order is compacted on the display path. Harmless, but it meansnormalizeis not a string-level identity for pretty-printed input; the tests only feed compact text. A one-line comment on the canonical-form expectation would prevent a future reader from treating it as a no-op.handleBeatInfonormalizes only in theelsebranch (first registration); an instance already ininstanceHealthBeatInfokeeps the value it was created with andsyncDBwrites it back unchanged. That is pre-existing behaviour (metadata is never refreshed), but combined with this change it means a long-running admin converges only after a restart. Worth one line in the description, because the issue is about what the dashboard shows.WebsocketCollector#onMessagenow callsGsonUtils.getInstance().fromJson(message, JsonObject.class). A message that contains the marker but is not a JSON object throwsJsonSyntaxExceptionout of@OnMessage; master behaved the same viaconvertToMap, so it is not a regression — but atry/catchthat logs and returns would be strictly better for a websocket entry point, and it is one line.- The HTTP path with the header absent is only covered implicitly. The parameterized websocket test covers null/empty, so a direct case on
doLongPollingwithout the header, asserting the event carries""and the heartbeat still lands, would close it.
CI
gh pr checks reported nothing on f375e0be; the runs were action_required (first-time contributor), so I approved the ci workflow run. The change spans 11 files and rewrites JSON semantics at three boundaries, so I am not recording an approval before pr_build is green on this head — I will flip it as soon as it is.
Aias00
left a comment
There was a problem hiding this comment.
Follow-up: pr_build is green on the current head (3m9s), so the CI evidence I was waiting for now exists. Everything in my previous review stands — flipping to an approval.
|
Pls add my WeChat:aias00 |
Aias00
left a comment
There was a problem hiding this comment.
Approved.
Good normalization work across all instance-info reporting paths. The HTTP long-polling listener no longer double-encodes the header JSON (the bug that wrapped object text in an extra string); the WebSocket collector reads only the outer frame and preserves inner JSON strings instead of re-serializing; and the new InstanceInfoUtils.normalize() canonicalizes legacy/over-encoded rows for display without rewriting stored values. Tests cover legacy compat, malformed-frame resilience, and the nested-string case (verified that double-wrapping collapses to {} as intended).
Minor note: CI checks have not reported on this branch yet; please confirm the workflow passes before merging.
|
Thanks for reviewing and approving the latest changes! The 8 workflow runs for the latest commit (78d2d4d) are still awaiting maintainer approval. Could you please approve them to run? I'll follow up on the results, especially the previous HTTP integration test timeout. |
|
I ran the HTTP integration tests in my fork against the latest PR commit, 78d2d4d, using Ubuntu 24.04, JDK 17, and Docker. I only added a workflow to run the tests in my fork, following the upstream HTTP setup with the Maven Wrapper. No application code, test cases, or timeout settings were changed. Run and test reports · Workflow diff I couldn't reproduce the earlier timeout in this run, though its cause is still unclear. Sharing the results here for reference while the upstream CI awaits approval. |
|
Follow-up on master status, plus one inconsistency to decide on. ContextMaster ( The cause is that those two tests use non-JSON payloads ( The inconsistency worth fixing
if (instanceHealthBeatInfo.containsKey(instanceKey)) {
...
if (StringUtils.isNotBlank(instanceBeatInfoDTO.getInstanceInfo())) {
instanceInfoVO.setInstanceInfo(instanceBeatInfoDTO.getInstanceInfo()); // raw, no normalize
}
} else {
...
instanceInfoVO.setInstanceInfo(InstanceInfoUtils.normalize(instanceBeatInfoDTO.getInstanceInfo())); // normalized
}For a non-JSON value the stored There is also a wording mismatch: the other call site documents normalize as "Normalize legacy rows for display without rewriting the stored value", while this call site does rewrite the stored value. Two coherent options, your call:
Either is fine as long as the storage contract is stated in one place. Option 1 is the smaller delta from what is on master today. |
Refs #7716. Scope coordination: #7716 (comment)
HTTP synchronization serializes an already-encoded instance-information string again, so the dashboard's single parse produces a string rather than an object. The ordinary WebSocket case already passes because
convertToMaprecursively decodes JSON-looking strings; that conversion instead changes nested metadata string values and can throw on malformed optional metadata.Keep
instanceInfoas a string containing JSON object text. Preserve HTTP text and extract WebSocket metadata from the outer JSON tree without recursive conversion. Normalize optional metadata at the shared heartbeat boundary and when reading stored records, accepting one known legacy JSON-string wrapper. Missing, malformed and non-object metadata falls back to{}without preventing heartbeat processing. Do not log metadata payloads or migrate rows on reads.Malformed WebSocket outer frames are ignored without logging their payload; a subsequent valid report still publishes a heartbeat. Direct tests cover a missing HTTP metadata header and whitespace compaction without key sorting or nested-string decoding.
Validation:
clean verify: all 28 projects succeeded; 2,744 reported tests, 2,742 passed and 2 skipped, no failures/errors. Admin itself reported 2,029 tests with 1 skipped. All 28 Checkstyle and 28 Apache RAT executions passed.git diff --checkpasses.Compatibility: the database/API field stays a string; current JSON-object text and one legacy string-wrapped object are accepted. Existing raw rows are not rewritten by read operations. Heartbeat normalization applies on first registration; existing cached instances keep their metadata under the pre-existing refresh behavior until the cache entry is recreated (for example, after an Admin restart). Display reads normalize either representation. Do not repeatedly decode arbitrary string layers or nested object values.
Coverage limits: no live mixed-version Admin/gateway deployment, complete dashboard rendering or local Docker/e2e run. Dashboard's existing one-pass JSON formatting was inspected in source; H2 and method-level regressions are executed locally. Unrelated instance metadata-refresh/type-identity work remains outside this change.
Make sure that:
./mvnw clean install -Dmaven.javadoc.skip=true. Validation covers Admin and its 28-project prerequisite reactor as documented above; a whole-repository clean install was not run for this change.