fix(ui): emit parseable layout JSON for malformed UTF-8 - #2913

Thanks for opening this — it has been seen, and it is queued.

This note is automated, but it is not a brush-off: it exists so you know where your PR stands instead of having to guess from silence.

Current review status: working through a backlog. 0.9.1-rc.1 is out, so the release freeze that held reviews is over — but it left a large queue of open pull requests behind it, and we are reading through them oldest-first. The background is in discussion #1144.

What that means for this PR, concretely:

• It will not be closed for inactivity. No stale bot touches pull requests here.
• It may still sit a while before a human reads it. That is on us, not on you.
• Older PRs are read first, so a recent one is not being skipped — it is behind a queue.

Things that will genuinely speed it up whenever review does happen:

• Keep it rebased on main — the tree is moving quickly right now, and a conflicting branch cannot be reviewed as the diff you intended.
• Get CI green, or say which failures you believe are pre-existing.
• Keep the change to one claim. Bundled features and refactors get split before they get merged, which costs you a round trip.
• Every commit needs a sign-off (git commit -s) — CI enforces DCO.

If this fixes a bug, a reproduction we can run is worth more than a description of the symptom.

Thanks for contributing, and sorry in advance for the wait.

Thank you, Matt - this is a really nice fix. You found the exact mismatch: the layout writer let malformed UTF-8 through with ALLOW_INVALID_UNICODE, and handle_layout then re-read that same string with a strict parser, so a single badly encoded heading was enough to turn the whole Galaxy/Architecture request into "JSON parse failed". Pairing ALLOW_INVALID_UNICODE with ESCAPE_UNICODE is the documented yyjson way to get U+FFFD instead of raw bytes, and it is the smallest change that makes the writer and the reader agree.

We checked it against every class of broken input we could think of (lone continuation bytes, truncated 2/3/4-byte sequences, overlong forms including an overlong NUL, UTF-16 surrogates and CESU-8 pairs, code points above U+10FFFF, 5- and 6-byte forms, 0xFE/0xFF) and pushed each one through the same steps handle_layout takes. All of them now come back as strict, valid JSON with U+FFFD in place of the bad bytes, while valid text (accents, CJK, emoji, U+10FFFF, BOM, U+2028/2029) still decodes byte for byte and the final /api/layout body is unchanged for valid input. Your new test fails on main at the strict re-read and passes with your change, so it really pins the bug - and covering quotes, backslashes and newlines right next to the bad bytes, plus the empty layout and NaN/Inf cases, is exactly the kind of coverage that keeps this from coming back.

We pushed two small commits on top of your branch:

1. style(ui): clang-format the layout writer call - the three lines of the yyjson_mut_write_opts call are rewrapped to the repository's clang-format style. It is whitespace only; the compiled code is identical. (src/ui is not part of the CI format check yet, which is why lint stayed green.)
2. Merge origin/main - another change that just merged also edits tests/test_ui.c (different tests, no conflict), so we merged current main to get a fresh CI run on the exact tree that will merge.

Nothing else changed. If you push to this branch again, please pull first so these commits are not lost. Thanks again for the careful work - we will merge as soon as CI is green on the new head.

Merged - thank you very much, Matt! Galaxy and Architecture now get strict, valid JSON from /api/layout even when a heading or label contains malformed UTF-8: the bad bytes come back as U+FFFD instead of failing the whole request. Thanks again for the precise diagnosis and the thorough test.

Appreciate the quick merge, @DeusData. With malformed UTF-8 now escaped to U+FFFD, the layout JSON should parse cleanly again and Galaxy and Architecture can draw the graph.