Skip to content

fix(commit): recompose commit text after compatibility folding - #10794

Merged
H4vC merged 1 commit into
can1357:mainfrom
ntdatt812:fix/commit-unicode-recompose
Sep 4, 2026
Merged

H4vC merged 1 commit into
can1357:mainfrom
ntdatt812:fix/commit-unicode-recompose

Conversation

@ntdatt812

Copy link
Copy Markdown
Contributor

What

normalizeCommitUnicode folds compatibility forms (fractions, super/subscripts, ligatures, fullwidth) to ASCII by running NFKD, but never recomposes afterwards. NFKD also decomposes every precomposed letter into a base plus combining marks, which is not part of that intent — and the function keeps the decomposition.

This change appends .normalize("NFC") to the pipeline. Compatibility mappings never recompose, so all the intended foldings survive; only the author's letters are put back together.

Why

postProcessCommitMessage measures the summary in UTF-8 bytes after that call:

if (Buffer.byteLength(summary) > config.summaryHardLimit) {
    throw new Error(`Summary exceeds ${config.summaryHardLimit} bytes`);
}

Decomposition inflates the measurement for any script that uses precomposed letters — Vietnamese by ~1.23x, Hangul by ~2.4x — so a summary that fits gets rejected for exceeding the limit. Measured with the repo's own code:

summary as written after normalizeCommitUnicode
sửa lỗi bộ đệm không được giải phóng khi người dùng đóng phiên làm việc giữa chừng 113 B 139 B
한국어 입력기에서 조합 중인 글자가 사라지는 문제를 수정 79 B 187 B

The limit is 128 bytes, so both are rejected. postProcessCommitMessage is called uncaught in validateAndProcess, so the throw propagates out of generateConventionalCommit and commit generation fails rather than degrading. The upstream cap does not help either: generate.ts:313 uses sliceCodePoints(chosen, 0, config.summaryHardLimit), which counts code points against a budget the guard then reads as bytes.

Text that is short enough to pass is still affected — it reaches Git decomposed rather than as the author typed it.

Testing

Reproduced and confirmed fixed against the byte guard itself, through the repo's own code path:

  • Before — postProcessCommitMessage with the 113-byte Vietnamese summary above threw Summary exceeds 128 bytes. normalizeCommitUnicode returned 139 bytes / 109 code points, byte-identical to NFKD of the input.
  • After — the same call returns the summary unchanged: 113 bytes, 82 code points, strictly equal to the input string.

Four tests added to commit-conventional.test.ts. Reverting the one-line change fails three of them, including accepts non-ASCII summaries that fit the byte limit as written, which fails with the production error Summary exceeds 128 bytes. The fourth (still folds compatibility forms down to ASCII) passes either way by design — it is the regression guard proving NFC does not undo the folding, checked over ½ ² ₂ fi ① full ≠ “” → α.

Checks run:

  • bun test packages/coding-agent/test/commit-conventional.test.ts — 20 pass, 0 fail
  • bun run check:ts — passes across the workspace (oxlint, oxfmt, and tsgo --noEmit per package). This change is TypeScript only, so check:rs was not run.
  • commit-execute.test.ts has one failure (abortOnGitFailure (issue #7834) > surfaces a refusing hook's message...). It reproduces identically on an unpatched tree — same test name, same 2 pass / 1 fail — so it does not come from this change.

  • bun run check:ts passes (Rust untouched; check:rs not run)
  • Tested locally
  • CHANGELOG updated (if user-facing)

@ntdatt812
ntdatt812 force-pushed the fix/commit-unicode-recompose branch from 19716a7 to 6cb5007 Compare September 4, 2026 09:28
@roboomp roboomp added cli CLI commands and arguments fix review:p0 triaged labels Sep 4, 2026

@roboomp roboomp left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm — P0: normalizeCommitUnicode() now fixes the UTF-8 byte-limit regression while retaining the existing compatibility folds; focused coverage exercises summaries, body details, footers, and ASCII mappings.
Verified bun test packages/coding-agent/test/commit-conventional.test.ts: 20 pass, 0 fail. Thanks @ntdatt812.

`normalizeCommitUnicode` runs NFKD to fold fractions, super/subscripts,
ligatures and fullwidth forms down to ASCII, but never recomposes. Every
precomposed letter is left split into a base plus combining marks, so the
byte guard in `postProcessCommitMessage` measures a decomposition the
author never wrote — Vietnamese inflates ~1.23x and Hangul ~2.4x.

A summary of 113 UTF-8 bytes is measured at 139 and rejected with
"Summary exceeds 128 bytes"; nothing catches that throw, so commit
generation fails outright.

Recompose with NFC after the replacement passes. Compatibility mappings
never recompose, so every folding is preserved while the author's letters
are restored and measured at the length they were written.
@H4vC
H4vC force-pushed the fix/commit-unicode-recompose branch from 6cb5007 to 7ef0883 Compare September 4, 2026 15:34
@H4vC
H4vC merged commit ddde7db into can1357:main Sep 4, 2026
darsrc pushed a commit to darsrc/dar-oh-my-pi that referenced this pull request Oct 10, 2026
…compose

fix(commit): recompose commit text after compatibility folding
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cli CLI commands and arguments fix review:p0 triaged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants