Repository navigation
fix(check): a Korean number and its counter count as one word in line reading time - #7343
Conversation
![heygengenesis[bot]](https://avatars.githubusercontent.com/in/2913576?s=60&v=4){kind=small}
There was a problem hiding this comment.
Request changes at dc55c8e8.
Blocker: Thai, Lao, Khmer and Burmese lines now count as one word. The new guard sends a run to the dictionary segmenter only if it contains Han or kana, so every other run counts as 1. Korean is spaced, so that's correct for it. But Thai, Lao, Khmer and Burmese don't use spaces either, and before this change ICU's dictionary split them. Same WORD_RUNS and segmenter as the head, old vs new count:
| text | main | head |
|---|---|---|
การทดสอบล้มเหลวสิบสองครั้งในวันนี้ (Thai) |
10 | 1 |
ທົດສອບລົ້ມເຫລວສິບສອງຄັ້ງ (Lao) |
5 | 1 |
ការធ្វើតេស្តបរាជ័យដប់ពីរដង (Khmer) |
5 | 1 |
စမ်းသပ်မှုဆယ့်နှစ်ကြိမ်ကျရှုံးခဲ့သည် (Burmese) |
7 | 1 |
So a long Thai caption now gets the 0.4 s short-line allowance, and line_unread misses it. That swaps a false warning for a missed one in a different script.
Suggested fix: flip the guard so that only Hangul skips the segmenter (e.g. /\p{Script=Hangul}/u.test(run) ? 1 : segment(run)). Then add a Thai case (readingWords("การทดสอบล้มเหลว…") > 1) next to the Japanese pin.
Verified otherwise:
npx vitest run src/utils/craftAudit.test.tsat head: 26 passed.fails without the fix:holds. With main'scraftAudit.tsand the PR's tests, "counts Korean by its spaces…" fails on실패 12회(3 ≠ 2).- The Japanese pins (
12回= 2,実行12回失敗= 4) and the Latin/%cases are unchanged.
There was a problem hiding this comment.
Approve at 7815874f.
My earlier blocker is fixed. Thai, Lao, Khmer and Myanmar now go through the dictionary segmenter again. Those are the unspaced scripts ICU has dictionaries for, along with Han and kana, so for every other script (Korean, Latin, …) one space-delimited run is one word. The new "still segments scripts written without spaces" test pins all four.
Verified at this head:
npx vitest run src/utils/craftAudit.test.ts: 27 passed.fails without the fix:holds. With main'scraftAudit.tsand the PR's tests, "counts Korean by its spaces…" fails.
Nit (not blocking): the test plan still says 26 passed; it's 27 now.
…-com#7336-heygen-com#7358 Adopt heygen-com#7304 (without CLI renderDiagnostic, needs heygen-com#6725), heygen-com#7331, heygen-com#7332, heygen-com#7334, heygen-com#7337, heygen-com#7336, heygen-com#7339, heygen-com#7289, heygen-com#5594, heygen-com#6900, heygen-com#7322, heygen-com#7342, heygen-com#7343, heygen-com#7350 (onProgress only), heygen-com#7352, heygen-com#7348, heygen-com#7344, heygen-com#7353, heygen-com#7356 and heygen-com#7355. Defer heygen-com#5708 (needs heygen-com#6911) and heygen-com#7345 (READ_VERSION 4 vs 7); heygen-com#7307 not applicable. Re-read 14 moved pins, pin 6 new PRs (monitored 126). Make checkReportFile module-private and suppress the cross-package closeTestChrome duplicate for fallow. Co-authored-by: Miguel Ángel <[email protected]> Co-authored-by: SophiaHuangCc <[email protected]> Co-authored-by: Dante <[email protected]> Co-authored-by: Muzzy5150 <[email protected]> Co-authored-by: ump45nose <[email protected]> Co-Authored-By: Claude Sonnet 5.5 <[email protected]> Co-Authored-By: Claude Opus 5.5 <[email protected]>
What
checkno longer counts a number and the Korean counter written against it (12회,3건) as two words, so a short Korean counter line gets the same 0.4 s short-line allowance asFailures: 12.Why
A film with Korean failure counters (
실패 12회) drewline_unreadwarnings that did not apply.readingWordsran every letter/number run through a Japanese dictionary segmenter. ICU cuts a digit run from the letters stuck to it in every locale (checkedja,ko,en,zh: all give12|회), so picking a Korean locale would not help.실패 12회counted 3 words, crossed the 2-word short-line limit, and needed a 1.05 s hold instead of 0.4 s.How
Only runs that contain Han, kana, Thai, Lao, Khmer or Myanmar letters go to the dictionary segmenter (these scripts have no spaces). Every other run (Korean, Latin, ...) is already one word per space-delimited run. Japanese counting is unchanged, and Thai, Lao, Khmer and Burmese lines are still segmented (test added):
12回stays 2 and実行12回失敗stays 4, pinned by a test.Size
A six-line fix in one function plus its tests. Nothing else in flight touches
readingWords, so there is no PR to fold it into.Test plan
cd packages/cli && npx vitest run src/utils/craftAudit.test.ts, 26 passed, 3 runs in a row.craftAudit.ts.craftAudit.tsgoes 5.09% to 5.05%.