Repository navigation
fix(pbl): don't cut an emoji in half when shortening evaluator prompt text - #2124
Merged
Merged
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
wyuc
approved these changes
Nov 3, 2026
wyuc
left a comment
Contributor
There was a problem hiding this comment.
Thanks @kevin9327. The cut now moves back only when it would split a surrogate pair, so plain text is unchanged. feedback.slice(0, 280) further down the same file still has the old pattern. It's model-written text, so it doesn't need to block this; a follow-up is welcome.
5 of 13 tasks
This was referenced Nov 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The evaluator prompt builders in
lib/pbl/v2/operations/runtime/eval-prompts.tsshorten long learner text withtext.slice(0, maxLength) + '…', which counts UTF-16 code units. When an emoji (or any character outside the BMP) sits across the limit, the slice keeps only its high surrogate, so the prompt sent to the evaluator model carries a lone surrogate. That is invalid Unicode: it shows up as\ud83din the JSON request body, and some providers reject or replace it.Two places do this:
promptText, used byformatProjectSynthesisChecksfor the core concept (240), question, learner answer (700) and quality (40) in the final evaluation.buildMilestoneEvalPrompt(140).Related Issues
None filed. I found this while reading the evaluator prompt builders.
Changes
sliceCodeUnits(text, maxLength)helper that cuts one code unit earlier when the cut would split a surrogate pair, so the emoji is dropped whole. Limits stay in code units, so plain text is cut exactly where it was before.promptTextand for the closing-check answer.tests/pbl/v2/eval-prompts.test.ts.Type of Change
Verification
Steps to reproduce / test
stage_synthesis_checkwhoselearner_answeris 699 ASCII characters followed by😀.formatProjectSynthesisChecks(project).\ud83d…(a lone high surrogate). After: it ends with the 699 characters and….What you personally verified
main(expected true to be falsefrom the lone-surrogate check) and pass with the change. The plain-text test passes on both.tests/pbl/v2/eval-prompts.test.tspass with vitest, run with a minimal config (the@/alias and the workspace packages pointed at their sources) instead of a fullpnpm install, so I did not run the repo's global setup files.prettier --check(3.8.1) passes on both changed files.pnpm lintornpx tsc --noEmitlocally. The change is a typedstringhelper and two call sites.Evidence
Before the fix:
After:
pnpm check && pnpm lint && npx tsc --noEmit)Checklist
AI-assisted: I used an AI coding assistant for this change, reviewed the code and ran the tests.