Repository navigation
fix(asynqdl): truncate dead-letter errors on a rune boundary - #3964
Merged
Merged
Conversation
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
This was referenced Oct 21, 2026
lyingbug
pushed a commit
that referenced
this pull request
Oct 21, 2026
…4554) The dead-letter callback capped the knowledge error_message at 8192 bytes by slicing the string, which can split a multi-byte UTF-8 rune. PostgreSQL rejects the invalid UTF-8, so the update fails and the knowledge stays in its in-flight status. Back off to a rune boundary, as asynqdl does for last_error since #3964.
This was referenced Oct 24, 2026
7 of 9 tasks
7 of 9 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
asynqdl.truncateErrorlimitslast_errorto 8192 bytes by slicing the string at a byte offset (s[:max-len(suffix)]). If the error text contains multi-byte characters, such as Chinese error messages, file names or model output, the cut can land inside a rune.last_errorthen ends with a partial UTF-8 sequence.PostgreSQL rejects such a value (
invalid byte sequence for encoding "UTF8"), and so does MySQL with utf8mb4 in strict mode (Incorrect string value). TheInsertfails, and the middleware only logs that failure. The dead-letter row for the task is then lost, and it is exactly the long, verbose failures that hit this path.This change backs the cut off to the previous rune start with
utf8.RuneStartbefore appending...(truncated). ASCII-only errors are truncated exactly as before, and the result is still at most 8192 bytes.Type of Change
Related Issue
None. I found this while reading the dead-letter middleware. Similar earlier fixes: #3056, #813.
Testing
TestMiddleware_LongMultiByteErrorStaysValidUTF8. It runs a ~12 KB CJK error through the middleware and checks thatlast_erroris at most 8192 bytes, is valid UTF-8, and keeps the suffix.mainit fails withgot trailing bytes "\xa5解\xe6\x9e...(truncated)".go test ./internal/middleware/asynqdl/: okgo vet ./internal/middleware/asynqdl/: okgolangci-lint run --new-from-rev=origin/main ./internal/middleware/asynqdl/...: 0 issuesgit diff --check: cleanChecklist
git diff --check origin/main...HEADpassesgolangci-lint run --new-from-rev=origin/main ./...)website-docs/, Swagger annotations, etc.) (not applicable)