Charset: Create compat-utf8.php module with fallback code.
#9825
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the Core Committers: Use this line as a base for the props when committing in SVN: To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
Test using WordPress PlaygroundThe changes in this pull request can previewed and tested using a WordPress Playground instance. WordPress Playground is an experimental project that creates a full WordPress instance entirely within the browser. Some things to be aware of
For more details about these limitations and more, check out the Limitations page in the WordPress Playground documentation. |
This was referenced Sep 10, 2025
dmsnell
force-pushed
the
utf8/new-compat-module
branch
3 times, most recently
from
219f167 to
b77dc49
Compare
dmsnell
force-pushed
the
utf8/new-compat-module
branch
from
b77dc49 to
28bcb62
Compare
pento
pushed a commit
that referenced
this pull request
Sep 15, 2025
This is the second in a series of patches to modernize and standardize UTF-8 handling. When the fallback UTF-8 validation code was added it was placed inside formatting.php; however, that validation logic can be reused for a number of related UTF-8 functions. To faciliate this it should move into a new location and be loaded early. This patch is the first half of doing that, whereby the original fallback function is moved unchanged to the `compat-utf8.php` module. The follow-up patch will abstract the UTF-8 scanning logic for reuse. Splitting this into a move and a separate change involves an extra step, but faciliates tracking the heritage of the code through the changes. Developed in #9825 Discussed in https://core.trac.wordpress.org/ticket/63863 Follow-up to: [60630]. See #63863. git-svn-id: https://develop.svn.wordpress.org/trunk@60743 602fd350-edb4-49c9-b593-d223f7449a82
markjaquith
pushed a commit
to markjaquith/WordPress
that referenced
this pull request
Sep 15, 2025
This is the second in a series of patches to modernize and standardize UTF-8 handling. When the fallback UTF-8 validation code was added it was placed inside formatting.php; however, that validation logic can be reused for a number of related UTF-8 functions. To faciliate this it should move into a new location and be loaded early. This patch is the first half of doing that, whereby the original fallback function is moved unchanged to the `compat-utf8.php` module. The follow-up patch will abstract the UTF-8 scanning logic for reuse. Splitting this into a move and a separate change involves an extra step, but faciliates tracking the heritage of the code through the changes. Developed in WordPress/wordpress-develop#9825 Discussed in https://core.trac.wordpress.org/ticket/63863 Follow-up to: [60630]. See #63863. Built from https://develop.svn.wordpress.org/trunk@60743 git-svn-id: http://core.svn.wordpress.org/trunk@60079 1a063a9b-81f0-0310-95a4-ce76da25c4cd
Member
Author
|
Merged in 31cac36 |
github-actions bot
pushed a commit
to platformsh/wordpress-performance
that referenced
this pull request
Sep 15, 2025
This is the second in a series of patches to modernize and standardize UTF-8 handling. When the fallback UTF-8 validation code was added it was placed inside formatting.php; however, that validation logic can be reused for a number of related UTF-8 functions. To faciliate this it should move into a new location and be loaded early. This patch is the first half of doing that, whereby the original fallback function is moved unchanged to the `compat-utf8.php` module. The follow-up patch will abstract the UTF-8 scanning logic for reuse. Splitting this into a move and a separate change involves an extra step, but faciliates tracking the heritage of the code through the changes. Developed in WordPress/wordpress-develop#9825 Discussed in https://core.trac.wordpress.org/ticket/63863 Follow-up to: [60630]. See #63863. Built from https://develop.svn.wordpress.org/trunk@60743 git-svn-id: https://core.svn.wordpress.org/trunk@60079 1a063a9b-81f0-0310-95a4-ce76da25c4cd
jonnynews
pushed a commit
to spacedmonkey/wordpress-develop
that referenced
this pull request
Sep 24, 2025
This is the second in a series of patches to modernize and standardize UTF-8 handling. When the fallback UTF-8 validation code was added it was placed inside formatting.php; however, that validation logic can be reused for a number of related UTF-8 functions. To faciliate this it should move into a new location and be loaded early. This patch is the first half of doing that, whereby the original fallback function is moved unchanged to the `compat-utf8.php` module. The follow-up patch will abstract the UTF-8 scanning logic for reuse. Splitting this into a move and a separate change involves an extra step, but faciliates tracking the heritage of the code through the changes. Developed in WordPress#9825 Discussed in https://core.trac.wordpress.org/ticket/63863 Follow-up to: [60630]. See #63863. git-svn-id: https://develop.svn.wordpress.org/trunk@60743 602fd350-edb4-49c9-b593-d223f7449a82
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Trac ticket: Core-63863
Follow-up to: #9317, [60630]
See: (
#9825),#9830,#9498,#9826,#9827, #9798,#9828,#9829This patch introduces a new early-loaded
compat-utf8.phpfile which provides the UTF-8 validation fallback. This is part of a broader effort to unify and standardize UTF-8 handling.This is an intermediate change in order to better facilitate source-code tracking. This code was introduced in
formatting.phpand originally was intended to be duplicated inside ofwp_check_invalid_utf8(), but the difference between the two functions is so minor that the part of the code which scans through the bytes in a string should be abstracted and reused. This re-use function would ideally load early enough to be used to polyfill methods likemb_substr(), so I am moving this early.While this is not the abstracted code, this change will be helpful by providing continuity between the function as it stands today and as it will transform when reused by
wp_check_invalid_utf8(). In other words, this change exists to make sure that source control shows that this function moved first, and then was changed later. To move it and change it in one go is likely to sever its history.This will turn into
_wp_scan_utf8(), the updated iteration of the #6883 which provides a fast, spec-compliant, and streamable UTF-8 parser.