Skip to content

Upgrade Unicode to 18.0 - #136937

Merged
jkotas merged 10 commits into
dotnet:mainfrom
am11:chore/modernize-unicodegen
Nov 8, 2026
Merged

jkotas merged 10 commits into
dotnet:mainfrom
am11:chore/modernize-unicodegen

Conversation

@am11

@am11 am11 commented Nov 3, 2026 •

Copy link
Copy Markdown
Member

@dotnet-policy-service dotnet-policy-service Bot added the community-contribution Indicates that the PR has been added by a community member label Nov 3, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 4 pipeline(s).
11 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@am11
am11 requested review from agocke and tarekgh November 3, 2026 21:32
@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

@am11
am11 force-pushed the chore/modernize-unicodegen branch from de52f27 to 8dd081e Compare November 3, 2026 21:34
@am11 am11 changed the title Modernize UnicodeDataGenerator tool Upgrade Unicode to 18.0 Nov 3, 2026
Comment thread eng/Versions.props Resolved
@am11
am11 force-pushed the chore/modernize-unicodegen branch from 5e44a17 to a37cbd5 Compare November 7, 2026 16:26

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The seven new casing pairs lack active IgnoreCase tests in both directions across regex engines, and the exhaustive Unicode casing test is disabled.

1 open issue
What changed in this PR

Updates the runtime’s shared Unicode data dependency to 18.0 and refreshes encoding, regex casing, and native casing tables. Regex named blocks remain unchanged because there are no new BMP blocks.

Changes:

  • Centralizes the Unicode version in eng/Versions.props and updates the Unicode data package.
  • Refreshes generated tables and marks Unicode-generated files for GitHub’s diff display.
  • Repairs generator projects, converts the minipal generator to a file-based app, and updates maintenance instructions.
File Description
THIRD-PARTY-NOTICES.TXT Updates the Unicode copyright year.
src/​native/​minipal/​UnicodeDataGenerator/​UnicodeDataGenerator.csproj Removes the project replaced by the file-based app.
src/​native/​minipal/​UnicodeDataGenerator/​Program.cs Removes the former generator entry point.
src/​native/​minipal/​UnicodeDataGenerator.cs Adds the file-based native casing generator.
src/​native/​minipal/​unicodedata.c Adds Unicode casing mappings.
src/​libraries/​System.Text.RegularExpressions/​tools/​Readme.md Removes duplicated Unicode version tracking.
src/​libraries/​System.Text.RegularExpressions/​tools/​GenRegexNamedBlocks/​README.md Removes the hardcoded current-version statement.
src/​libraries/​System.Text.RegularExpressions/​tools/​GenerateRegexCasingTable.csproj Excludes the nested named-block generator sources.
src/​libraries/​System.Text.RegularExpressions/​tools/​DataTable.cs Fixes a generated comment’s formatting.
src/​libraries/​System.Text.RegularExpressions/​src/​System/​Text/​RegularExpressions/​Symbolic/​UnicodeCategoryRangesGenerator.cs Removes runtime-version information from generated comments.
src/​libraries/​System.Text.RegularExpressions/​src/​System/​Text/​RegularExpressions/​RegexCaseEquivalences.Data.cs Refreshes regex case-equivalence tables.
src/​libraries/​System.Text.Encodings.Web/​tools/​updating-encodings.md Documents centralized Unicode version selection.
src/​libraries/​System.Text.Encodings.Web/​tests/​System.Text.Encodings.Web.Tests.csproj Uses the shared Unicode version.
src/​libraries/​System.Text.Encodings.Web/​src/​System/​Text/​Unicode/​UnicodeHelpers.generated.cs Refreshes the defined-character bitmap.
src/​libraries/​System.Runtime/​tests/​System.Globalization.Tests/​System.Globalization.Tests.csproj Uses the shared Unicode version.
src/​libraries/​System.Runtime/​tests/​System.Globalization.Tests/​NlsTests/​System.Globalization.Nls.Tests.csproj Uses the shared version for NLS tests.
src/​libraries/​System.Runtime/​tests/​System.Globalization.Tests/​Hybrid/​System.Globalization.IOS.Tests.csproj Uses the shared version for iOS tests.
src/​libraries/​System.Runtime/​tests/​System.Globalization.Tests/​Hybrid/​AppleHybrid.Tests.csproj Uses the shared version for Apple hybrid tests.
src/​libraries/​System.Private.CoreLib/​Tools/​GenUnicodeProp/​Updating-Unicode-Versions.md Updates version-setting and native-generator instructions.
src/​libraries/​System.Private.CoreLib/​Tools/​GenUnicodeProp/​Readme.md Points to the centralized version property.
src/​libraries/​System.Private.CoreLib/​Tools/​GenUnicodeProp/​GenUnicodeProp.csproj Uses the current tool framework and shared Unicode version.
src/​libraries/​Common/​tests/​TestUtilities.Unicode/​TestUtilities.Unicode.csproj Uses the shared Unicode version for test resources.
eng/​Versions.props Defines Unicode 18.0 centrally.
eng/​Version.Details.xml Updates the Unicode package version and source commit.
eng/​Version.Details.props Updates the Unicode package version property.
.gitattributes Marks Unicode tables as generated files.

Review effort: Balanced

@tarekgh tarekgh left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks @am11!

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Invalid arguments produce a diagnostic but do not stop generation, causing exceptions rather than clean termination with a nonzero exit code.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

@jkotas

jkotas commented Nov 8, 2026

Copy link
Copy Markdown
Member

/ba-g intermittent github networking issues

@jkotas
jkotas merged commit 4b73d78 into dotnet:main Nov 8, 2026
149 of 156 checks passed
@am11
am11 deleted the chore/modernize-unicodegen branch November 8, 2026 15:48
@dotnet-milestone-bot dotnet-milestone-bot Bot added this to the 12.0-preview1 milestone Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-System.Globalization community-contribution Indicates that the PR has been added by a community member

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants