Skip to content

[Tool] Tolerate malformed UTF-8 in process streaming decoders#188453

Merged
auto-submit[bot] merged 3 commits into
flutter:masterfrom
bkonyi:fix-utf8-stream-decoding
Jun 29, 2026
Merged

[Tool] Tolerate malformed UTF-8 in process streaming decoders#188453
auto-submit[bot] merged 3 commits into
flutter:masterfrom
bkonyi:fix-utf8-stream-decoding

Conversation

@bkonyi

@bkonyi bkonyi commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

The Flutter tool crashes with a FormatException: Missing extension byte when helper processes output non-UTF-8 bytes (e.g., on Windows/WSL under certain non-UTF-8 system locales).

This change updates the process streaming mechanism to use the fault-tolerant utf8AllowMalformedLineDecoder instead of the strict utf8LineDecoder. This replaces invalid bytes with the Unicode replacement character (U+FFFD) instead of throwing a fatal FormatException, preventing crashes while preserving log output.

Fixes #188448

The Flutter tool crashes with a `FormatException: Missing extension byte` when helper processes output non-UTF-8 bytes (e.g., on Windows/WSL under certain non-UTF-8 system locales).

This change updates the process streaming mechanism to use the fault-tolerant `utf8AllowMalformedLineDecoder` instead of the strict `utf8LineDecoder`. This replaces invalid bytes with the Unicode replacement character (`U+FFFD`) instead of throwing a fatal `FormatException`, preventing crashes while preserving log output.

Fixes flutter#188448
@flutter-dashboard flutter-dashboard Bot added the CICD Run CI/CD label Jun 23, 2026
@github-actions github-actions Bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Jun 23, 2026
@bkonyi
bkonyi changed the base branch from main to master June 23, 2026 21:50
@bkonyi bkonyi added CICD Run CI/CD and removed CICD Run CI/CD labels Jun 23, 2026
@fluttergithubbot

Copy link
Copy Markdown
Contributor

An existing Git SHA, 33abeb06bc11c3e182360a499b65b9e39985e404, was detected, and no actions were taken.

To re-trigger presubmits after closing or re-opeing a PR, or pushing a HEAD commit (i.e. with --force) that already was pushed before, push a blank commit (git commit --allow-empty -m "Trigger Build") or rebase to continue.

@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 23, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 23, 2026
@bkonyi
bkonyi requested a review from chingjun June 25, 2026 20:50
@bkonyi
bkonyi marked this pull request as ready for review June 25, 2026 20:50

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the process utility to use utf8AllowMalformedLineDecoder instead of utf8LineDecoder for decoding stdout and stderr streams, allowing the command output to tolerate malformed UTF-8 bytes. Accompanying tests are added to verify this behavior. The review feedback suggests improving the readability of the new test descriptions by replacing confusing suffixes like (Worker A) and (Worker B) with more descriptive names that explain the specific scenarios being tested.

Comment thread packages/flutter_tools/test/general.shard/base/process_test.dart Outdated
Comment thread packages/flutter_tools/test/general.shard/base/process_test.dart Outdated
- Replace confusing `(Worker A)` and `(Worker B)` suffixes with descriptive test names explaining the scenarios.
@github-actions github-actions Bot removed the CICD Run CI/CD label Jun 25, 2026
@bkonyi bkonyi added the CICD Run CI/CD label Jun 25, 2026
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request Jul 7, 2026
…#12081)

Manual roll Flutter from 0c80830e465b to ca9f874f5284 (119 revisions)

Manual roll requested by [email protected]

flutter/flutter@0c80830...ca9f874

2026-06-30 [email protected] [tool] Don't require a Flutter compile task when staging jniLibs (flutter/flutter#188805)
2026-06-30 [email protected] [Impeller] Fix potential overflow when allocating buffers at the next power of two size (flutter/flutter#188742)
2026-06-30 [email protected] [Framework/Tool] Decouple preview theme imports (flutter/flutter#188176)
2026-06-30 [email protected] Add import of `dart:_js_interop_wasm` in sdk rewriter tool (flutter/flutter#188620)
2026-06-30 [email protected] Detach LLDB and print stack trace on process stop (flutter/flutter#188576)
2026-06-30 [email protected] [Linux] Fix FlCompositorOpenGL.pixels comment (flutter/flutter#188754)
2026-06-30 [email protected] [flutter_tools] Fix crash in flutter create when pubspec.yaml is empty (flutter/flutter#188385)
2026-06-30 [email protected] Roll Packages from 656ccaa to 274ed3e (23 revisions) (flutter/flutter#188792)
2026-06-30 [email protected] In AndroidImageGenerator, check that the destination pixel buffer has sufficient capacity for the decoded data (flutter/flutter#188752)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188773)
2026-06-30 [email protected] [Impeller] Add a flat VertexAttributeFormat for vertex inputs (flutter/flutter#188684)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188764)
2026-06-30 [email protected] Roll Dart SDK from 0cb483880b6b to e1bdb9ce3327 (2 revisions) (flutter/flutter#188763)
2026-06-29 [email protected] Handle 'no permissions' adb device state (flutter/flutter#187248)
2026-06-29 [email protected] [windows]: adjusts uniform buffers to hit hlsl optimization (flutter/flutter#188538)
2026-06-29 [email protected] Roll Skia from bfb7860cb9c7 to 71947c4110b0 (9 revisions) (flutter/flutter#188747)
2026-06-29 [email protected] ci: extract wait-for-engine-build logic into a reusable composite action (flutter/flutter#188748)
2026-06-29 [email protected] Provide guided migration logs when iOS app crashes on simulator (flutter/flutter#188736)
2026-06-29 [email protected] Revert "[flutter_tools] Track asset transformer dependencies for hot reload" (flutter/flutter#188751)
2026-06-29 [email protected] Migrate ABI splits to new AGP dsl (flutter/flutter#188369)
2026-06-29 [email protected] Add Impeller+OpenGLES startup benchmark for mokey (flutter/flutter#188495)
2026-06-29 [email protected] Increase macOS minimum supported version from 10.15 to 12 to support Xcode 27  (flutter/flutter#188520)
2026-06-29 [email protected] Moves test ownership validation to flutter/flutter (flutter/flutter#188655)
2026-06-29 [email protected] Add --flavor support for Windows desktop builds (flutter/flutter#187034)
2026-06-29 [email protected] Android_hardware_smoke_test: Enable pixel exact local file comparator to read goldens from flutter asset URI (flutter/flutter#188587)
2026-06-29 [email protected] [flutter_tools] Use DeviceHub.app for iOS simulator path on Xcode 27+ (flutter/flutter#187910)
2026-06-29 [email protected] Roll Skia from 111e7582d081 to bfb7860cb9c7 (2 revisions) (flutter/flutter#188731)
2026-06-29 [email protected] Use `revert` label instead of `revert_wf` (flutter/flutter#188639)
2026-06-29 [email protected] Properly await Dart Development Service shutdown with timeout (flutter/flutter#188387)
2026-06-29 [email protected] [flutter_tools] Track asset transformer dependencies for hot reload (flutter/flutter#187947)
2026-06-29 [email protected] [Tool] Tolerate malformed UTF-8 in process streaming decoders (flutter/flutter#188453)
2026-06-29 [email protected] [flutter_tools] Use new ddc modules in test (flutter/flutter#188240)
2026-06-29 [email protected] Roll Packages from c1f7d92 to 656ccaa (12 revisions) (flutter/flutter#188728)
2026-06-29 [email protected] Remove unused fields (flutter/flutter#188705)
2026-06-29 [email protected] Clear text input handler widget on view dispose (flutter/flutter#188701)
2026-06-29 [email protected] Free compositor in view renderer finalize to avoid use-after-free (flutter/flutter#188702)
2026-06-29 [email protected] [linux] Use GWeakRef in mock signal handler test helper (flutter/flutter#188700)
2026-06-29 [email protected] Fixing few related editing issues with LTR/RTL text (flutter/flutter#188503)
2026-06-29 [email protected] [Flutter GPU] Add Texture.fromImage to wrap a ui.Image texture (flutter/flutter#188605)
2026-06-29 [email protected] [Flutter GPU] Honor the enable argument in RenderPass.setDepthWriteEnable (flutter/flutter#188715)
2026-06-29 [email protected] Roll Skia from ba1942d8c3e1 to 111e7582d081 (1 revision) (flutter/flutter#188721)
2026-06-29 [email protected] Roll Skia from 587d8befe1ee to ba1942d8c3e1 (6 revisions) (flutter/flutter#188717)
2026-06-29 [email protected] Remove some refs to package:intl (flutter/flutter#188504)
2026-06-29 [email protected] [VPAT] Update a11y assessment app FAB example to announce value change when it's updated. (flutter/flutter#188466)
...
kalyujniy pushed a commit to brickit-app/camera that referenced this pull request Jul 8, 2026
…flutter#12081)

Manual roll Flutter from 0c80830e465b to ca9f874f5284 (119 revisions)

Manual roll requested by [email protected]

flutter/flutter@0c80830...ca9f874

2026-06-30 [email protected] [tool] Don't require a Flutter compile task when staging jniLibs (flutter/flutter#188805)
2026-06-30 [email protected] [Impeller] Fix potential overflow when allocating buffers at the next power of two size (flutter/flutter#188742)
2026-06-30 [email protected] [Framework/Tool] Decouple preview theme imports (flutter/flutter#188176)
2026-06-30 [email protected] Add import of `dart:_js_interop_wasm` in sdk rewriter tool (flutter/flutter#188620)
2026-06-30 [email protected] Detach LLDB and print stack trace on process stop (flutter/flutter#188576)
2026-06-30 [email protected] [Linux] Fix FlCompositorOpenGL.pixels comment (flutter/flutter#188754)
2026-06-30 [email protected] [flutter_tools] Fix crash in flutter create when pubspec.yaml is empty (flutter/flutter#188385)
2026-06-30 [email protected] Roll Packages from 656ccaa to 274ed3e (23 revisions) (flutter/flutter#188792)
2026-06-30 [email protected] In AndroidImageGenerator, check that the destination pixel buffer has sufficient capacity for the decoded data (flutter/flutter#188752)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188773)
2026-06-30 [email protected] [Impeller] Add a flat VertexAttributeFormat for vertex inputs (flutter/flutter#188684)
2026-06-30 [email protected] Roll pub packages (flutter/flutter#188764)
2026-06-30 [email protected] Roll Dart SDK from 0cb483880b6b to e1bdb9ce3327 (2 revisions) (flutter/flutter#188763)
2026-06-29 [email protected] Handle 'no permissions' adb device state (flutter/flutter#187248)
2026-06-29 [email protected] [windows]: adjusts uniform buffers to hit hlsl optimization (flutter/flutter#188538)
2026-06-29 [email protected] Roll Skia from bfb7860cb9c7 to 71947c4110b0 (9 revisions) (flutter/flutter#188747)
2026-06-29 [email protected] ci: extract wait-for-engine-build logic into a reusable composite action (flutter/flutter#188748)
2026-06-29 [email protected] Provide guided migration logs when iOS app crashes on simulator (flutter/flutter#188736)
2026-06-29 [email protected] Revert "[flutter_tools] Track asset transformer dependencies for hot reload" (flutter/flutter#188751)
2026-06-29 [email protected] Migrate ABI splits to new AGP dsl (flutter/flutter#188369)
2026-06-29 [email protected] Add Impeller+OpenGLES startup benchmark for mokey (flutter/flutter#188495)
2026-06-29 [email protected] Increase macOS minimum supported version from 10.15 to 12 to support Xcode 27  (flutter/flutter#188520)
2026-06-29 [email protected] Moves test ownership validation to flutter/flutter (flutter/flutter#188655)
2026-06-29 [email protected] Add --flavor support for Windows desktop builds (flutter/flutter#187034)
2026-06-29 [email protected] Android_hardware_smoke_test: Enable pixel exact local file comparator to read goldens from flutter asset URI (flutter/flutter#188587)
2026-06-29 [email protected] [flutter_tools] Use DeviceHub.app for iOS simulator path on Xcode 27+ (flutter/flutter#187910)
2026-06-29 [email protected] Roll Skia from 111e7582d081 to bfb7860cb9c7 (2 revisions) (flutter/flutter#188731)
2026-06-29 [email protected] Use `revert` label instead of `revert_wf` (flutter/flutter#188639)
2026-06-29 [email protected] Properly await Dart Development Service shutdown with timeout (flutter/flutter#188387)
2026-06-29 [email protected] [flutter_tools] Track asset transformer dependencies for hot reload (flutter/flutter#187947)
2026-06-29 [email protected] [Tool] Tolerate malformed UTF-8 in process streaming decoders (flutter/flutter#188453)
2026-06-29 [email protected] [flutter_tools] Use new ddc modules in test (flutter/flutter#188240)
2026-06-29 [email protected] Roll Packages from c1f7d92 to 656ccaa (12 revisions) (flutter/flutter#188728)
2026-06-29 [email protected] Remove unused fields (flutter/flutter#188705)
2026-06-29 [email protected] Clear text input handler widget on view dispose (flutter/flutter#188701)
2026-06-29 [email protected] Free compositor in view renderer finalize to avoid use-after-free (flutter/flutter#188702)
2026-06-29 [email protected] [linux] Use GWeakRef in mock signal handler test helper (flutter/flutter#188700)
2026-06-29 [email protected] Fixing few related editing issues with LTR/RTL text (flutter/flutter#188503)
2026-06-29 [email protected] [Flutter GPU] Add Texture.fromImage to wrap a ui.Image texture (flutter/flutter#188605)
2026-06-29 [email protected] [Flutter GPU] Honor the enable argument in RenderPass.setDepthWriteEnable (flutter/flutter#188715)
2026-06-29 [email protected] Roll Skia from ba1942d8c3e1 to 111e7582d081 (1 revision) (flutter/flutter#188721)
2026-06-29 [email protected] Roll Skia from 587d8befe1ee to ba1942d8c3e1 (6 revisions) (flutter/flutter#188717)
2026-06-29 [email protected] Remove some refs to package:intl (flutter/flutter#188504)
2026-06-29 [email protected] [VPAT] Update a11y assessment app FAB example to announce value change when it's updated. (flutter/flutter#188466)
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Tool] FormatException in UTF-8 Process Output Decoding

3 participants