Repository navigation
[flutter_tools] Compute LSP Content-Length header using UTF-8 byte length - #191377
Conversation
…ngth Fixes an issue where `flutter analyze` crashes with a `FormatException: Unexpected end of input` when analyzing projects whose path contains non-ASCII characters. In `AnalysisServer._writeMessage`, the `Content-Length` header was being calculated using `message.length` (UTF-16 code units) rather than the UTF-8 encoded byte length required by the LSP specification. When messages contained multibyte Unicode characters (such as in `rootUri` or `workspaceFolders`), the header under-reported the payload length, causing the analysis server's JSON parser to receive a truncated payload and exit prematurely. Also updates `MockLspServerProcess` to parse stdin as a stream of raw bytes and measure outgoing message length in bytes to faithfully simulate the LSP transport layer. Fixes flutter#191309
github-actions
Bot
added
the
tool
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the AnalysisServer and its mock implementation to calculate and handle LSP message lengths in bytes rather than string characters, resolving issues with non-ASCII characters in project paths. A new test case is also added to verify this behavior. The review feedback suggests optimizing the message writing in both production and mock code to avoid encoding the message string multiple times by writing the pre-encoded bytes directly.
| final List<int> encodedMessage = utf8.encode(message); | ||
| _process?.stdin.write('Content-Length: ${encodedMessage.length}\r\n\r\n$message'); |
There was a problem hiding this comment.
To avoid encoding the potentially large message string twice (once to calculate the byte length with utf8.encode, and a second time implicitly when writing the string to _process?.stdin), you can write the header and the pre-encoded message bytes directly to stdin using add.
final List<int> encodedMessage = utf8.encode(message);
_process?.stdin.add(utf8.encode('Content-Length: ${encodedMessage.length}\r\n\r\n'));
_process?.stdin.add(encodedMessage);| final List<int> bytes = utf8.encode(message); | ||
| _stdoutController.add(utf8.encode('Content-Length: ${bytes.length}\r\n\r\n$message')); |
There was a problem hiding this comment.
Similar to the production code, we can avoid encoding the message string twice here by combining the encoded header and the pre-encoded message bytes before adding them to the stream controller.
| final List<int> bytes = utf8.encode(message); | |
| _stdoutController.add(utf8.encode('Content-Length: ${bytes.length}\r\n\r\n$message')); | |
| final List<int> bytes = utf8.encode(message); | |
| _stdoutController.add(<int>[ | |
| ...utf8.encode('Content-Length: ${bytes.length}\r\n\r\n'), | |
| ...bytes, | |
| ]); |
DanTup
left a comment
DanTup
left a comment
There was a problem hiding this comment.
lgtm!
It'd be nice if we could reuse the classes from the server here (probably by moving them into a publishable package - which we've previously discussed doing to share the generated types) to avoid having multiple copies that might be slightly different.
I agree! It'd be great to have a shared implementation across our tooling. |
flutter-dashboard
Bot
removed
the
autosubmit
…utter analyze fails flutter analyze of Flutter 3.44 fails in a directory whose path has letters beyond ASCII, on every system: it sends the analysis server a message whose length it counts in UTF-16 code units rather than in bytes (fixed in flutter/flutter#191377). In such a directory the matrix runs dart analyze --fatal-infos, which reports the same issues, and says so in its log.
…ASCII too Flutter 3.47.5, the stable one now, still lacks the fix of flutter/flutter#191377, so the matrix analyzes an app in a directory with letters beyond ASCII with dart analyze on it as well. The comments of CI and of the matrix, AGENTS.md and CONTRIBUTING.md name both versions.
Description
Fixes
flutter analyzecrashing withFormatException: Unexpected end of inputwhen analyzing a project whose path contains non-ASCII characters (e.g.Alexander’s MacBook Prowith right single quotation mark’/ U+2019).In
AnalysisServer._writeMessage, theContent-Lengthheader was being calculated usingmessage.length(UTF-16 code units) rather than the UTF-8 encoded byte length required by the LSP specification. When messages contained multibyte Unicode characters (such as inrootUriorworkspaceFolders), the header under-reported the payload length, causing the analysis server's JSON parser to receive a truncated payload and exit prematurely with code 255.Also updates
MockLspServerProcessto parse stdin as a stream of raw bytes and compute outgoing message length in bytes to faithfully simulate the LSP transport layer.Related Issues
Fixes #191309
Tests
AnalysisServer handles non-ASCII project pathunit test inpackages/flutter_tools/test/commands.shard/hermetic/analyze_continuously_test.dart.Pre-launch Checklist
///).