-
Notifications
You must be signed in to change notification settings - Fork 29.7k
Description
Currently Flutter error messages are at the core large blobs of text with limited structure which has been often slows users down fixing errors as it is hard to distinguish the signal from the noise and so users get lost in long non interactive stack traces and distinguishing what parts of the error is the core problem that occurred, background information, or hints on how to fix the problem.
We would like to add structure to error messages making the errors practical to format beautifully on the client. The goal is to have enough structure that users can quickly scan the message to see hints for how to solve the problem, the core error message, links to interactive ui to fix the ui, and interactive links to open objects related to the error in the Inspector. Similarly, errors will start with the minimum useful information to show expanded with ui affordances to expand out to see additional information such as extra stack frames and additional parent widgets.
Basic examples of what we could do with added structure:
Note in this example, the widget names are clickable navigating the user to the matching location in the widget tree. Lists of ancestors and stack frames stack out truncated and only expand when a user clicks. Portions of the error that are hints are shown with a different background color and additional padding. The core error message is in red so it is harder to miss.

Example for an error on the rendering layer. In this case the RenderObject is similarly clickable to navigate over to the inspector to view.

Example of a ui mock of a more advanced error message that really lets the user understand a layout issue without switching contexts. Note the links to interactive UI elements to apply fixes, graphical visualization of the overflow failure that occurred, and header titles calling out core elements of the error. All of these ui features are speculative and may or may not prove out with user studies but they are features are are promising enough that we would like to iterate on them without churning through refactoring code in package:flutter for every UI iteration.

Prototype CL showing what it would take to add structure to all flutter errors at the cost of
more structured syntax writing error messages than writing unstructured blob of text.
#27202
If nothing else this CL provides a good overview of what blocks of content exist in the existing error messages.
Some observations: there are common tasks like embedding a structured object for a RenderObject that are called 20 times across the code base. Currently the same boilerplate code to emit strings describing a widget and its immediate children is repeated for each of these cases. If you look at FlutterErrorBuilder, WidgetErrorBuilder, and RenderErrorBuilder you will see the full list of all commonly occurring patterns of structured data that is included in errors (StackTrace, RenderObject, Widget, ancestor chain of widget) and error parts (e.g. hint, error, constraint).
The core concern about this prototype CL is that the builder syntax used is not productive enough for users to use and will reduce the number or quality of errors. A counter point is that it took about 2 days to refactor all errors in package:flutter to use the format. For someone who wants to write an error message particularly in the simpler case an alternate approach is to add named arguments provided on the default FlutterError constructor but unfortunately once you exceed the capabilities of that constructor you will have to start using builder syntax. Builder syntax is needed to handle multiple messages of the same type such as multiple hints. Looking at the facts on the ground there are multiple flutter errors with multiple separate hints.
Benefits are it is hard for users to do the wrong thing and there are builder classes that clarify what helper methods should be used depending on the context the error is being thrown in (base, Render, or Widget).
The structure of the error messages is the same DiagnosticsNode format used by the WidgetInspector making it reasonable to include errors directly in the widget tree as properties of a widget indicating where the problem occurred.
Risks:
The builder syntax may be unwieldy and demotivate framework authors and users from writing good structured errors.
Alternate design:
Markdown lite for Flutter errors.
Add magic characters in the error templates that indicate error parts (e.g. hint, error, contract).
Risks:
Markdown like templates may be fragile and error prone as users will get extremely limited compile time checking that they are using the scheme in contrast to the other design where you will get an immediate compile time error if you attempt to pass the wrong type of object to a builder method.
Similarly, opening and closing blocks in this case will be done with some level of tags embedded as magic unicode characters so mismatched tags will cause the usual problems. This can be mitigated with careful test coverage although the current code base shows that there is limited coverage of errors.
Breaking change:
This change will modify the toString for all objects implementing Diagnosticable. As Diagnosticable objects have great toString functions, there tend to be a lot of tests that use them to easily write golden tests. Currently modifying the toString for Diagnosticable breaks 19 tests in package:flutter and would break third party tests as well. Modifying the equalsIgnoreHashCodes matcher to ignore the temporary object ids embedded in the toString as well would help somewhat but still leads to 16 tests failing. This is because not all tests using the equalsIgnoringHashCodes matcher as it wouldn't make sense for them to as the golden output did not contain hash codes and because adding the magic unicode character impacts line breaking for multiple diagnostics output so even though equalsIgnoreHashCodes is used. Fixing the line breaking helper used reduces the number of test failures in package:flutter to 13. The failures would be easy enough to fix by switching to a matcher that ignores the magic characters but users would need to know such a matcher exists and use it. Otherwise the tests will become mysteriously flaking as the object ids in the matchers would change unpredictably.
This breaks anyone who linebreaks debug output and doesn't know about the magic characters. Lines will be broken incorrectly resulting in shorter lines than otherwise. The common debugWrap helper can be updated to reflect the magic characters to mitigate this risk. With debugWrap modified to ignore magic characters the # of failing tests in package:flutter is reduced to
People with golden tests should expect to have to rebaseline them periodically so perhaps this isn't that big a deal.
Alternatives to minimize the breaking:
Make the magic unicode characters be off by default in tests. This will mean that debuggers depending on them will be broken for tests and tests testing debug output are testing the wrong thing but would minimize the breakage.
Example failing test:
expect(description, <String>[
'backgroundColor: Color(0xff123456)',
'elevation: 8.0',
'titleTextStyle: TextStyle(inherit: true, color: Color(0xffffffff))',
'contentTextStyle: TextStyle(inherit: true, color: Color(0xff000000))',
]);
now the toString for each TextStyle contains at least one marker character.
Fixing debugWrap to account for the magic characters still leaves 13 failures.
Unsolved challenges:
Object lifecycles using this scheme are fragile as Dart string templating is quite limited so we have no efficient way of distinguishing a toString call part of a template that will be used for an error from a time someone is just calling toString arbitrarily on an object. This suggests the only feasible approach to avoid memory leaks is to have only a ring buffer of object references.
This can work alright as long as care is taken to quickly deserialize the object references out of the format when capturing them for display at a later point either in a logging view or in an interactive view on the device or in the inspector such as a FlutterError widget.