Skip to content

Migrate flutter CLI tool to null safety #71511

Closed

Description

@Hixie

We need to migrate the flutter tool's codebase to null-safe Dart.

This will be a long-term effort and is blocked on issues around dependencies that haven't migrated and figuring out a solution to our reliance on mockito.


Migrating the Flutter Tool to null safety

jonahwilliams@

Background

The flutter command line tool is an approximately 125,000 line Dart application (with an additional 90,000 lines of test code) that powers the Flutter development/build cycle, as well as the underlying support used by IDE plugins and google3. While migration to null safety of this tool does not block user applications from migrating to null safety, in the interest of moving the ecosystem forward, making it easier to contribute to the tool, and reducing common errors caused by NPEs, the tool must eventually migrate to null safe Dart.

Overview

Migration Process

Prerequisites

  • All dependencies are migrated to null safety (In Progress). See Appendix for a list of all unmigrated deps.
  • Plan for migration to null safety is drafted (In Progress)

Process

  • Flutter tool SDK constraint is set to 2.12 and all files are opted into 2.8 ([flutter_tools] opt all flutter tool libraries and tests out of null safety. #74832)
  • Begin removing mockito usage and enforce ban on additional mockito tests
  • Begin migrating individual libraries with no dependencies on unmigrated code (?)
  • Application entrypoint is migrated
    • Blocked on migration of all dependencies
  • Test code without mockito dependencies is migrated
  • Test code with mockito dependencies is migrated.

Challenges

The Flutter tool code base is large enough that a one-shot dart migrate PR will be essentially un-reviewable. As of the writing of this document, dart migrate does support partial migrations, which would allow the tool to more gradually opt in. While the Flutter tool is layered, it is not as structured as the framework - the majority of the libraries are in lib/src and there are significant circular imports between these files.

The Flutter tool unit tests make extensive use of mockito 4.X, which has required breaking changes in 5.X (that introduce codegen) for null safe Dart. These tests would have to be substantially rewritten, or additional dependencies introduced to support mockito 5.X

Work Estimates

Note: this excludes the cost/time for migration of external dependencies

The Flutter framework code is as of writing 366,880 lines of Dart. The first PR to migrate foundation to null safety (#61188 ) was opened on Jul 15, 2020. The last PR to migrate material (#66633) landed on Sept 28, 2020. This was approximately 73 working days, with the work primarily performed by the contributor a14n and reviewed by multiple members of the Flutter team. With a bit of napkin math, assuming a similar pace of work at ~5,000 lines a day on average, that would take the tool 25 working days to complete. This does not account for significant pre-work that may have been done by a14n that I have not counted, nor can it account for substantial differences in the type of codebase between the Flutter framework and CLI tool (namely the lack of layering in the latter).

The difficulty of the test migrating is harder to estimate. As of writing there are 365 instances of "extends Mock" in the Flutter tools test shard. In order to migrate the Flutter framework tests to null safety, a handful of tests that made limited use of mockito were re-written. Based on my experience there, I would estimate that it could take a few hours for trivial usage, and multiple days for substantial rewrite. With some more napkin math, and based on 130 tests that use mockito, that could take between 30 - 130 working days to fully rewrite.

Unless more individuals contribute or more time is dedicated to the migration, it will take an extremely long time. For example, If I were to devote my ~8 hours a week of heads down time alone to migration, this could take until August, 2023!

Priorities and Planning

The Flutter tools team does not currently have many spare cycles to dedicate to migration (See estimate above). At best, we could devote a % of "code health" time to it, and keep track of overall progress to estimate a final completion date. Based on my experience trying to remove most Zone injected dynamic values (similar widespread cleanup effort, less analysis assistance), this is likely to take an indeterminate amount of time - as other high priority issues will continually interrupt progress. This makes approaches such as a manual migration or migration off of mockito tests infeasible in the short term (2021).

In order to make the process easier, the tools team will begin enforcing a total ban on additional mockito usage (i.e. new imports, mocks, extends) on a given date.

Migration Date Start

The Flutter tool could begin either partially migrating to null safety and/or removing mockito dependencies immediately. Many of the analyzer dependencies, for example, are transitive through the test_core dependency which is only exposed through the flutter_test command. This may allow libraries in base/ or /android (for example) to migrate ahead of that.

Similarly, removal of mockito usage can begin immediately - but unless we can proactively ban further usage of it in tests, we risk backsliding significantly.

The date where all packages are migrated to null-safety isn't yet clear. The largest individual dependencies are analyzer (transitive through test), dwds, and devtools (transitive through dwds). At the time of writing, none of these packages have been migrated yet.

Appendix

For more information on the breaking changes in mockito, see https://github.com/dart-lang/mockito/blob/master/NULL_SAFETY_README.md

For an up to date listing of the currently migrated tool dependencies, the best source is the pubspec.yaml itself. (There are too many to otherwise count).


Outstanding non-null-safe Tool Files

NOTE: re-generate this list with grep --recursive --files-with-matches 'dart = 2\.8' flutter_tools | grep -v '\.dart_tool' | sort | awk '{print " - "$1}' | pbcopy:

  • flutter_tools/bin/flutter_tools.dart
  • flutter_tools/bin/fuchsia_asset_builder.dart
  • flutter_tools/bin/fuchsia_tester.dart
  • flutter_tools/test/commands.shard/hermetic/analyze_continuously_test.dart
  • flutter_tools/test/commands.shard/hermetic/analyze_suggestion_test.dart
  • flutter_tools/test/commands.shard/hermetic/analyze_test.dart
  • flutter_tools/test/commands.shard/hermetic/assemble_test.dart
  • flutter_tools/test/commands.shard/hermetic/attach_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_darwin_framework_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_fuchsia_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_ios_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_ipa_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_linux_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_macos_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_web_test.dart
  • flutter_tools/test/commands.shard/hermetic/build_windows_test.dart
  • flutter_tools/test/commands.shard/hermetic/clean_test.dart
  • flutter_tools/test/commands.shard/hermetic/config_test.dart
  • flutter_tools/test/commands.shard/hermetic/create_usage_test.dart
  • flutter_tools/test/commands.shard/hermetic/custom_devices_test.dart
  • flutter_tools/test/commands.shard/hermetic/daemon_test.dart
  • flutter_tools/test/commands.shard/hermetic/devices_test.dart
  • flutter_tools/test/commands.shard/hermetic/doctor_test.dart
  • flutter_tools/test/commands.shard/hermetic/downgrade_test.dart
  • flutter_tools/test/commands.shard/hermetic/drive_test.dart
  • flutter_tools/test/commands.shard/hermetic/generate_localizations_test.dart
  • flutter_tools/test/commands.shard/hermetic/ide_config_test.dart
  • flutter_tools/test/commands.shard/hermetic/install_test.dart
  • flutter_tools/test/commands.shard/hermetic/logs_test.dart
  • flutter_tools/test/commands.shard/hermetic/precache_test.dart
  • flutter_tools/test/commands.shard/hermetic/proxied_devices_test.dart
  • flutter_tools/test/commands.shard/hermetic/pub_get_test.dart
  • flutter_tools/test/commands.shard/hermetic/run_test.dart
  • flutter_tools/test/commands.shard/hermetic/screenshot_command_test.dart
  • flutter_tools/test/commands.shard/hermetic/shell_completion_test.dart
  • flutter_tools/test/commands.shard/hermetic/symbolize_test.dart
  • flutter_tools/test/commands.shard/hermetic/test_test.dart
  • flutter_tools/test/commands.shard/hermetic/update_packages_test.dart
  • flutter_tools/test/commands.shard/hermetic/upgrade_test.dart
  • flutter_tools/test/commands.shard/permeable/build_aar_test.dart
  • flutter_tools/test/commands.shard/permeable/build_apk_test.dart
  • flutter_tools/test/commands.shard/permeable/build_appbundle_test.dart
  • flutter_tools/test/commands.shard/permeable/build_bundle_test.dart
  • flutter_tools/test/commands.shard/permeable/create_test.dart
  • flutter_tools/test/commands.shard/permeable/devices_test.dart
  • flutter_tools/test/commands.shard/permeable/format_test.dart
  • flutter_tools/test/commands.shard/permeable/migrate_abandon_test.dart
  • flutter_tools/test/commands.shard/permeable/migrate_apply_test.dart
  • flutter_tools/test/commands.shard/permeable/migrate_status_test.dart
  • flutter_tools/test/commands.shard/permeable/packages_test.dart
  • flutter_tools/test/commands.shard/permeable/upgrade_test.dart
  • flutter_tools/test/general.shard/android/gradle_errors_test.dart
  • flutter_tools/test/general.shard/application_package_test.dart
  • flutter_tools/test/general.shard/args_test.dart
  • flutter_tools/test/general.shard/base/logger_test.dart
  • flutter_tools/test/general.shard/commands/build_test.dart
  • flutter_tools/test/general.shard/dart_plugin_test.dart
  • flutter_tools/test/general.shard/drive/drive_service_test.dart
  • flutter_tools/test/general.shard/drive/web_driver_service_test.dart
  • flutter_tools/test/general.shard/emulator_test.dart
  • flutter_tools/test/general.shard/fuchsia/fuchsia_device_start_test.dart
  • flutter_tools/test/general.shard/fuchsia/fuchsia_device_test.dart
  • flutter_tools/test/general.shard/hot_test.dart
  • flutter_tools/test/general.shard/ios/ios_device_start_nonprebuilt_test.dart
  • flutter_tools/test/general.shard/project_test.dart
  • flutter_tools/test/general.shard/resident_runner_test.dart
  • flutter_tools/test/general.shard/resident_web_runner_cold_test.dart
  • flutter_tools/test/general.shard/resident_web_runner_test.dart
  • flutter_tools/test/general.shard/runner/flutter_command_runner_test.dart
  • flutter_tools/test/general.shard/runner/flutter_command_test.dart
  • flutter_tools/test/general.shard/runner/runner_test.dart
  • flutter_tools/test/general.shard/tester/flutter_tester_test.dart
  • flutter_tools/test/general.shard/vmservice_test.dart
  • flutter_tools/test/general.shard/web/devfs_web_test.dart
  • flutter_tools/test/general.shard/web/golden_comparator_process_test.dart
  • flutter_tools/test/general.shard/web/golden_comparator_test.dart
  • flutter_tools/test/general.shard/web/migrations/scrub_generated_plugin_registrant_test.dart
  • flutter_tools/test/general.shard/web/web_asset_server_test.dart
  • flutter_tools/test/general.shard/web/web_expression_compiler_test.dart
  • flutter_tools/test/integration.shard/test_data/basic_project.dart
  • flutter_tools/test/web.shard/sdk_web_configuration_test.dart

(last updated July 11, 2022 82 outstanding unmigrated libraries)

Activity

  1. added
    c: contributor-productivityTeam-specific productivity, code health, technical debt.
    toolAffects the "flutter" command-line tool. See also t: labels.
    a: null-safetySupport for Dart's null safety feature
    P3Issues that are less important to the Flutter project
    c: tech-debtTechnical debt, code quality, testing, etc.
    on Dec 1, 2020
  2. Hixie commented on Dec 1, 2020

    @Hixie
    ContributorAuthor

    Our dependencies analyzer and vmservice likely won't be migrated until mid to late next year, for what it's worth; one option is to migrate out of order if we decide we don't want to wait.

  3. jmagman commented on Jun 11, 2021

    @jmagman
    Member

    Update: the flutter_tools package was opted out of null safety with #74832. Since then, 52% of lib/src dart libraries have been migrated. 27% of test libraries have been migrated. Almost all of the rest is stuck behind dwds dart-lang/webdev#1327 and any other null unsafe dependencies that prevent upgrading dds to a null safe version.

    When we started, there were 365 mockito mock objects in 130 flutter_tools test libraries. There are now 10 mock objects left in 3 libraries.

  4. jonahwilliams commented on Jun 23, 2021

    @jonahwilliams
    Contributor

    As of #85076, mockito will have been removed and we only have null-safe updates in dependencies remaining

  5. jonahwilliams commented on Jun 21, 2022

    @jonahwilliams
    Contributor

    So the new status of this: We've migrated about all the code we can, but are now hard blocked migrating more on dwds null safety

  6. mit-mit commented on Aug 25, 2022

    @mit-mit
    Member

    package:dwds has been migrated!

    Anything else this is blocked on?

  7. jonahwilliams commented on Aug 26, 2022

    @jonahwilliams
    Contributor

    @mit-mit I just checked but it looks like null safe dwds has not been published yet

  8. mit-mit commented on Aug 26, 2022

    @mit-mit
    Member

    @annagrin can you help with that?

  9. annagrin commented on Aug 29, 2022

    @annagrin
    Contributor

    @elliette do you think we are ready to publish the null safe dwds?

  10. annagrin commented on Aug 31, 2022

    @annagrin
    Contributor

    See discussion and progress of dwds migration and publishing in dart-lang/webdev#1327

  11. mit-mit commented on Sep 12, 2022

    @mit-mit
    Member

    @jonahwilliams can you confirm that you are unblocked now (dwds was published about a week ago)?

  12. jonahwilliams commented on Sep 12, 2022

    @jonahwilliams
    Contributor

    Yes we're unblocked now, the last PR is #110706

  13. christopherfujino commented on Sep 12, 2022

    @christopherfujino
    Contributor

    Done as of #110706

  14. mit-mit commented on Sep 13, 2022

    @mit-mit
    Member

    Woohoo, thanks!

  15. github-actions commented on Sep 27, 2022

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  16. locked as resolved and limited conversation to collaborators on Sep 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Issues that are less important to the Flutter projecta: null-safetySupport for Dart's null safety featurec: contributor-productivityTeam-specific productivity, code health, technical debt.c: tech-debtTechnical debt, code quality, testing, etc.toolAffects the "flutter" command-line tool. See also t: labels.

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions