an update: I'll plan to re-run benchmark numbers locally, and get a better sense of the current performance characteristics

Here's the latest - a comparison between flutter at head and this PR, run locally on my machine:

current:

flutter analyze --flutter-repo

• 43.3s
• 41.9s
• 40.7s

flutter analyze (DartConf / conference_app)

• 6.0s
• 5.2s
• 5.3s

flutter analyze --flutter-repo switching branches (31156ae 4f31a3f 9e0a610)

• 42.3s
• 45.0s
• 41.4s

refactor PR:

flutter analyze --flutter-repo

• 73.3s
• 70.2s
• 70.3s

flutter analyze (DartConf)

• 7.4s
• 6.9s
• 6.9s

flutter analyze --flutter-repo modifying core files, w/o clearing the analysis cache

• 44.4s
• 7.9s
• 43.9s

flutter analyze --flutter-repo, few files changed, w/o clearing the analysis cache

• 10.1s
• 10.2s
• 10.1s

flutter analyze (DartConf / conference_app), few files changed, w/o clearing the analysis cache

• 1.9s
• 2.6s
• 2.4s

Overall:

• flutter analyze --flutter-repo, first analysis is 40.7s ==> 70.2s or 72% slower
• flutter analyze --flutter-repo, 2nd analysis is 40.7s ==> 10.1s or 4x faster
• flutter analyze on a user app, first analysis is 5.2s ==> 6.9s or 33% slower
• flutter analyze on a user app, 2nd analysis is 5.2s ==> 1.9s or 2.7x faster
• flutter analyze --flutter-repo, simulated branch switching, 41.4s ==> [7.9s to 43.9s], so from between the same speed as before to up to 5x faster

I think the net-net is that we'll regress the times for cold analysis for the flutter repo and for project (and what a CI would see), but substantially increase 2nd and later analysis times for the flutter repo and for user projects, and not regress analysis times for switching between branches when working on the flutter repo itself.

I'd like to re-land this, based on the original review from @tvolkert and @danrubel, and after a discussion w/ @Hixie about the performance regression for the flutter analyze --flutter-repo use case.

LGTM

the dartdocs benchmark went crazy right on this commit.

OK, thanks for the heads up. I'm OOO this week, but will take a look when I'm back.