Skip to content

Conversation

@auto-submit
Copy link
Contributor

@auto-submit auto-submit bot commented Oct 24, 2023

Reverts #136851
Initiated by: CaseyHillers
This change reverts the following previous change:
Original Description:
One of the reasons gathering coverage information is expensive is that we have to force compile every function in the libraries we're interested in. Without this, functions that haven't been invoked (so haven't been compiled) won't have any line number information, so the coverage tool doesn't know which lines to add to the list of misses. In flutter's case, the test infra spawns many VMs, and each of these needs to recompile all those libraries.

To fix this, we need a way of skipping force compilation for libraries we've already seen in previous tests, without losing the information about which lines in each library are coverable. So I added the coverableLineCache to coverage.collect in package:coverage v1.7.0. This cache starts out empty, but fills up with lists of all the lines that are coverable for every library as coverage is gathered. package:coverage can then tell the VM not to force compile any libraries in this cache (using getSourceReport's librariesAlreadyCompiled param). So the first test suite will still have to compile everything, but subsequent test suites will be much faster.

This speeds up coverage collection significantly, for large test suites:

Running flutter/packages/flutter tests... Time Overhead
without coverage 8:53 -
with coverage 20:25 130%
with coverableLineCache 12:21 40%

Bug: #100751

@auto-submit auto-submit bot added the revert of Bot Only: Tracking label for bot. Tracks new revert of pull requests. label Oct 24, 2023
@github-actions github-actions bot added the tool Affects the "flutter" command-line tool. See also t: labels. label Oct 24, 2023
@auto-submit auto-submit bot merged commit 8df6218 into master Oct 24, 2023
@auto-submit auto-submit bot deleted the revert_fb297e1d3bcc261563e254f9821cdf4bf11a0555 branch October 24, 2023 02:38
@CaseyHillers
Copy link
Contributor

Tracking bug is b/307433938 with the breakages.

@zanderso
Copy link
Member

cc @liamappelbe

@CaseyHillers Not sure if Liam was notified, but when reverting a change, please be sure to notify the PR author.

@CaseyHillers
Copy link
Contributor

@CaseyHillers Not sure if Liam was notified, but when reverting a change, please be sure to notify the PR author.

Thanks for letting me know! We chatted offline, but I filed #137156 as it seems like a bug with people not getting notified.

engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Oct 24, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Oct 25, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Oct 25, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Oct 25, 2023
engine-flutter-autoroll added a commit to engine-flutter-autoroll/packages that referenced this pull request Oct 25, 2023
auto-submit bot pushed a commit to flutter/packages that referenced this pull request Oct 25, 2023
Roll Flutter from 5e8b5f4 to 5dd2a4e (59 revisions)

flutter/flutter@5e8b5f4...5dd2a4e

2023-10-25 [email protected] Ensure Xcode project is setup to start debugger (flutter/flutter#136977)
2023-10-25 [email protected] Marks Windows build_tests_6_6 to be unflaky (flutter/flutter#137216)
2023-10-25 [email protected] Remove gem and docker files. (flutter/flutter#137200)
2023-10-25 98614782+auto-submit[bot]@users.noreply.github.com Reverts "[gallery] Reland roll gallery to  ecfb9e5352bd12032301b12b30d5853d83d89bda" (flutter/flutter#137264)
2023-10-25 [email protected] [gallery] Reland roll gallery to  ecfb9e5352bd12032301b12b30d5853d83d89bda (flutter/flutter#137199)
2023-10-25 [email protected] Roll Packages from 2faf992 to f751ffb (11 revisions) (flutter/flutter#137254)
2023-10-25 [email protected] Marks Windows build_tests_3_6 to be unflaky (flutter/flutter#137214)
2023-10-25 [email protected] Marks Windows build_tests_5_6 to be unflaky (flutter/flutter#137215)
2023-10-25 [email protected] Marks Windows build_tests_2_6 to be unflaky (flutter/flutter#137213)
2023-10-25 [email protected] Revert "Update `OutlinedButton` tests for Material 3 (#136809)" (flutter/flutter#137242)
2023-10-25 [email protected] Update `OutlinedButton` tests for Material 3 (flutter/flutter#136809)
2023-10-25 [email protected] Let `OverflowBox` be shrink-wrappable (flutter/flutter#129095)
2023-10-25 [email protected] Add dependency on leak_tracker to flutter_test. (flutter/flutter#137069)
2023-10-25 [email protected] fix SliverReorderableLists item wrong offset (flutter/flutter#136828)
2023-10-25 [email protected] Remove `bringup: true` from realm_checker and remove the redundant tool test. (flutter/flutter#137186)
2023-10-25 [email protected] Revert "Fix Gradle lockfiles." (flutter/flutter#137198)
2023-10-24 [email protected] Fix Gradle lockfiles. (flutter/flutter#137190)
2023-10-24 [email protected] Fix Typos (flutter/flutter#137173)
2023-10-24 [email protected] Roll Flutter Engine from d6a48e9963df to 6e09ee14e244 (1 revision) (flutter/flutter#137185)
2023-10-24 [email protected] Roll Flutter Engine from 602b5131ee4d to d6a48e9963df (6 revisions) (flutter/flutter#137180)
2023-10-24 [email protected] TextField - allow to customize cursor color in error state (flutter/flutter#136121)
2023-10-24 [email protected] [macOS] Refactor macOS build/codesize analysis (flutter/flutter#137164)
2023-10-24 [email protected] Roll Flutter Engine from dc00de7dacf4 to 602b5131ee4d (1 revision) (flutter/flutter#137174)
2023-10-24 [email protected] Check the realm file in its own shard. (flutter/flutter#137160)
2023-10-24 [email protected] Roll Flutter Engine from 1890bd7dc412 to dc00de7dacf4 (1 revision) (flutter/flutter#137172)
2023-10-24 [email protected] Roll Flutter Engine from ca7ef01c99c6 to 1890bd7dc412 (2 revisions) (flutter/flutter#137158)
2023-10-24 [email protected] Migrate mac builds to ruby dep. (flutter/flutter#136929)
2023-10-24 [email protected] Roll Packages from 4bf5114 to 2faf992 (7 revisions) (flutter/flutter#137154)
2023-10-24 [email protected] Roll Flutter Engine from 7224835bb65f to ca7ef01c99c6 (1 revision) (flutter/flutter#137153)
2023-10-24 [email protected] fix some typos (flutter/flutter#137144)
2023-10-24 [email protected] Roll Flutter Engine from 8f7488e0b4b5 to 7224835bb65f (1 revision) (flutter/flutter#137150)
2023-10-24 [email protected] Roll Flutter Engine from d58895093d0a to 8f7488e0b4b5 (3 revisions) (flutter/flutter#137135)
2023-10-24 [email protected] Roll Flutter Engine from 2a8471f188cd to d58895093d0a (4 revisions) (flutter/flutter#137130)
2023-10-24 [email protected] Roll Flutter Engine from 71600d89aa3f to 2a8471f188cd (2 revisions) (flutter/flutter#137129)
2023-10-24 [email protected] Roll Flutter Engine from 3dd197dcf9e5 to 71600d89aa3f (1 revision) (flutter/flutter#137127)
2023-10-24 [email protected] Roll Flutter Engine from bdbd1058858b to 3dd197dcf9e5 (2 revisions) (flutter/flutter#137125)
2023-10-24 [email protected] Roll Flutter Engine from 0ef5caa91c40 to bdbd1058858b (1 revision) (flutter/flutter#137119)
2023-10-24 98614782+auto-submit[bot]@users.noreply.github.com Reverts "Use `coverage.collect`'s `coverableLineCache` param to speed up coverage" (flutter/flutter#137121)
2023-10-24 [email protected] Roll Flutter Engine from dd6a6531e816 to 0ef5caa91c40 (1 revision) (flutter/flutter#137114)
2023-10-24 [email protected] Roll Flutter Engine from 0bd703132cc1 to dd6a6531e816 (2 revisions) (flutter/flutter#137111)
2023-10-23 [email protected] Roll Flutter Engine from abeab897a29c to 0bd703132cc1 (3 revisions) (flutter/flutter#137107)
2023-10-23 [email protected] Upgrade packages in flutter and flutter_test. (flutter/flutter#137106)
2023-10-23 49699333+dependabot[bot]@users.noreply.github.com Bump ossf/scorecard-action from 2.2.0 to 2.3.1 (flutter/flutter#137103)
2023-10-23 [email protected] Dartdoc warnings (flutter/flutter#137077)
2023-10-23 [email protected] Roll Flutter Engine from e89f0421bcab to abeab897a29c (5 revisions) (flutter/flutter#137100)
2023-10-23 [email protected] Cover last test/material tests with leak tracking. (flutter/flutter#137004)
...
liamappelbe added a commit that referenced this pull request Oct 29, 2023
Relands #136851, which was rolled back in #137121

package:coverage has been rolled, so the breakages should be fixed.
Also, in this reland I've changed the `coverableLineCache` parameter to
be optional, which is safer.
ksokolovskyi pushed a commit to ksokolovskyi/flutter that referenced this pull request Oct 31, 2023
…7385)

Relands flutter#136851, which was rolled back in flutter#137121

package:coverage has been rolled, so the breakages should be fixed.
Also, in this reland I've changed the `coverableLineCache` parameter to
be optional, which is safer.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

revert of Bot Only: Tracking label for bot. Tracks new revert of pull requests. tool Affects the "flutter" command-line tool. See also t: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants