Skip to content

"Mac tool_integration_tests" builds take too long, add subshards #189039

Description

@jmagman
Builder Machine Type P25 Duration In Minutes Average Duration In Minutes P75 Duration In Minutes Total Builds Duration Bucket
Mac tool_integration_tests_4_5 n1-highcpu-8 33.62 34.72 34.97 88 30-35
Mac tool_integration_tests_1_5 n1-highcpu-8 30.68 31.45 31.73 90 30-35
Mac tool_integration_tests_5_5  n1-highcpu-8 29.48 30.40 30.87 89 30-35
Mac tool_integration_tests_3_5  n1-highcpu-8 20.30 20.68 21.53 95 20-25
Mac tool_integration_tests_2_5 n1-highcpu-8 16.03 16.76 16.90 83 15-20

All 5 Mac tool_integration_tests_* are regularly taking more than the targeted 10-15 minutes. While we could be smarter about bucketing, we could also take the easier road and just increase the number of subshards. Mac currently has 5 shards. For comparison, Windows has 10 and Linux has 7. Increase the number of shards until each are under 15 minutes.
If there are still outliers then consider smarter organization, like pulling out the longest running ones into separate shards.

Currently, the try pool capacity for this test is 129 bots and the prod pool has 56 bots.

Set like this:

flutter/.ci.yaml

Line 4576 in bcabd8e

- name: Mac tool_integration_tests_1_5

flutter/.ci.yaml

Line 4591 in bcabd8e

subshard: "1_5"

Activity

  1. added
    c: contributor-productivityTeam-specific productivity, code health, technical debt.
    fyi-infraFor the attention of Infrastructure team
    team-iosOwned by iOS platform team
    fyi-toolFor the attention of Flutter Tool team
    on Jul 7, 2026
  2. jmagman commented on Jul 7, 2026

    @jmagman
    MemberAuthor

    @jtmcdole it would be helpful if "Flutter Builders Machine Times" included queue time metrics. If I go extreme and make 100 shards, then individual build time will be very small, but we would saturate capacity and queue time would increase.
    go/flutter-infra-metrics has queue time under Devicelab LUCI metrics, for example.

  3. changed the title [-]"Mac tool_integration_tests" builds take too long, add shards[/-] [+]"Mac tool_integration_tests" builds take too long, add subshards[/+] on Jul 7, 2026
  4. added
    fyi-iosFor the attention of iOS platform team
    team-toolOwned by Flutter Tool team
    a: tests"flutter test", flutter_test, or one of our tests
    toolAffects the "flutter" command-line tool. See also t: labels.
    and removed
    team-iosOwned by iOS platform team
    fyi-toolFor the attention of Flutter Tool team
    on Jul 7, 2026
  5. jtmcdole commented on Jul 9, 2026

    @jtmcdole
    Member

    [@jtmcdole](https://github.com/jtmcdole) it would be helpful if "Flutter Builders Machine Times" included queue time metrics. If I go extreme and make 100 shards, then individual build time will be very small, but we would saturate capacity and queue time would increase. go/flutter-infra-metrics has queue time under Devicelab LUCI metrics, for example.

    Good ask - I bet we have that tracked in BigQuery somewhere.

  6. jtmcdole commented on Jul 9, 2026

    @jtmcdole
    Member

    Done.

  7. 8 remaining items

  8. jmagman commented on Jul 10, 2026

    @jmagman
    MemberAuthor

    #189040 landed, but the new builder Mac tool_integration_tests_1 didn't show up, and the old ones like Mac tool_integration_tests_1_5 disappeared and reappeared:

    Image

    Infra team, did something go wrong with the infra config syncing?

  9. added
    team-infraOwned by Infrastructure team
    and removed
    fyi-infraFor the attention of Infrastructure team
    team-toolOwned by Flutter Tool team
    on Jul 10, 2026
  10. jmagman commented on Jul 13, 2026

    @jmagman
    MemberAuthor

    #189040 landed, but the new builder Mac tool_integration_tests_1 didn't show up, and the old ones like Mac tool_integration_tests_1_5 disappeared and reappeared:

    Image Infra team, did something go wrong with the infra config syncing?

    It showed up, guess it took another commit to sync.

  11. added
    team-toolOwned by Flutter Tool team
    and removed
    team-infraOwned by Infrastructure team
    on Jul 13, 2026
  12. github-actions commented on Jul 27, 2026

    @github-actions

    This thread has been automatically locked since there has not been any recent activity after it was closed. If you are still experiencing a similar issue, please open a new bug, including the output of flutter doctor -v and a minimal reproduction of the issue.

  13. locked as resolved and limited conversation to collaborators on Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

a: tests"flutter test", flutter_test, or one of our testsc: contributor-productivityTeam-specific productivity, code health, technical debt.team-toolOwned by Flutter Tool teamtoolAffects the "flutter" command-line tool. See also t: labels.

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions