Skip to content

Increase number of 'Mac tool_integration_tests' subshards from 5 to 10#189040

Merged
auto-submit[bot] merged 5 commits into
flutter:masterfrom
jmagman:mac-tool-test-shards
Jul 10, 2026
Merged

Increase number of 'Mac tool_integration_tests' subshards from 5 to 10#189040
auto-submit[bot] merged 5 commits into
flutter:masterfrom
jmagman:mac-tool-test-shards

Conversation

@jmagman

@jmagman jmagman commented Jul 7, 2026

Copy link
Copy Markdown
Member

Rename builders from tool_integration_tests_1_5 to tool_integration_tests_1 so the total number isn't encoded in the builder name. Because it currently is encoded in the builder name (which isn't parsed, it's just informational) the original builders need to become bringup:true unnecessarily, since adding shards creates new builders.

Queue times are ~4 seconds, which means we seem to have enough capacity, so throwing more shards at this problem shouldn't be an issue. Will keep an eye on queue times after this change, though.

Fixes #189039

7 shards

Shard Time
1 23 minutes
2 14 minutes
3 14 minutes
4 16 minutes
5 25 minutes

9 shards

Shard Time
1 19+ minutes
2 13 minutes
3 11 minutes
4 15 minutes
5 13 minutes

10 shards

Shard Time
1 13 minutes
2 22 minutes
3 9 minutes
4 12 minutes
5 9.5 minutes

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

@jmagman jmagman added the CICD Run CI/CD label Jul 7, 2026
@jmagman jmagman changed the title Increase number of 'Mac tool_integration_tests' shards Increase number of 'Mac tool_integration_tests' subshards Jul 7, 2026
@jmagman
jmagman force-pushed the mac-tool-test-shards branch from 1687ca1 to ff95767 Compare July 10, 2026 21:00
@jmagman
jmagman marked this pull request as ready for review July 10, 2026 21:19
@jmagman jmagman changed the title Increase number of 'Mac tool_integration_tests' subshards Increase number of 'Mac tool_integration_tests' subshards from 5 to 10 Jul 10, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the CI configuration in .ci.yaml to split the Mac tool_integration_tests into 10 shards instead of 5, adding bringup: true and adjusting subshard configurations. Feedback on the changes highlights that the newly added shards (6 through 10) unnecessarily include extra dependencies such as android_sdk, chrome_and_driver, open_jdk, and ruby, along with the add_recipes_cq property. Since these are host-only tests, they should only require goldctl to match the existing shards and avoid increased setup times.

Comment thread .ci.yaml
Comment on lines +4876 to +4884
add_recipes_cq: "true"
dependencies: >-
[
{"dependency": "android_sdk", "version": "version:36v9unmodified"},
{"dependency": "chrome_and_driver", "version": "version:145.0.7632.117"},
{"dependency": "open_jdk", "version": "version:21"},
{"dependency": "ruby", "version": "ruby_3.1-pod_1.13"},
{"dependency": "goldctl", "version": "git_revision:c845c41b9b81bfcb11f2f0ab17b5b2386d634c31"}
]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The new shards (6 to 10) have been configured with several extra dependencies (android_sdk, chrome_and_driver, open_jdk, ruby) and the add_recipes_cq: "true" property, which are not present in the existing shards (1 to 5). Since these are host-only tool integration tests, they only require goldctl. Including these extra dependencies will unnecessarily increase setup times and resource usage on the bots.

      dependencies: >-
        [
          {"dependency": "goldctl", "version": "git_revision:c845c41b9b81bfcb11f2f0ab17b5b2386d634c31"}
        ]

@jmagman jmagman Jul 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, the existing shards have these dependencies:

      add_recipes_cq: "true"
      dependencies: >-
        [
          {"dependency": "android_sdk", "version": "version:36v9unmodified"},
          {"dependency": "chrome_and_driver", "version": "version:145.0.7632.117"},
          {"dependency": "open_jdk", "version": "version:21"},
          {"dependency": "ruby", "version": "ruby_3.1-pod_1.13"},
          {"dependency": "goldctl", "version": "git_revision:c845c41b9b81bfcb11f2f0ab17b5b2386d634c31"}
        ]

Comment thread .ci.yaml
Comment thread .ci.yaml
Comment thread .ci.yaml
Comment thread .ci.yaml
@jmagman
jmagman requested a review from jtmcdole July 10, 2026 21:20
@jmagman jmagman added the autosubmit Merge PR when tree becomes green via auto submit App label Jul 10, 2026
@auto-submit
auto-submit Bot added this pull request to the merge queue Jul 10, 2026
Merged via the queue into flutter:master with commit 4375491 Jul 10, 2026
173 checks passed
@flutter-dashboard flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Jul 10, 2026
@jmagman
jmagman deleted the mac-tool-test-shards branch July 10, 2026 23:00
@jmagman jmagman added cp: beta cherry pick this pull request to beta release candidate branch cp: stable cherry pick this pull request to stable release candidate branch labels Jul 10, 2026
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request Jul 12, 2026
flutter/flutter@f7b66f3...cf9e8af

2026-07-12 [email protected] Flip from deprecated strict analysis modes to lint rules (flutter/flutter#187692)
2026-07-12 [email protected] Roll Skia from 6ecffccd32d5 to 8bf65996caba (5 revisions) (flutter/flutter#189333)
2026-07-11 [email protected] Roll Fuchsia Linux SDK from czpzDg9ABY2oKLAOY... to vhIlDkWIy21IrlB9E... (flutter/flutter#189309)
2026-07-11 [email protected] iOS: clean up swiftc.py after migration to target triples (flutter/flutter#189240)
2026-07-10 [email protected] Increase number of 'Mac tool_integration_tests' subshards from 5 to 10 (flutter/flutter#189040)
2026-07-10 [email protected] Fix PopupWindowControllerLinux missing WindowControllerLinux implementation (flutter/flutter#189189)
2026-07-10 [email protected] [android_hardware_smoke_test] Synchronize platform view compositing (flutter/flutter#189151)
2026-07-10 [email protected] UberSDF thin roundrect handling (flutter/flutter#189224)
2026-07-10 [email protected] Roll Skia from ab3a7b98c94d to 6ecffccd32d5 (15 revisions) (flutter/flutter#189270)
2026-07-10 [email protected] [skwasm][canvaskit] Honor Paint.filterQuality in Canvas.drawAtlas (flutter/flutter#186108)
2026-07-10 [email protected] Android_hardware_smoke_test: Cache flutter engine when running vulkan backend instrumented tests (flutter/flutter#189026)

If this roll has caused a breakage, revert this CL and stop the roller
using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
bkonyi pushed a commit to bkonyi/flutter that referenced this pull request Jul 13, 2026
New (renamed) builders from
flutter#189040 are passing.

Fixes flutter#189039

## Pre-launch Checklist

- [x] I read the [Contributor Guide] and followed the process outlined
there for submitting PRs.
- [x] I read the [AI contribution guidelines] and understand my
responsibilities, or I am not using AI tools.
- [x] I read the [Tree Hygiene] wiki page, which explains my
responsibilities.
- [x] I read and followed the [Flutter Style Guide], including [Features
we expect every widget to implement].
- [x] I signed the [CLA].
- [x] I listed at least one issue that this PR fixes in the description
above.
- [x] I updated/added relevant documentation (doc comments with `///`).
- [x] I added new tests to check the change I am making, or this PR is
[test-exempt].
- [x] I followed the [breaking change policy] and added [Data Driven
Fixes] where supported.
- [x] All existing and new tests are passing.

If you need help, consider asking for advice on the #hackers-new channel
on [Discord].

If this change needs to override an active code freeze, provide a
comment explaining why. The code freeze workflow can be overridden by
code reviewers. See pinned issues for any active code freezes with
guidance.

**Note**: The Flutter team is currently trialing the use of [Gemini Code
Assist for
GitHub](https://developers.google.com/gemini-code-assist/docs/review-github-code).
Comments from the `gemini-code-assist` bot should not be taken as
authoritative feedback from the Flutter team. If you find its comments
useful you can update your code accordingly, but if you are unsure or
disagree with the feedback, please feel free to wait for a Flutter team
member's review for guidance on which automated comments should be
addressed.

<!-- Links -->
[Contributor Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#overview
[AI contribution guidelines]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#ai-contribution-guidelines
[Tree Hygiene]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md
[test-exempt]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#tests
[Flutter Style Guide]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md
[Features we expect every widget to implement]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Style-guide-for-Flutter-repo.md#features-we-expect-every-widget-to-implement
[CLA]: https://cla.developers.google.com/
[flutter/tests]: https://github.com/flutter/tests
[breaking change policy]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Tree-hygiene.md#handling-breaking-changes
[Discord]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Chat.md
[Data Driven Fixes]:
https://github.com/flutter/flutter/blob/main/docs/contributing/Data-driven-Fixes.md
auto-submit Bot pushed a commit that referenced this pull request Jul 15, 2026
…189299)

This pull request is created by [automatic cherry pick workflow](https://github.com/flutter/flutter/blob/main/docs/releases/Flutter-Cherrypick-Process.md#automatically-creates-a-cherry-pick-request)
Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request.

### Issue Link:
What is the link to the issue this cherry-pick is addressing?

#189039

### Impact Description:
What is the impact (ex. visual jank on Samsung phones, app crash, cannot ship an iOS app)?
Does it impact development (ex. flutter doctor crashes when Android Studio is installed),
or the shipping of production apps (the app crashes on launch).
This information is for domain experts and release engineers to understand the consequences of saying yes or no to the cherry pick.

Release builders should match main. These builders were renamed in #189040 to increase the number of shards. Without this change, the beta release branch will try to run the old builder names and fail because the infra config is missing.

There will be a follow-up cherry-pick to bring it out of bringup when they pass in post-submit #189368

### Changelog Description:
Explain this cherry pick:
* In one line that is accessible to most Flutter developers.
* That describes the state prior to the fix.
* That includes which platforms are impacted.
See [best practices](https://github.com/flutter/flutter/blob/main/docs/releases/Hotfix-Documentation-Best-Practices.md) for examples.

< Replace with changelog description here >
[flutter/189039] Improve speed of Mac Flutter tool integration tests in CI

### Workaround:
Is there a workaround for this issue?

No

### Risk:
What is the risk level of this cherry-pick?

### Test Coverage:
Are you confident that your fix is well-tested by automated tests?

### Validation Steps:
What are the steps to validate that this fix works?

This PR correctly skips `Mac tool_integration_tests` in presubmit.
auto-submit Bot pushed a commit that referenced this pull request Jul 15, 2026
…189300)

This pull request is created by [automatic cherry pick workflow](https://github.com/flutter/flutter/blob/main/docs/releases/Flutter-Cherrypick-Process.md#automatically-creates-a-cherry-pick-request)
Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request.

### Issue Link:
What is the link to the issue this cherry-pick is addressing?

#189039

### Impact Description:
What is the impact (ex. visual jank on Samsung phones, app crash, cannot ship an iOS app)?
Does it impact development (ex. flutter doctor crashes when Android Studio is installed),
or the shipping of production apps (the app crashes on launch).
This information is for domain experts and release engineers to understand the consequences of saying yes or no to the cherry pick.

Release builders should match main. These builders were renamed in #189040 to increase the number of shards. Without this change, the stable release branch will try to run the old builder names and fail because the infra config is missing.

There will be a follow-up cherry-pick to bring it out of bringup when they pass in post-submit #189368

### Changelog Description:
Explain this cherry pick:
* In one line that is accessible to most Flutter developers.
* That describes the state prior to the fix.
* That includes which platforms are impacted.
See [best practices](https://github.com/flutter/flutter/blob/main/docs/releases/Hotfix-Documentation-Best-Practices.md) for examples.

< Replace with changelog description here >
[flutter/189039] Improve speed of Mac Flutter tool integration tests in CI

### Workaround:
Is there a workaround for this issue?

No

### Risk:
What is the risk level of this cherry-pick?

### Test Coverage:
Are you confident that your fix is well-tested by automated tests?

### Validation Steps:
What are the steps to validate that this fix works?

This PR correctly skips `Mac tool_integration_tests` in presubmit.
@eyebrowsoffire eyebrowsoffire added cp: beta cherry pick this pull request to beta release candidate branch and removed cp: beta cherry pick this pull request to beta release candidate branch labels Jul 17, 2026
@camsim99 camsim99 added cp: beta cherry pick this pull request to beta release candidate branch and removed cp: beta cherry pick this pull request to beta release candidate branch labels Jul 20, 2026
auto-submit Bot pushed a commit that referenced this pull request Jul 21, 2026
…om 5 to 10 (#189743)

> [!NOTE]  
> This was re-created from #189299, which pointed to the old beta branch (3.46).

This pull request is created by [automatic cherry pick workflow](https://github.com/flutter/flutter/blob/main/docs/releases/Flutter-Cherrypick-Process.md#automatically-creates-a-cherry-pick-request)
Please fill in the form below, and a flutter domain expert will evaluate this cherry pick request.

### Issue Link:
What is the link to the issue this cherry-pick is addressing?

#189039

### Impact Description:
What is the impact (ex. visual jank on Samsung phones, app crash, cannot ship an iOS app)?
Does it impact development (ex. flutter doctor crashes when Android Studio is installed),
or the shipping of production apps (the app crashes on launch).
This information is for domain experts and release engineers to understand the consequences of saying yes or no to the cherry pick.

Release builders should match main. These builders were renamed in #189040 to increase the number of shards. Without this change, the beta release branch will try to run the old builder names and fail because the infra config is missing.

There will be a follow-up cherry-pick to bring it out of bringup when they pass in post-submit #189368

### Changelog Description:
Explain this cherry pick:
* In one line that is accessible to most Flutter developers.
* That describes the state prior to the fix.
* That includes which platforms are impacted.
See [best practices](https://github.com/flutter/flutter/blob/main/docs/releases/Hotfix-Documentation-Best-Practices.md) for examples.

< Replace with changelog description here >
[flutter/189039] Improve speed of Mac Flutter tool integration tests in CI

### Workaround:
Is there a workaround for this issue?

No

### Risk:
What is the risk level of this cherry-pick?

### Test Coverage:
Are you confident that your fix is well-tested by automated tests?

### Validation Steps:
What are the steps to validate that this fix works?

This PR correctly skips `Mac tool_integration_tests` in presubmit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD cp: beta cherry pick this pull request to beta release candidate branch cp: stable cherry pick this pull request to stable release candidate branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

"Mac tool_integration_tests" builds take too long, add subshards

4 participants