Skip to content

Moves linux to ThinLTO. - #196314

Merged
auto-submit[bot] merged 1 commit into
flutter:masterfrom
gaaclarke:linux-thin-lto
Nov 11, 2026
Merged

auto-submit[bot] merged 1 commit into
flutter:masterfrom
gaaclarke:linux-thin-lto

Conversation

@gaaclarke

@gaaclarke gaaclarke commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

The actual measurements are in #196308, but the tradeoffs are roughly:

  • 7%(1.2MB) bigger engine binary
  • 6%(10s) slower build
  • 13% faster CPU workloads

This is a far cry from the 100% build time with full LTO that caused us to turn off LTO for linux. I think this is a fair tradeoff, especially for a platform like linux with is more accepting off deviations on binary size.

issue #196308

cc @Rusino

Pre-launch Checklist

If you need help, consider asking for advice on the #hackers-new channel on Discord.

If this change needs to override an active code freeze, provide a comment explaining why. The code freeze workflow can be overridden by code reviewers. See pinned issues for any active code freezes with guidance.

Note: The Flutter team is currently trialing the use of Gemini Code Assist for GitHub. Comments from the gemini-code-assist bot should not be taken as authoritative feedback from the Flutter team. If you find its comments useful you can update your code accordingly, but if you are unsure or disagree with the feedback, please feel free to wait for a Flutter team member's review for guidance on which automated comments should be addressed.

@github-actions github-actions Bot added the engine flutter/engine related. See also e: labels. label Nov 10, 2026
@jtmcdole jtmcdole added the CICD Run CI/CD label Nov 10, 2026
@gaaclarke
gaaclarke marked this pull request as ready for review November 10, 2026 20:43
@gaaclarke
gaaclarke requested a review from jtmcdole November 10, 2026 20:43

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for ThinLTO by adding --thin-lto and --no-thin-lto command-line arguments to the GN build tool, configuring compiler build settings to apply -flto=thin when enabled, and updating CI builder configurations to use --thin-lto. It also adds corresponding unit tests in gn_test.py. Feedback suggests expanding the unit tests to explicitly assert the behavior of the --no-thin-lto flag.

Comment on lines +75 to +87
@unittest.skipIf(sys.platform.startswith('win'), 'LTO GN args are not set on Windows')
def test_thin_lto(self):
release = ['--runtime-mode', 'release']
self.assertFalse(self._gn_args(release)['use_thin_lto'])
self.assertTrue(self._gn_args(release + ['--thin-lto'])['use_thin_lto'])
self.assertTrue(self._gn_args(release + ['--thin-lto'])['enable_lto'])
# ThinLTO has no effect when LTO is disabled.
no_lto_args = self._gn_args(release + ['--thin-lto', '--no-lto'])
self.assertFalse(no_lto_args['enable_lto'])
self.assertNotIn('use_thin_lto', no_lto_args)
unopt_args = self._gn_args(release + ['--thin-lto', '--unoptimized'])
self.assertFalse(unopt_args['enable_lto'])
self.assertNotIn('use_thin_lto', unopt_args)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To ensure comprehensive test coverage for the new CLI options, consider adding an assertion that verifies the behavior of the --no-thin-lto flag. This ensures that explicitly disabling ThinLTO works as expected when combined with other flags.

Suggested change
@unittest.skipIf(sys.platform.startswith('win'), 'LTO GN args are not set on Windows')
def test_thin_lto(self):
release = ['--runtime-mode', 'release']
self.assertFalse(self._gn_args(release)['use_thin_lto'])
self.assertTrue(self._gn_args(release + ['--thin-lto'])['use_thin_lto'])
self.assertTrue(self._gn_args(release + ['--thin-lto'])['enable_lto'])
# ThinLTO has no effect when LTO is disabled.
no_lto_args = self._gn_args(release + ['--thin-lto', '--no-lto'])
self.assertFalse(no_lto_args['enable_lto'])
self.assertNotIn('use_thin_lto', no_lto_args)
unopt_args = self._gn_args(release + ['--thin-lto', '--unoptimized'])
self.assertFalse(unopt_args['enable_lto'])
self.assertNotIn('use_thin_lto', unopt_args)
@unittest.skipIf(sys.platform.startswith('win'), 'LTO GN args are not set on Windows')
def test_thin_lto(self):
release = ['--runtime-mode', 'release']
self.assertFalse(self._gn_args(release)['use_thin_lto'])
self.assertTrue(self._gn_args(release + ['--thin-lto'])['use_thin_lto'])
self.assertTrue(self._gn_args(release + ['--thin-lto'])['enable_lto'])
# ThinLTO can be explicitly disabled.
self.assertFalse(self._gn_args(release + ['--thin-lto', '--no-thin-lto'])['use_thin_lto'])
# ThinLTO has no effect when LTO is disabled.
no_lto_args = self._gn_args(release + ['--thin-lto', '--no-lto'])
self.assertFalse(no_lto_args['enable_lto'])
self.assertNotIn('use_thin_lto', no_lto_args)
unopt_args = self._gn_args(release + ['--thin-lto', '--unoptimized'])
self.assertFalse(unopt_args['enable_lto'])
self.assertNotIn('use_thin_lto', unopt_args)

@gaaclarke gaaclarke added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 10, 2026
@auto-submit auto-submit Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 11, 2026
@auto-submit

auto-submit Bot commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

autosubmit label was removed for flutter/flutter/196314, because - The status or check suite Google testing has failed. Please fix the issues identified (or deflake) before re-applying this label.

@Rusino

Rusino commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Thanks, this matches what we measured on our side: on a dedicated Linux x86-64 machine, txt_benchmarks (19 scenarios) got 15.6% faster (geomean) with ThinLTO and 24.2% faster with full LTO, both compared to --no-lto.

Four things to consider, in order of importance:

  1. This change does not bring the engine benchmarks back. They have not run since the monorepo merge (Engine benchmarks are no longer running after monorepo #185368, still open), and flutter-engine-perf.luci.app has shown no points since Dec 24, 2024. So a text regression in Skia would still go unnoticed on the Flutter side.

  2. The benchmark builder does not follow this change. engine/src/flutter/ci/builders/standalone/linux_benchmarks.json passes neither --no-lto nor --thin-lto, and in engine/src/flutter/tools/gn --lto defaults to true while --thin-lto defaults to false. So ci/host_release_benchmarks, which builds and uploads txt_benchmarks, keeps measuring full LTO, while Linux desktop ships ThinLTO. Full LTO also hides some regressions: Skia CL 1406297 cost +19.3% on JustifyLayout without LTO but only +7.6% with full LTO. It may be worth passing --thin-lto there too, or running both modes, since mobile still ships full LTO.

  3. There is no -import-instr-limit, so ThinLTO uses the Clang default of 100. Chrome sets -Wl,-mllvm,-import-instr-limit=30 on desktop to keep binary size down. That might win back part of the +1.2 MB while keeping most of the speedup. We have not measured this for Flutter.

  4. There is no ThinLTO cache. -Wl,--thinlto-cache-dir=<dir> (lld) should make the 13.6 s incremental relink much cheaper for local release and profile builds. It matters less on RBE CI.

macOS desktop stays on --no-lto and Windows has no LTO, so no-LTO benchmark numbers still matter for those platforms.

@gaaclarke gaaclarke added the autosubmit Merge PR when tree becomes green via auto submit App label Nov 11, 2026
@auto-submit
auto-submit Bot added this pull request to the merge queue Nov 11, 2026
Merged via the queue into flutter:master with commit f9d2a0a Nov 11, 2026
31 checks passed
@flutter-dashboard flutter-dashboard Bot removed the autosubmit Merge PR when tree becomes green via auto submit App label Nov 11, 2026
@jmagman jmagman mentioned this pull request Nov 11, 2026
10 of 11 tasks
auto-submit Bot pushed a commit to flutter/packages that referenced this pull request Nov 12, 2026
flutter/flutter@b98afb5...9c427b5

2026-11-12 [email protected] Roll Skia from d67d0f22e3ec to 9aa2513a4ac9 (2 revisions) (flutter/flutter#196429)
2026-11-12 [email protected] [flutter_tools] Move CpuArch to flutter_tools_core and define HostPlatform by osName and cpuArch (flutter/flutter#195841)
2026-11-12 [email protected] Roll Skia from 23e7e63789b8 to d67d0f22e3ec (2 revisions) (flutter/flutter#196424)
2026-11-12 [email protected] Uber compactify (flutter/flutter#196392)
2026-11-12 [email protected] Roll Skia from 3d7e335da165 to 23e7e63789b8 (12 revisions) (flutter/flutter#196422)
2026-11-12 [email protected] Roll Dart SDK from 13af329ea4f2 to e5add7a25b32 (1 revision) (flutter/flutter#196413)
2026-11-12 [email protected] Don't terminate the shared EGL display when an engine is disposed (flutter/flutter#194520)
2026-11-11 [email protected] Adds manifest flag to turn on SDFs for Android. (flutter/flutter#196056)
2026-11-11 [email protected] Remove bringup from Linux_pixel_7pro integration_ui_keyboard_inset_jump (flutter/flutter#196334)
2026-11-11 [email protected] Uses system default locale for text rendering when preferred locale isn't supported (flutter/flutter#195997)
2026-11-11 [email protected] [android]: prewarms shaders when the context is created (flutter/flutter#196069)
2026-11-11 [email protected] iOS: Recomposite platform views only if their parameters change (flutter/flutter#196287)
2026-11-11 [email protected] [flutter_tools] Forward encoding in ErrorHandlingFile.readAsStringSync (flutter/flutter#196136)
2026-11-11 [email protected] Roll Skia from b89c24bbab8c to 3d7e335da165 (8 revisions) (flutter/flutter#196390)
2026-11-11 [email protected] Add Flutter Gardener instructions on how to disable a test (flutter/flutter#196312)
2026-11-11 [email protected] [Impeller] Enable malioc analysis for framebuffer_blend shaders (flutter/flutter#196343)
2026-11-11 [email protected] [flutter_tools] Migrate general.shard Android, asset, and build tests off globals (flutter/flutter#196125)
2026-11-11 [email protected] Roll Dart SDK from 8f15c4ca2d41 to 13af329ea4f2 (1 revision) (flutter/flutter#196386)
2026-11-11 [email protected] [Impeller] Do not clamp the local coverage limit to the root render target (flutter/flutter#195860)
2026-11-11 [email protected] [Impeller] Use separate uniform buffers on VMware SVGA3D (flutter/flutter#196022)
2026-11-11 [email protected] [flutter_tools] Introduce AndroidSdkProvider for lazy native_assets SDK lookup (flutter/flutter#196122)
2026-11-11 [email protected] Moves linux to ThinLTO. (flutter/flutter#196314)
2026-11-11 [email protected] [flutter_tools] Migrate bin/fuchsia_tester.dart off globals and runInContext (flutter/flutter#196074)
2026-11-11 [email protected] Changes the tolerances for frame_rate_main_test.dart. (flutter/flutter#196304)
2026-11-11 [email protected] Roll Packages from 08faf64 to 653628e (16 revisions) (flutter/flutter#196381)
2026-11-11 [email protected] Allow SliverCrossAxisGroup children to lay out with zero cross-axis extent instead of asserting (flutter/flutter#191287)
2026-11-11 [email protected] Roll Skia from cc4c54f752ba to b89c24bbab8c (4 revisions) (flutter/flutter#196375)
2026-11-11 [email protected] [Linux] Fix layer offset when compositing with glBlitFramebuffer (flutter/flutter#196189)
2026-11-11 [email protected] [flutter_tools] Inject CrashReportSender into SwiftPM migration (flutter/flutter#196315)

If this roll has caused a breakage, revert this CL and set the roller
to dry run mode using the controls here:
https://autoroll.skia.org/r/flutter-packages
Please CC [email protected],[email protected] on the revert to ensure that a human
is aware of the problem.

To file a bug in Packages: https://github.com/flutter/flutter/issues/new/choose

To report a problem with the AutoRoller itself, please file a bug:
https://issues.skia.org/issues/new?component=1389291&template=1850622

Documentation for the AutoRoller is here:
https://skia.googlesource.com/buildbot/+doc/main/autoroll/README.md
chingjun added a commit to chingjun/flutter that referenced this pull request Nov 12, 2026
Upstream moved the Linux profile and release builds to ThinLTO
(flutter#196314): tools/gn --thin-lto passes GN's use_thin_lto under
enable_lto, and //build/config/compiler:optimize then has -flto=thin
where full LTO has -flto. The merge left the seven of those builds that
run under Bazel on --no-lto. This takes upstream's flag in them and
gives the Bazel build the setting.

- //engine/src/flutter/build/bazel:thin_lto, a flag beside `lto`, and
  the toolchain configuration's `thin_lto` attribute: under it the
  `lto` feature has -flto=thin on compiles and links (the Linux,
  Android, macOS and iOS toolchains share the configuration). It does
  nothing without `lto`, as in GN.
- The converter maps --thin-lto and --no-thin-lto to the setting in a
  build with LTO and to nothing otherwise; to nothing for the web too,
  for which tools/gn reads no LTO flag. Seven configurations gain
  `lto` and `thin_lto`: ci_host_release, host_profile_desktop,
  host_release_desktop, linux_profile_arm64_desktop,
  linux_release_arm64_desktop, host_profile_test, host_release_test.
- in_configuration carries the setting, and the tests' Android
  artifacts say `thin_lto = "false"`: the Android builders build with
  full LTO whatever the build around them has.
- The recipe's --no-lto, in a release builder's presubmit, turns LTO off
  in such a build as in any other (bazel_ci.py's match is unchanged; a
  test). ThinLTO therefore runs in the three release builders'
  postsubmit builds, and in every build of linux_host_engine_test.

Not done: the wrapped Dart SDK build still compiles the VM without LTO
in every build. GN compiles it with -flto=thin in these builds (gn desc
of libdart_vm_aotruntime_product), as with -flto in the Android AOT and
iOS device builds. The README's known gaps now say so.

Checked against GN (tools/gn --runtime-mode release --thin-lto, gn
desc): //flutter/fml:fml's cflags and flutter_tester's ldflags have
-flto=thin once, as Bazel's 46 compiles of fml and the link of
flutter_tester under --config=host_release_desktop do; the host tools
(the execution configuration) have it too, as GN's one toolchain does;
with --//engine/src/flutter/build/bazel:lto=false after the
configuration fml's compiles have no LTO flag.

Built and run locally: `bazelisk test --config=host_release_test
//engine/src/flutter/fml:fml_unittests
//engine/src/flutter/shell/common:shell_unittests` passes (fml's 46
objects are ThinLTO bitcode), and `bazelisk build
--config=host_release_desktop
//engine/src/flutter/ci/builders:host_release_desktop` builds; its
libflutter_linux_gtk.so exports the same 310 symbols as the debug
build's, which has no LTO (that one has four more, the debug mode's
snapshot data).

Also: gn_to_bazel's tests (284) and --check, check_build_configs.sh,
check_generated_bazel_files.sh, bazel_ci_test.py (23),
engine_build_configs (84), bazel_tools (46), buildifier, yapf; Bazel
analysis of the seven builds' install targets, of ci_host_debug and
ci_android_release_arm64 (whose compiles keep the bare -flto), and of
//:tool_tests //:dart_tests.

Not built: the arm64 cross builds, and the profile mode. Not measured:
the test builders' ThinLTO links of every unit test on RBE workers.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CICD Run CI/CD engine flutter/engine related. See also e: labels.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants