Skip to content

[boring] Move package:boring to dart-lang/tools (3/4): prebuilt binaries and tree shaking - #2616

Merged
mosuem merged 10 commits into
mainfrom
boring/3-prebuilt-link-hook
Oct 23, 2026
Merged

mosuem merged 10 commits into
mainfrom
boring/3-prebuilt-link-hook

Conversation

@mosuem

@mosuem mosuem commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Part 3 of 4 of moving package:boring to dart-lang/tools. For an overview, see #2614.

Build modes. hook/build.dart gains three build modes:

  • fetch (default) downloads the library for the pinned release. It verifies the file against the SHA-256 hashes in lib/src/hook_helpers/hashes.dart. If there's no prebuilt library, it falls back to the CMake build.
  • checkout always builds from source.
  • local uses a library at a given path.

Tree shaking. When linking is enabled, hook/link.dart links a dynamic library that contains only the functions the application uses. The Dart compiler records these for the @RecordUse() bindings. For the example, the library shrinks from 2.9 MB to 240 KB. If linking fails in fetch mode, the hook bundles the prebuilt dynamic library instead. This happens, for example, when cross-compiling without a C toolchain for the target.

Tests. test/tree_shaking_test.dart builds the example natively, and for linux-arm64 both with and without a cross toolchain. The test job installs a cross toolchain on each OS. The Windows download is pinned by its SHA-256.

Release workflow. boring_binaries.yaml runs when a change to the native sources (src/, the vendored BoringSSL, tool/REVISION, or the pipeline itself) lands on main, and on workflow_dispatch on main:

  • The precompile job builds the dynamic and the static library for linux-{x64,arm64}, macos-{arm64,x64} and windows-x64, checks their ABI floors (glibc ≤ 2.35, macOS 10.15/11.0, no dynamically linked C runtime on Windows), and attests their provenance.
  • The release job attaches them to a boring-binaries-<date>-<commit> pre-release. The name doesn't match publish.yaml's <package>-v<version> pattern, so nothing is published.
  • The pin job writes the repository, tag, source commit and SHA-256 hashes to lib/src/hook_helpers/hashes.dart and opens a pull request with the Ecosystem Bot token, like the refresh job. The pull request's generated job re-hashes the release assets, and the tree-shaking tests download them in fetch mode.
  • Pull requests that change the pipeline or src/ only run precompile, so the build is validated before merge.
  • The analyze job of boring.yaml fails a release (a non--wip version) whose pinned libraries weren't built from the current native sources, and warns otherwise.

The hashes still point at the 0.4.1 release of mosuem/boring. Merging this PR changes src/, which triggers the first release from this repository and the pull request that switches them.

Review commits individually. The first commit is the port from mosuem/boring; the four that follow are review fixes: macOS deployment targets, a static C runtime on Windows and the ABI checks; strict buildMode parsing, a hash tool that fails on missing assets, download timeouts and proxy support, and unit tests; documentation; and the automated release flow above.

Labeled skip-changelog-check, because the hooks already shipped in 0.4.1; the CHANGELOG lists what changed since.

@mosuem
mosuem added this pull request to stack #2618 September 29, 2026 13:23
@github-actions github-actions Bot added the type-infra A repository infrastructure change or enhancement label Sep 29, 2026
@mosuem mosuem changed the title [boring] Add prebuilt binaries and tree shaking with a link hook [boring] Move package:boring to dart-lang/tools (3/4): prebuilt binaries and tree shaking Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

Package publishing

Package Version Status Publish tag (post-merge)
package:boring ⭐ 0.4.2-wip WIP (no publish necessary)
  • 21 already published.
  • 21 WIP (no publish necessary).

Documentation at https://github.com/dart-lang/ecosystem/wiki/Publishing-automation.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces tree-shaking support for BoringSSL functions in release builds, adds native asset build modes (fetch, checkout, and local), and includes a comprehensive guide on reusing the BoringSSL packaged with the Dart SDK. The feedback highlights a few important improvements: adding robust bounds and termination checks to the archive parsing logic in link.dart to prevent crashes on malformed files, ensuring the HTTP response stream is drained in fetch.dart when the status code is not 200 to avoid socket leaks, and dynamically constructing the asset ID in link.dart using input.packageName instead of hardcoding the package name.

Comment thread pkgs/boring/hook/link.dart Resolved
Comment thread pkgs/boring/hook/link.dart Resolved
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown

PR Health

Coverage ⚠️
File Coverage
pkgs/boring/hook/build.dart 💔 Not covered
pkgs/boring/hook/link.dart 💔 Not covered
pkgs/boring/lib/src/hook_helpers/build_options.dart 💚 88 %
pkgs/boring/lib/src/hook_helpers/fetch.dart 💚 17 %
pkgs/boring/lib/src/hook_helpers/hashes.dart 💔 Not covered
pkgs/boring/lib/src/hook_helpers/targets.dart 💚 100 %
pkgs/boring/tool/precompile_binaries.dart 💔 Not covered
pkgs/boring/tool/regenerate_hashes.dart 💔 Not covered

This check for test coverage is informational (issues shown here will not fail the PR).

This check can be disabled by tagging the PR with skip-coverage-check.

License Headers ✔️
// Copyright (c) 2026, the Dart project authors. Please see the AUTHORS file
// for details. All rights reserved. Use of this source code is governed by a
// BSD-style license that can be found in the LICENSE file.

Files
no missing headers

All source files should start with a license header.

Unrelated files missing license headers
Files
pkgs/html/test/differential_test.dart
pkgs/html/test/dom_parsing_test.dart
pkgs/html/test/mxss_test.dart
pkgs/html/test/noah_ark_clause_limit_test.dart

This check can be disabled by tagging the PR with skip-license-check.

Breaking changes ✔️
Package Change Current Version New Version Needed Version Looking good?
boring Breaking 0.4.1 0.4.2-wip 0.4.2-wip ✔️

This check can be disabled by tagging the PR with skip-breaking-check.

Unused Dependencies ✔️
Package Status
boring ✔️ All dependencies utilized correctly.

For details on how to fix these, see dependency_validator.

This check can be disabled by tagging the PR with skip-unused-dependencies-check.

API leaks ✔️

The following packages contain symbols visible in the public API, but not exported by the library. Export these symbols or remove them from your publicly visible API.

Package Leaked API symbol Leaking sources

This check can be disabled by tagging the PR with skip-leaking-check.

@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch 2 times, most recently from d1aa234 to 4b2d3b8 Compare September 30, 2026 10:48
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch from 4b2d3b8 to 58bcaf8 Compare September 30, 2026 12:25
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch from 58bcaf8 to 5370276 Compare September 30, 2026 12:26
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch 2 times, most recently from 0a5c929 to 5aa0b2f Compare September 30, 2026 12:49
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch from 5aa0b2f to 2c7519c Compare September 30, 2026 14:14
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch 2 times, most recently from 1ee3875 to 78b445f Compare October 7, 2026 09:35
Base automatically changed from boring/2-helpers to main October 9, 2026 06:54
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch 2 times, most recently from 548a899 to 87a3cba Compare October 12, 2026 09:05
@mosuem
mosuem removed this pull request from stack #2618 October 12, 2026 09:08
@mosuem
mosuem changed the base branch from main to boring/fix-api-leaks October 12, 2026 09:09
@mosuem
mosuem added this pull request to stack #2661 October 12, 2026 09:09
@mosuem
mosuem marked this pull request as ready for review October 12, 2026 09:18
@mosuem
mosuem requested a review from a team as a code owner October 12, 2026 09:18
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch from 87a3cba to c6f3597 Compare October 12, 2026 09:19
Comment thread .github/workflows/boring.yaml Resolved
Comment thread .github/workflows/boring.yaml Resolved
Comment thread pkgs/boring/README.md Resolved
Comment thread pkgs/boring/README.md Resolved
Comment thread pkgs/boring/README.md Resolved
Base automatically changed from boring/fix-api-leaks to main October 23, 2026 05:07
mosuem added 10 commits October 23, 2026 05:09
hook/build.dart gains the `fetch` (default), `checkout` and `local`
build modes, configured under `hooks.user_defines.boring`. `fetch`
downloads the library of the pinned release and verifies it against the
SHA-256 hashes in lib/src/hook_helpers/hashes.dart, falling back to the
CMake build if there is none.

When linking is enabled, the build hook provides a static library, and
hook/link.dart links a dynamic library with only the functions that the
application uses, as recorded by the Dart compiler for the @RecordUse()
bindings. In the `fetch` mode, it falls back to the prebuilt dynamic
library if linking fails, for example without a C toolchain for the
target when cross-compiling.

- The hashes are computed with package:crypto.
- test/tree_shaking_test.dart builds the example natively and for
  linux-arm64, with and without a cross toolchain. The `test` job
  installs one on each OS.
- Pushing a `boring-binaries-v<version>` tag triggers the
  boring_binaries.yaml workflow, which builds the release libraries and
  attaches them to that pre-release. tool/regenerate_hashes.dart then
  pins their SHA-256 hashes.

The hashes and download URLs still point at the 0.4.1 release of
github.com/mosuem/boring until the first release from this repository.
…libraries

tool/precompile_binaries.dart didn't set CMAKE_OSX_DEPLOYMENT_TARGET, so the
released macOS libraries only loaded on the macOS version of the build
machine or newer (the 0.4.1 dylibs have `minos 14.0`), while Dart supports
macOS 12 and Flutter's deployment target defaults to 10.15. Set it to 10.15
for x64 and 11.0 for arm64.

Link the C runtime statically on Windows (CMAKE_MSVC_RUNTIME_LIBRARY), like
the Dart SDK does, so the DLL doesn't require the Visual C++ Redistributable.
The static library's objects carry the choice, so the DLL that hook/link.dart
links inherits it.

boring_binaries.yaml checks these ABI floors (newest glibc symbol version,
macOS minos, no C runtime imports) after building, so they can't regress
unnoticed. It now also runs the `precompile` job on pull requests that change
the pipeline or the native build, which until now only ran on tag pushes, and
attests the provenance of the release libraries, which `gh attestation verify`
checks.
… assets

An unknown `buildMode` user-define, such as a typo of `checkout`, silently
selected `fetch`, so a user who wanted to build from source got downloaded
binaries instead. BuildOptions.fromDefines now throws a BuildError that lists
the valid values. The undocumented `build` alias is gone.

tool/regenerate_hashes.dart probed all 18 supported targets, 13 of which have
no prebuilt libraries, and skipped missing assets and errors, so a failed
build or a flaky download silently dropped a hash, after which `fetch`
silently compiled from source for that target. It now hashes exactly the
targets of the binaries workflow (`prebuiltTargets`) and fails without
writing anything if one is missing. The `generated` job runs it, so CI
checks that hashes.dart matches the pinned release. The release location is
one constant (`releaseAssetUrl`), version.dart is folded into hashes.dart,
and a unit test checks that hashes.dart pins exactly the assets of the
prebuilt targets.

Downloads honor the HTTPS_PROXY family of environment variables, which the
hooks runner passes through, and time out instead of hanging the build on a
stalled connection.
…s BoringSSL

doc/reusing_dart_sdk_boringssl.md was a design exploration of reusing the
BoringSSL in the Dart SDK, unrelated to the package, referring to a build
mode that doesn't exist, and shipped in the pub package. It belongs in an
issue.

The README says which targets have prebuilt libraries, that a hash mismatch
is an error rather than a fallback, and that other build modes are rejected.
pubspec.yaml explains why the package sets `buildMode: checkout` for itself,
and boring.yaml why the macOS cross toolchain isn't pinned.

hashes.dart is no longer marked linguist-generated: it pins what `fetch`
trusts, so its changes should be visible in reviews by default.
Releasing the prebuilt libraries took a hand-pushed boring-binaries-v<version>
tag, a local run of tool/regenerate_hashes.dart, and a pull request, and tied
the libraries to package versions although they only depend on the native
sources. Nothing checked that the pinned libraries were built from the
BoringSSL the package vendors, so the weekly BoringSSL update landing between
the tag and the pull request would have given `fetch` users older code than
`checkout` users.

boring_binaries.yaml now runs when a change to the native sources (src/, the
vendored BoringSSL, or the pipeline itself) lands on main, or on
workflow_dispatch on main:

- The `release` job attaches the libraries to a pre-release named after the
  build, boring-binaries-<date>-<commit>, created from the commit.
- The `pin` job writes the repository, the tag, the commit and the hashes of
  the libraries it just built to hashes.dart and opens a pull request with
  the Ecosystem Bot token, like the refresh job of boring.yaml, so that CI
  runs: the `generated` job re-hashes the release assets, and the tree-shaking
  tests download them in the `fetch` build mode.
- The `analyze` job of boring.yaml fails a release (a non -wip version) whose
  pinned libraries weren't built from the current src/, third_party/boringssl/
  and tool/REVISION, and warns otherwise.

hashes.dart therefore pins `releaseRepository`, `releaseTag` and
`sourceCommit` instead of a version, and tool/regenerate_hashes.dart takes the
tag, `--repository`, `--source-commit` (looked up with `git ls-remote` if
omitted) and `--assets`. Until the first release from this repository, which
merging this change triggers, the hashes still point at mosuem/boring v0.4.1;
the first pin pull request switches them.
Flutter apps compiled BoringSSL from source for every Android ABI and iOS
build, which takes minutes and needs CMake and Ninja. tool/precompile_binaries.dart
now cross-compiles for Android with the NDK (arm, arm64 and x64, API 21 and
later, 16 KB page alignment, stripped of the NDK's debug info) and for iOS
with Xcode (arm64, and the arm64 and x64 simulators, iOS 12 and later), and
boring_binaries.yaml builds and checks them like the desktop targets.

With exceptions disabled, libc++'s throw helpers call
std::__libcpp_verbose_abort, a function in libc++'s library, which the
libraries don't link; Android's linker rejects the undefined reference, where
the desktop toolchains let it slide. Define _LIBCPP_VERBOSE_ABORT, libc++'s
customization point, to abort directly.

tool/regenerate_hashes.dart and the unit test still require every asset of a
release built by the workflow, but accept missing ones from a release built
elsewhere, which predates these targets: the pinned mosuem/boring v0.4.1 has
no Android or iOS libraries, so those compile from source until the first
release from this repository, which merging this change triggers.
The arm64 iOS simulator only exists since iOS 14, so clang raises the
deployment target of that slice to 14.0 anyway. Set it explicitly, like
11.0 for macOS arm64, so the ABI check of the binaries workflow is exact.
…ok dependencies

tool/regenerate_hashes.dart wrote a hard-coded 2026 copyright line. It now
keeps the license header of the existing hashes.dart, whose year is the
year the file was created, like every other file's.

hook/build.dart listed itself and hashes.dart as dependencies. The hooks
runner compiles the hook with a depfile and reruns it when any of its Dart
sources change, so only pubspec.yaml remains.
…ch mode, move release docs to CONTRIBUTING.md, extract check script
@mosuem
mosuem force-pushed the boring/3-prebuilt-link-hook branch from cb1f8e0 to 444a32a Compare October 23, 2026 05:10
@mosuem
mosuem merged commit 0bed5a8 into main Oct 23, 2026
40 checks passed
@mosuem
mosuem deleted the boring/3-prebuilt-link-hook branch October 23, 2026 05:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

package:boring skip-changelog-check type-infra A repository infrastructure change or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants