Configure remote builds to use EngFlow#11887
Conversation
This allows us to use EngFlow's remote build grid to execute our builds. Once we hook this up into our CI actions, this will lead to significantly faster CI runs.
p0deje
left a comment
There was a problem hiding this comment.
I see there is a whole new common/remote-build directory with many autogenerated files. Is there any way to shrink those? For example, I assumed we don't have to manually configure C++ toolchains because I thought it's built-in to Bazel.
|
The files are auto-generated by (effectively, if not in reality) running bazel on the remote machine and then copying the generated files into the repo. It's not a great idea to modify them. We don't have a hermetic C++ toolchain, so we still need to manage that ourselves. Our remote build images have a C++ toolchain installed, so we need to tell Bazel about it. Because the build is remote, we can't figure this stuff out at run time (since we need to know everything before sending the actions to the remote machine), and this is the way to configure it. |
Got it. I think as long as it's documented how to re-generate these files should we need - we are fine.
Ok, so Bazel doesn't ship with a hermetic C++ toolchain. Would using something like https://github.com/uber/hermetic_cc_toolchain help us? Probably not now, but eventually. |
I spent an ungodly amount of time attempting to get the zig-cc toolchain working and gave up. If all we wanted to do was support Linux, it'd be easier, but we want to support the matrix of compiling for and on Windows, Mac OS, and Linux. For that we need various sysroots, and the legality of distributing these is unknown. That makes it a really difficult problem to solve. It's also not something that we can solve in this PR :) |
Codecov ReportPatch coverage:
📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more Additional details and impacted files@@ Coverage Diff @@
## trunk #11887 +/- ##
==========================================
+ Coverage 54.87% 54.89% +0.01%
==========================================
Files 85 86 +1
Lines 5698 5733 +35
Branches 230 233 +3
==========================================
+ Hits 3127 3147 +20
- Misses 2341 2353 +12
- Partials 230 233 +3
☔ View full report in Codecov by Sentry. |
This allows us to use EngFlow's remote build grid to execute our builds. Once we hook this up into our CI actions, this will lead to significantly faster CI runs.