When there are several Bazel actions calling golang.org/x/tools/go/packages.Load, the build time of each of those action increase as the number of such concurrent actions increases. One example of such action is in the source mode of gomock rule, which calls packages.Load through mockgen.
We observed similar slow down in a Bazel rule that calls Mockery, which also calls packages.Load.
I understand that packages.Load is generally slow because it needs to fill GOCACHE every time, which is not persist across Bazel builds. However, that doesn't explain why different actions slow each other down.
This issue may not be related to rules_go, but is more related to Bazel's action environment and the behavior of golang.org/x/tools/go/packages.Load and the underlying go list package driver. I put it here just because it is related to @jayconrod and @jmhodges's work. Feel free to refer to other venues.
What version of rules_go are you using?
v0.21.2
What version of gazelle are you using?
v0.20.0
What version of Bazel are you using?
2.1.0
Does this issue reproduce with the latest releases of all the above?
Yes
What operating system and processor architecture are you using?
macOS 10.15.3, amd64, 6-Core Intel Core i7
What did you do?
git clone [email protected]:linzhp/bazel_examples.git
cd bazel_examples
git checkout f1dbf19487a2822df0aa049509f40f6bb6be66be
bazel build --experimental_generate_json_trace_profile //sqlite:mocks
bazel build --experimental_generate_json_trace_profile //sqlite:mocks //sqlite2:mocks //sqlite3:mocks
bazel build --experimental_generate_json_trace_profile //sqlite:mocks //sqlite2:mocks //sqlite3:mocks //sqlite4:mocks //sqlite5:mocks
bazel build --experimental_generate_json_trace_profile //sqlite:mocks //sqlite2:mocks //sqlite3:mocks //sqlite4:mocks //sqlite5:mocks //sqlite6:mocks //sqlite7:mocks
What did you expect to see?
Action sqlite/mocks.go should finish in about the same time in all builds (maybe except the last one as I only have 6 cores).
What did you see instead?
Wall duration for Action sqlite/mocks.go
1 action: 35s
3 concurrent actions: 38s
5 concurrent actions: 47s
7 concurrent actions: 58s
Also the CPU usage is very low even when 7 concurrent Action sqlite/mocks.go actions are running.
When there are several Bazel actions calling
golang.org/x/tools/go/packages.Load, the build time of each of those action increase as the number of such concurrent actions increases. One example of such action is in the source mode of gomock rule, which callspackages.Loadthrough mockgen.We observed similar slow down in a Bazel rule that calls Mockery, which also calls
packages.Load.I understand that
packages.Loadis generally slow because it needs to fillGOCACHEevery time, which is not persist across Bazel builds. However, that doesn't explain why different actions slow each other down.This issue may not be related to rules_go, but is more related to Bazel's action environment and the behavior of
golang.org/x/tools/go/packages.Loadand the underlyinggo listpackage driver. I put it here just because it is related to @jayconrod and @jmhodges's work. Feel free to refer to other venues.What version of rules_go are you using?
v0.21.2
What version of gazelle are you using?
v0.20.0
What version of Bazel are you using?
2.1.0
Does this issue reproduce with the latest releases of all the above?
Yes
What operating system and processor architecture are you using?
macOS 10.15.3, amd64, 6-Core Intel Core i7
What did you do?
What did you expect to see?
Action sqlite/mocks.goshould finish in about the same time in all builds (maybe except the last one as I only have 6 cores).What did you see instead?
Wall duration for
Action sqlite/mocks.go1 action: 35s
3 concurrent actions: 38s
5 concurrent actions: 47s
7 concurrent actions: 58s
Also the CPU usage is very low even when 7 concurrent
Action sqlite/mocks.goactions are running.