Remote: Limit max number of gRPC connections by --remote_max_connecti… (cherry pick for 4.2.2)#14318
Merged
meteorcloudy merged 1 commit intobazelbuild:release-4.2.2-patchesfrom Nov 24, 2021
Conversation
…ons. `--remote_max_connections` is only applied to HTTP remote cache. This PR makes it apply to gRPC cache/executor as well. Note that `--remote_max_connections` limits the number of concurrent connections. For HTTP remote cache, one connection could handle one request at one time. For gRPC remote cache/executor, one connection could handle 100+ concurrent requests. So the default value `100` means we could make up to `100` concurrent requests for HTTP remote cache or `10000+` concurrent requests for gRPC remote cache/executor. Fixes: bazelbuild#14178. Closes bazelbuild#14202. PiperOrigin-RevId: 410249542 (cherry picked from commit 8d5973d)
Contributor
Author
|
I took a look to the failing tests in the CI but I do not see how this can be related. It seems some issue with a test regarding floating point. Any way to trigger the same tests for the branch release-4.2.2-patches to see if the issue also appears there? |
Member
|
I'm retrigging the test again 🤔 |
Contributor
Author
|
It looks like the issue is also present in the branch https://buildkite.com/bazel/bazel-bazel/builds?branch=release-4.2.2-patches, but indeed there are more failures there 🤔 |
Member
|
Succeeded at 4th try 😅 |
bazel-io
pushed a commit
that referenced
this pull request
Dec 2, 2021
Baseline: 37a429a Cherry picks: + a689d67: Use getRunfilesPath for run_under executable path generation. getRootRelativePath doesn't return a valid runfiles path for external source files anymore after the recent external source root change. Also, it won't work for external labels either once the --nolegacy_external_runfiles becomes default. This fixes issue #12545. + d90ec67: Fix NPE when coveragerunner is not set on the toolchain. + 8555789: Fix the classic query package-loading cutoff optimization with external workspaces. + f28f697: Update turbine + 69b4362: Update turbine + 4d4ab50: Revert "Update turbine" + 89b9a04: Update turbine + d31f6df: Update turbine version + 57672ac: Update turbine + bef4bbb: Update turbine + d113d74: Update turbine + 1489f0f: Support Scala3 .tasty files + 0d2d95c: Update to java_tools javac11 release 10.5 (#12647) + a9419f3: Fix common prefix for instrumentation filter + 84fadcf: Fix builds for filegroup targets with incompatible dependencies + e43825d: Revert "Remove --incompatible_blacklisted_protos_requires_proto_info" + 082d58d: Transform roots along with paths during output deletion. + e8835c1: AttributeContainer.Large now handles more than 127 attributes. + e1e8734: Add an env attribute to all test and binary rule classes + a87d7ed: Take no action to prefetch empty artifacts. + 3e969ff: Fix a couple of bugs with Incompatible Target Skipping + e667082: Pass --host_action_env to host options hostActionEnvironment attribute + 07400c0: Add --{no,}autodetect_server_javabase. + c833660: Only treat "env" and "env_inherit" attrs specially for native rules + 6a60b30: Fix coverage support when using default_java_toolchain. (#12801) + 4158a6f: Revert JacocoCoverage target to remote_java_tools_java_import and add a new target for remore_java_tools_filegroup. (#12813) + f6d30cf: Add windows_msvc back to conditions in bazel_tools. + 6b33bdb: Release 4.0.0 (2021-01-21) + 8811e27: Fix error message from getPrerequisites to not print internal details. + 27e15ad: Clean up ConfiguredTargetValueAccessor and ConfiguredTargetAccessor + e87feb8: Move getConfigConditions into ConfiguredTarget. + 34d9823: Change ConfiguredTargetQuery to use KeyedConfiguredTarget as a value. + 079bb7d: Clean up old dependencies that are unused since 34d98234324da83e93ba0d 5ef5702880d5ac7c5c. + e03cb63: Update bazelbuild/platforms to a current release. - Roll forward 0a4533420a3de467fd211d 7f925cf88e0cd5b76a with kythe fix. + 2eb1bf5: Update docs and tests to use the @platforms//:incompatible constraint + c71697c: Clarify test_suite behaviour in the Platforms docs + dfb70ea: Enable toolchain resolution for filegroup targets. + 24d0864: PlatformProviderUtils should ignore targets that don't have the needed + ba60c0b: ijar: fix manifest sections handling + 58bb42a: Revert "Switch to -fdebug-compilation-dir" + f28f697: Update turbine + 69b4362: Update turbine + 4d4ab50: Revert "Update turbine" + 89b9a04: Update turbine + d31f6df: Update turbine version + 57672ac: Update turbine + bef4bbb: Update turbine + d113d74: Update turbine + ad241fb: Allow cquery to filter out incompatible targets + 1782f0a: Patch grpc to fix cares selecting the wrong source when building for darwin_arm64 cpu. + 8f7bc2f: [1/3] Bump grpc to 1.33.1 to fix corruption when downloading CAS blobs + 848a517: [2/3] Bump grpc to 1.33.1 to fix corruption when downloading CAS blobs + 9b30172: [3/3] Bump grpc to 1.33.1 to fix corruption when downloading CAS blobs + 1e258d2: Allow exec groups to inherit from the rule or other exec groups. + d067669: Support execution constraints per exec group + f1e0d34: Clean up RuleContext to use a Table instead of a Map of Maps. + 8186fbb: Documentation for #13110 + 321fe3b: Prevent --repo_env from triggering unnecessary fetches + 3ebf658: Prevent a crash when using --repo_env=VAR without a value + 913a985: Report digest of failed uploads + 5122617: Status error presentation with details + 9a70805: Fix double shutdown of BuildEventArtifactUploader when BES+File output enabled. + 325eb95: Add rxjava3 to third_party + ceaac96: remote: set executable bit of an input file based on its real value + 5b786da: Remote: correctly implement equals and hashCode. + 4864850: Fixed an error that bazel binary is not executable when testing with remote execution. + bc54c64: Remote: Use parameters instead of thread-local storage to provide tracing metadata. + 92955e6: Remote: Use parameters instead of thread-local storage to provide tracing metadata. (Part 2) + 75bd1ff: Remote: Use parameters instead of thread-local storage to provide tracing metadata. (Part 3) + 37ee252: Remote: Use parameters instead of thread-local storage to provide tracing metadata. (Part 4) + 71e35b1: Remote: Use parameters instead of thread-local storage to provide tracing metadata. (Part 5) + 32fc451: Write/QueryWriteStatus logging refinement/addition + 97963c5: Remote: gRPC load balancing. (Part 1) + e2b9a42: Remote: gRPC load balancing. (Part 2) + 6667ad7: Remote: gRPC load balancing. (Part 3) + 7c081eb: Remote: gRPC load balancing. (Part 4) + 17afbe4: Implement getMessage for BulkTransferException + a6293b3: Remote: gRPC load balancing. (Part 5) + 7a62c2d: Remote: Add interoperability between Rx and ListenableFuture. + 1fcb18a: Update to latest remote-execution proto + dad9630: Set Platform on Action not just Command + 6c5a3ee: Remote: Add AsyncTaskCache which is used to deduplicate task executions and cache the results. + 9d0c732: Remote: Use AsyncTaskCache inside RemoteActionInputFetcher. + f54fe07: Add --experimental_repository_disable_download to allow users disable download for external repos + b243584: Report errors parsing rewriter config file + 63bc1c7: Downloader rewriter config has all_blocked_message + 495ac92: Allow UrlRewriter to change protocol, i.e. https->http, and http->https + 8dbbde0: Allow overriding the hostname and instance name in bytestream:// URIs + 0881c80: Don't set requestId on non-multiplex requests. + e3b7e17: When generating a symlink in _virtual_includes, add the original header to the 'allowed to use' set too + f8f66f3: Make SimpleLogHandler not swallow interrupts. + f8606e5: linux-sandbox: don't assume -lrt, -D__STDC_FORMAT_MACROS + dac0d40: Improve "Common Attributes" section + a607d9d: Never create more than one process per WorkerMultiplexer. + 80c03ef: Move sending requests and reading responses for multiplex workers into separate subthreads. + 003cfcd: Allow use of JSON protocol in multiplex workers. + 308bce3: Actively kill off still-active workers when stopping work on interrupt. + 8959dff: Add sanitizer support to Apple platforms + 32f16e9: Fix a Google-internal broken link. + c9e2be5: Add SHA-1 to subresource integrity format for download() checksums + 3b3e642: Remove fallback strategy support for workers, add flag for it in sandbox. + 3457f2a: Update to java_tools javac11 10.6 (#13245) + 4928295: Allow .S files in C++ Starlark cc_common.compile. + 1b18d65: Automatic code cleanup. + b5d6c38: Change short output of worker type to have the same logic as the worker creation for sandboxing vs. multiplex. + e7a0a71: More properly destroy workers on interrupt. + 7056711: Make WorkRequestHandler do a GC after some amount of CPU time has been used on requests. For Bazel and Blaze, defaults to 10s based on benchmarking. + 596653d: Allow tree artifacts to be source or header inputs to cc_common.compile() + 055c93d: Switch to path autocompletion after -- for bazel run commands. + 807f2a1: Fix Incompatible Target Skipping for test args + 9a5cd85: Fix order of build request id and command id + 706f5ac: Fix bazel crash when passing config_setting to target_compatible_with + 61da1d2: Support multiple --bazelrc on command line + 5593358: Update ConfiguredTargetFunction.computeUnloadedToolchainContexts to + 662cf54: Remote: Fix an issue that a failed action could lead to RuntimeException caused by InterruptedException thrown when acquiring gRPC connections. #13239 + a3a1763: Pass more `--add-exports=` flags + d2b9428: Remote: Fixed a bug that remote cache is missed due to executable bit is changed + 616dc26: Fix Bazel Coverage with C++ to work with Remote Execution + 5f40d12: Fix external_path_test with newer Xcode versions. + b416193: Allow using embedded tools in sandboxed spawn runners. + eb762d4: Fix racy write of temporary files while staging virtual inputs for the sandbox. + f31e867: Update platforms_test to not rely on filegroup not using toolchain + 13031e5: Update SkyframeTests to not rely on filegroup not using toolchain + 1165182: Update ConfigurableAttributesTest to not rely on filegroup not using … + 4b68532: Make WorkerExecRoot not be a subclass of SandboxedSpawn. + 31db460: Make WorkerExecRoot not be re-created on each createFileSystem() call. Preparation for holding a map of existing links, but also just nicer. + a2cc046: Start the file existence check traversal from the execroot base instead of execroot so that external repo files at "<execroot>/../<path>" are correctly handled when the sibling repository layout is enabled. + b048282: Use readdir for cleanExisting in WorkerExecRoot. + 270f00d: Add native support for Apple Silicon + 8e56b94: Explicitly state that embedding macOS OpenJDK is for x86_64 + 09c621e: Remote: Fix a race that AsyncTaskCache#Execution could be reused after disposed which results in CancellationException("disposed") propagated to downstream. + 0299cd7: Remove wrapped_clang params files after use + 47edc57: Silence swiftmodule timestamp warnings + f6e1074: Remote: Use shutdownNow() instead of shutdown() in ChannelConnection#close() as a workaround to a gRPC bug. + 71be4ea: And mnemonic and label to remote metadata + 615e1b1: Change `set -x` in coverage to be set by var + fe4daea: Bump minimal JDK install base maximum size from 290 to 295 MB. + 4840a68: Remote: Check the return value of ActionOwner.getLabel() since it could be `null`. + 14abe4f: Allow `DiffAwareness` to share precomputed information about the workspace and propagate it to the `WorkspaceStatusAction`. + 082d987: Implement available() method for Windows subprocesses. + c2bdd03: Move --repo_env to common options + e09f274: Revert "Documentation for #13110" + a165baa: Revert "Clean up RuleContext to use a Table instead of a Map of Maps." + 51fb9e1: Revert "Support execution constraints per exec group" + cb6e5c2: Revert "Allow exec groups to inherit from the rule or other exec groups." + 2ac6581: Release 4.1.0 (2021-05-21) + 7a0f36e: Change gceMachineType of highcpu platform from n1-highcpu-32 to e2-highcpu-32 + 19491a9: Fix #10127: Remove Python 2 dependency from tools/android. + 80c59de: fix main repo starlark options parsing - now flags passed on the command line as --@main_workspace//flag and --//flag will both parse to --//flag. Before this CL, the former maintained its workspace prefix and we would get different entries for these two formats. + 451b296: Update threshold for long path shortening to be MAX_PATH - 4 + 671e048: Force source files to be readable before copying them from sandbox. + 6080c1e: Let workers finish lost races without delaying dynamic execution. + ee738da: Fix label_flag and label_setting to not have a dependency on the default + 74de0ba: Java coverage: fix handling of external files + 48eee8b: [Bazel] Fix mobile-install for python2 + 763dd0c: Add `stub_shebang` to `py_runtime` + b2231c5: Move use of legacy sandbox -> local fallback to only be used after
…ons.
--remote_max_connectionsis only applied to HTTP remote cache. This PR makes it apply to gRPC cache/executor as well.Note that
--remote_max_connectionslimits the number of concurrent connections. For HTTP remote cache, one connection could handle one request at one time. For gRPC remote cache/executor, one connection could handle 100+ concurrent requests. So the default value100means we could make up to100concurrent requests for HTTP remote cache or10000+concurrent requests for gRPC remote cache/executor.Fixes: #14178.
Closes #14202.
PiperOrigin-RevId: 410249542
(cherry picked from commit 8d5973d)