Cherry-pick proto_lang_toolchain Starlarkfication and proto_common module #15854
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
comius
force-pushed
the
cherry-pick-proto_lang_toolchain
branch
5 times, most recently
from
5cfba19 to
1650633
Compare
Renamed StrictProtoDepsViolationMessage to ProtoConstants and added protoc label there. PiperOrigin-RevId: 409142099
…lify ProtoCompileActionBuilder. `$(OUT)` placeholder is replaced with `%s` so it can be directly used in addFormatted. This simplifies construction of proto compile action and makes it possible for Starlark version to have the same performance. github shows no uses of the placeholder: https://github.com/search?q=%22%24%28PLUGIN_OUT%29%22&type=Repositories PiperOrigin-RevId: 412286743
This makes it possible to pair it with command_line attribute which could contain dependent data,
for example:
```
proto_lang_toolchain(
name = "j2objc_proto_toolchain",
blacklisted_protos = [],
command_line = "--PLUGIN_j2objc_out=file_dir_mapping,generate_class_mappings:$(OUT)",
+ plugin_format_flag = "--plugin=protoc-gen-PLUGIN_j2objc=%s",
plugin = "//third_party/java/j2objc:proto_plugin",
runtime = "//third_party/java/j2objc:proto_runtime",
visibility = ["//visibility:public"],
)
```
It also retains reference to the flag on proto_lang_toolchain rule and so doesn't cause a memory regression when proto_lang_libraries are Starlarkyfied.
PiperOrigin-RevId: 412287195
This is needed for the proto_common.generate_code function. This change doesn't modify any of proto_lang_toolchain public attributes. PiperOrigin-RevId: 437713619
…ttributes. This is needed for proto_common.generate_sources function. Two public attributes are added to proto_lang_toolchain rule. Both have defaults, so no immediate migration of targets is needed. PiperOrigin-RevId: 437714094
Added StarlarkProtoLangToolchainTest which uses starlarkified rule for verification. I've deleted blacklisted_protos and forbidden_protos since they are not needed anymore. PiperOrigin-RevId: 444223497
…vider and flip proto_lang_toolchain rule NEW: I've moved ProtoLangToolchainProvider from providers.bzl file to proto_common.bzl file since proto_common doesn't allow loading other components. I've also deleted providers.bzl file since we don't need it anymore. Automated rollback of commit 5a6b1a8. *** Reason for rollback *** Good to submit since the blocking error has been resolved. *** Original change description *** Automated rollback of commit ae349e9. *** Reason for rollback *** This CL breaks proto_common.bzl file which seems that can't load providers.bzl file. *** Original change description *** Export ProtoLangToolchainInfo provider and flip proto_lang_toolchain rule I’ve exported ProtoLangToolchainInfo provider from it’s native class by adding two new functions: one that’s creating starlark provider (create function), and the other that’s wrapping the starlark provider as a nat *** PiperOrigin-RevId: 446401388
…in proto_lang_toolchain. The problem with latter is, that transitive_source can contain "renamed" files (in _virtual_includes subdirectory), which doesn't work for the detection that needs original files (ProtoSource.original_source_file). PiperOrigin-RevId: 446924924
It's the same behaviour in native code. PiperOrigin-RevId: 446988764
PiperOrigin-RevId: 446995789
Added a wrapper for proto_lang_toolchain in order to have two implementations - one with public proto_compiler attribute, and the other one with the private one. Restructured proto_lang_toolchain rules' implementation - merged two rule's definitions into one. PiperOrigin-RevId: 447016335
comius
force-pushed
the
cherry-pick-proto_lang_toolchain
branch
from
1650633 to
421cfec
Compare
PiperOrigin-RevId: 413105213
Added optional `resource_set` parameter to `run` and `run_shell` in StarlarkActionApi. `resource_set` is `StarlarkCallable` object that returns dict with resource set (cpu, memory, local_test). PiperOrigin-RevId: 415224490
This will make migration to the new call easier (because we don't have progress_message set yet on the proto_lang_toolchain rules). Design doc: https://docs.google.com/document/d/1dY_jfRvnH8SjRXGIfg8av-vquyWsvIZydXJOywvaR1A/edit PiperOrigin-RevId: 440098602
PiperOrigin-RevId: 445149402
Adding it will make it possible to migrate the uses from proto_library_target to proto_info. When the migration is done proto_library_target will be removed. The cost of changing this now is still low, because it's not yet released/used in Bazel. PiperOrigin-RevId: 455616355 Change-Id: Ieb0f03b0600e1f90b72a61f90420675075c79a9e
…nerated_files. PiperOrigin-RevId: 456475071 Change-Id: I2882d80cd4f7fdd9b8dcba3347930eaf1f194d0a
…al_should_generate_code PiperOrigin-RevId: 460162832 Change-Id: I57a6fa4c6e6c9618cf9edb8518e17b46fc90be9f
comius
changed the title
Contributor
Author
Contributor
Author
|
This is a "cherry-pick" of proto_common module into Bazel 5.3.0. Sorry for the XXL PR. |
copybara-service bot
pushed a commit
that referenced
this pull request
Aug 22, 2022
Baseline: 8d66a41 Cherry picks: + becd149: Remote: Cache merkle trees + d7628e1: Update DEFAULT_IOS_CPU for M1 arm64 simulator support + 80c56ff: Compile Apple tools as fat binaries if possible + 3c09f34: Add protobuf as a well known module + 3a5b360: Remote: Merge target-level exec_properties with --remote_default_exec_properties + 917e15e: Add -no_uuid for hermetic macOS toolchain setup + f5cf8b0: Remote: Fixes an issue when --experimental_remote_cache_async encounter flaky tests. + 77a002c: Remove DigestUtils.getDigestInExclusiveMode() now that SsdModule has … + 557a7e7: Fixes for the Starlark transition hash computation (#14251) + 34c7146: Do location expansion in copts of objc_library + 50274a9: [5.x] Remote: Add support for compression on gRPC cache (#14277) + 61bf2e5: Automated rollback of commit 34c7146. + 79888fe: Silence a zstd-jni GCC warning. + 063b5c9: Remote: Limit max number of gRPC connections by --remote_max_connections. + fd727ec: Do location expansion in copts of objc_library + 23d0969: Fix _is_shared_library_extension_valid + 5cf1d6e: Remove merging of java_outputs in JavaPluginInfo. + cea5f4f: Cherrypick Bzlmod documentation (#14301) + 227e49e: Format work requests according to ndjson spec + ae0a6c9: Enable user_link_flags_feature for macosx cc_toolchain_config + 8c2c78c: Remote: Use Action's salt field to differentiate cache across workspaces. + f948989: [5.x] Remote: Fix "file not found" error when remote cache is changed from enabled to disabled. (#14321) + 3069ac4: Delete marker file before fetching an external repository + c05c626: Remote: Fix file counting in merkletree.DirectoryTreeBuilder + d84f799: Fix remote spawn tests for remote_merkle_tree_cache=true + 59e16e9: Show skipped tests as a warning + 76b3c24: Build xcode-locator as a universal binary + aa52f2d: Exit collect_coverage.sh early if LCOV_MERGER is not set. + 4256d46: Automated rollback of commit d84f799. + dce2435: [apple] fix issues compiling C in objc_library for watchos/armv7k + bfc2413: 5.x: Remote: Ignore blobs referenced in BEP if the generating action cannot be cached remotely. (#14389) + 5aef53a: Remote: Don't blocking-get when acquiring gRPC connections. (#14420) + 005361c: Disable IncludeValidation for ObjC in bazel + d703b7b: Update java_tools v11.6 + 90965b0: Stop remote blob upload if upload is complete. (#14467) + dc59d9e: [5.x] Make remote BES uploader better (#14472) + 2edab73: Avoid too verbose warnings in terminal when cache issues + 1160485: Rename --project_id to --bes_instance_name + c63d9ec: Automated rollback of commit bfdfa6e. + b341802: [apple] support watchos_arm64 in toolchain + 43bcf80: Disable implicitly collecting baseline coverage for toolchain targets. + 302971e: Automated rollback of commit 7d09b4a. + 6200202: Bzlmod: Starlarkify default attr values for TypeCheckedTags + 38117d4: Fix build after rc4 cherrypicks (#14581) + 41feb61: Release 5.0.0 (2022-01-19) + 486d153: Find runfiles in directories that are themselves runfiles + 0de7bb9: Don't resolve symlinks for --sandbox_base + 8b60c90: Remove uses of -lstdc++ on darwin + 60f757c: Allow Label instances as keys in select (#14755) + 3836ad0: Remote: Only waits for background tasks from remote execution. + 8734ccf: Add the default solib dir to the rpath for cc_imports with transitions + 9e16a64: Flip --experimental_worker_allow_json_protocol + fce7ea8: Fix `ctx.fragments.apple.single_arch_cpu` returning incorrect cpu for tools when host cpu and exec cpu are different + 0c1d09e: Propagate --experimental_cc_implementation_deps to host config + 1c3a245: Support select() on constraint_value for aliases. + 67a133b: Improve documentation for select() + 5356fed: Cherrypicks for experimental cc_shared_library (#14773) + ffdd633: [apple] support tvos_sim_arm64 in toolchain (#14779) + a58ddea: Cherry pick win arm64 (#14794) + dc41a20: [5.1.0] cherrypick subpackages support (#14780) + 86e2db7: Add a helper method for rules to depend on the cpp toolchain type. + 6990c02: UrlRewriter should be able to load credentials from .netrc (#14834) + 32d1606: Add "arch" struct field to repository_os + 2cfdcea: [5.x] bzlmod: Add support for WORKSPACE.bzlmod (#14813) + c2ddbd1: Ignore missing include directory in JDK distribution. + 16de035: Fix bazel coverage false negative + 0c74741: Remote: Postpone the block waiting in `afterCommand` to `BlockWaitingModule` (#14833) + 3297d92: Switch to `ProcessHandle` for getting the PID (#14842) + a987b98: Fix uses of std++ on bsd + d184e48: Remote: handle early return of compressed blobs uploads + 0b09e9e: Add removeprefix/removesuffix to Starlark strings + d42ab0c: Fix default CPU for macOS and iOS (#14923) + cd24f39: Add paramfile support for def_parser, since in rare cases on Windows command line character limit was reached. + 0b1beef: Normalize rpath entries to guard against missing default solib dir + 24e8242: Fix aggressive params file assumption + c45838b: Fix precompiled libs not in runfiles of cc_shared_library (#14943) + 764614e: Bzlmod: Allow multiple `use_extension`s on the same extension (#14945) + fa761f8: Fix typo in `apple_common.platform` docs + f7d8288: Yield a Proxy for addresses without protocol + 8cefb8b: Avoid merging URLs in HttpUtils + b480480: Make protocOpts() public. (#14952) + 113eaca: Do not hide BulkTransferException messages when there were more than one exception + b1bf9d6: merkle_tree_cache: change default size to 1000 + f15e0c7: Add --experimental_repository_cache_urls_as_default_canonical_id to help detect broken repository URLs (#14989) + f421474: Expose the logic to read user netrc file + b858ec3: Correct cpu and os values of `local_config_cc_toolchains` targets + 5e79972: Expose CoverageOutputGenerator on a Fragment (#14997) + 78f0311: Correct error runfiles cc_shared_library (#14998) + 7937dd1: [5.1] Adding Starlark dependencies to the package //external (#14991) + a73aa12: Remote: Fix crashes with InterruptedException when using http cache. + f8707c0: Account for interface libraries in cc_shared_library + a570f5f: Fix coverage runfiles directory issue + 95de355: Do not validate input-only settings in transitions (#15048) + 71747cc: Filter out system headers on macOS. + cb6500a: Update Bazel bootstrap documentation and remove obsolete flags. (#15065) + 4c031d1: [5.1] Undocument --bes_best_effort (#15066) + 267142f: Fix conflicting actions error when specifying --host_macos_minimum_os (#15068) + f192362: [5.1] Remote: Action should not be successful and cached if outputs were not created (#15071) + 00d74ff: Support decompressing zstd tar archives for repository rules. + f585783: Remote: Don't check TreeArtifact output + efb2b80: osx_cc_wrapper: Only expand existing response files + c771c43: Remote: Fix crashes by InterruptedException when dynamic execution is enabled. (#15091) + 3785677: Use python3 on macOS + 815d9e4: Release 5.1.0 (2022-03-24) + 1fbb69e: Prepare 5.1.1 release + df153df: Fix CODEOWNERS syntax + 2b92a31: Remote: Don't check declared outputs for failed action + b47aa71: Upgrade abseil version to the latest + c49c45d: Revert default export all symbols on Windows + 7d3fb99: Support ZIP files with total number of disks = 0 + 0f5dc11: Release 5.1.1 (2022-04-08) + 2422cfb: Update CODEOWNERS + bbcff18: [5.2.0] Update java_tools 11.7.1 (#15231) + 9c98120: Add support for .ar archives (and .deb files) + d3435b0: Seperate GetSelfPath implementation for Blaze and Bazel + c94572b: Include jdk.crypto.mscapi in minimized Windows embedded JDK + 299022c: remote: Proactively close the ZstdInputStream in ZstdDecompressingOutputStream. + 2770799: Collect coverage from cc_binary data deps of java_test + 3442179: Configure Apple crosstool to return a complete target triple that includes minimum OS version and target environment + bb6f1a7: Collect C++ lcov coverage if runtime object not in runfiles + dbb6e99: Fixing dependencies of //external package + f0213bb: [5.2] Upgrade Google Auth Version (#15383) + a1a74c9: Fix chocolatey package - docsUrl must not 404 (#15395) + fe644be: Fix cache leak when applying transitions when only a rule's attributes change. + ad74d52: Fix checking remote cache for omitted files in buildevent file (#15405) + ac21910: fix(bzlmod): throw on json parse exception + 3d85b88: Add a flag to expose undeclared test outputs in unzipped form. (#15431) + abd7a9f: Remove -U_FORTIFY_SOURCE when thin_lto is enabled (#15433) + 53b9cb8: Catch NumberFormatException while trying to parse thread id. + 19740b5: Improve the --sandbox_debug error message + 0a2a43f: Set keywords on appropriate lifecycle events. + 394ddb8: Record additional profiling information for remotely executed actions. + 652b48e: Fix downloading remote execution output files inside output dirs. (#15444) + 73f1ecb: Fix android emulator darwin_arm64 select + 2649c7c: Fix --use_top_level_targets_for_symlinks with aliases (#15446) + fa1081c: Filter libtool warning about table of contents + 26f8783: Unify sandbox/remote handling of empty TreeArtifact inputs (#15449) + 6b21b77: Revert "Fixes incorrect install names on darwin platforms" + e133e66: config doesn't error on duplicate `--define` values (#15473) + 84d5917: Collect coverage from cc_binary data deps of py_test (#15298) + 519d2da: SolibSymlinkAction does not need exec platform or properties + 6e54699: Let Starlark tests inherit env variables (#15217) + 9610ae8: Update PythonZipper action to use CommandLineItem.CapturingMapFn + 2f1ff6f: Make `coverage --combined_report=lcov` skip incompatible tests + 9fad5a3: Disable ReturnValueIgnored checks to unblock java_tools release + 0120118: Bump the limit of Bazel install base size (#15585) + 668805a: Upgrade zlib to 1.2.12 + 4d900ce: [5.2] Remote: Fix a bug that outputs of actions tagged with no-remote are u... (#15453) + b703cb9: Add feature to produce serialized diagnostics files (#15600) + 2e8458b: Release 5.2.0 (2022-06-07) + 536f8d9: Fix fail message construction in cc_shared_library + 2d42925: Define cc-compiler-darwin in Xcode toolchain + a1d7d1f: Fix alwayslink in objc_import + d273cb6: Unify URL/URLs parameter code across http_archive, http_file, http_jar + fea32be: Preserve --experimental_allow_unresolved_symlinks in exec cfg + e4bc370: Ck/cherry pick cc shared library (#15754) + dbdfa07: Let Starlark executable rules specify their environment (#15766) + e2a6a2b: Fix string formatting when java_home path is missing. + d54a288: Optionally enable LLVM profile continuous mode + ad17b44: Print remote execution message when the action times out (#15772) + 240e3d1: Add missing line to cherrypick e4bc370 (#15784) + 804b474: Replace strdupa with strdup + 62be9ea: Bzlmod: Better canonical repo names for modules with overrides (#15793) + d4663a1: Add repo env test (#15768) + 594962c: Add is_root struct field to bazel_module (#15815) + 3dd2b93: Fix null pointer crash with `bazel coverage` on only incompatible tests + 4175018: Add util for finding credential helper to use + 3ea9eb2: Merge ManifestMergerAction-related commits into release-5.3.0 (#15824) + 64571a4: Ck/cherrypick 15669 (#15788) + 1404651: Create output directories for remote execution (#15818) + ae523f8: Use tree artifacts in bootclasspath rule + 37f181c: [credentialhelper] Add types to communicate with the subprocess + 06ca634: Add a flag to force Bazel to download certain artifacts when using --remote_download_minimal (#15870) + d35f923: RemoteExecutionService: fix outputs not being uploaded + 78af34f: Cherry-pick proto_lang_toolchain Starlarkfication and proto_common module (#15854) + afb434d: Fix behavior of `print()` in module extensions + 6714c30: [credentialhelper] Implement invoking credential helper as subprocess + 0f05904: Add register_{execution_platforms,toolchains} directives to MODULE.bazel files (#15852) + 33516e2: [remote] Improve .netrc test in RemoteModuleTest + aa2a1f3: Fix ZipDecompressor windows 0x80 (file attribute normal) + 30f16e5: Replace uses of `cfg = "host"` with `cfg = "exec"` (#15922) + 2a8d0ad: target pattern file: allow comments + 6f73205: Add factory for creating paths relative to well-known roots (#15931) + 32cc8e6: Update CODEOWNERS (#15910) + 63bc14b: Implement native analysis_test call. (#15940) + 4df77f7: Increase osx_cc_configure timeouts + cdf01a3: Allow string_list flags to be set via repeated flag uses + 05e758d: [credentialhelper] Add parser for flag syntax (#15929) + e4ee344: Docs should mention the new no-remote-cache-upload tag (#15965) + 96d23d3: Add netrc support to --bes_backend (#15970) + c5bc34e: Add CommandLinePathFactory to CommandEnvironment (#15971) + 508f185: Move newCredentialHelperProvider into GoogleAuthUtils (#15973) + 14c944a: Wire up credential helper to command-line flag(s) (#15976) + 04c373b: Add `--output=files` mode to cquery (#15979) + edfe2a1: Make cpp assembly file extensions case sensitive again + 4ae8538: Prevent aspects from executing on incompatible targets (#15984) + f440f8e: Remote: Fix performance regression in "upload missing inputs". (#15998) + 0109031: Updated Codeowners file (#16032) + 6102d33: Propagate the error message when a credential helper fails. (#16030) + a8dacc7: Migrate legacy desugar wrapper to new rlocation() (#16025) + 11368be: Correctly report errors thrown by CommandLinePathFactory#create. + 82452c7: Fix an issue that `incompatible_remote_build_event_upload_respect_no_… (#16045) + e745468: Fix rpath for binaries in external repositories (#16079) + 83041b1: Refactor combined cache. (#16110) + c62496f: C++: Add compound error linked statically but not exported (#16113) Important changes: - alias() can now select() directly on constraint_value() Fixes #13047. Closes #14310. - Fixed an issue where Bazel could erroneously report a test passes in coverage mode without actually running the test. - Make protocOpts() publicly accessible. - Add coverage configuration fragment, used to expose output_generator label. - Bazel now no longer includes system headers on macOS in coverage reports (#14969). Closes #14971. - Starlark test rules can use the new inherited_environment parameter of testing.TestEnvironment to specify environment variables whose values should be inherited from the shell environment. Closes #14849. - none RELNOTES:none - Enable merging permissions during Android manifest merging with the --merge_android_manifest_permissions flag. - Added new register_{execution_platforms,toolchains} directives to the MODULE.bazel file, to replace the {execution_platforms,toolchains}_to_register attributes on the module() directive. - Add support for fetching RPC credentials from credential helper. Progress on #15856 Closes #15947. - `cquery`'s new output mode [`--output=files`](https://bazel.build/docs/cquery#files-output) lists the output files of the targets matching the query. It takes the current value of `--output_groups` into account. Closes #15552. - Fix for desugaring failure on Bazel+Android+Windows build scenario. This release contains contributions from many people at Google, as well as amberdixon, Andreas Fuchs, Benjamin Peterson, Brentley Jones, Chenchu Kolli, Dan Fleming, Danny Wolf, Emil Kattainen, Fabian Meumertzheim, Gowroji Sunil, hvadehra, Juh-Roch, Keith Smiley, kshyanashree, Niyas Sait, Noa Resare, Oliver Eikemeier, oquenchil, Peter Mounce, Philipp Schrader, Thi Doãn, Xùdōng Yáng, Yannic, Zhongpeng Lin.
ShreeM01
pushed a commit
that referenced
this pull request
Aug 22, 2022
Baseline: 8d66a41 Cherry picks: + becd149: Remote: Cache merkle trees + d7628e1: Update DEFAULT_IOS_CPU for M1 arm64 simulator support + 80c56ff: Compile Apple tools as fat binaries if possible + 3c09f34: Add protobuf as a well known module + 3a5b360: Remote: Merge target-level exec_properties with --remote_default_exec_properties + 917e15e: Add -no_uuid for hermetic macOS toolchain setup + f5cf8b0: Remote: Fixes an issue when --experimental_remote_cache_async encounter flaky tests. + 77a002c: Remove DigestUtils.getDigestInExclusiveMode() now that SsdModule has … + 557a7e7: Fixes for the Starlark transition hash computation (#14251) + 34c7146: Do location expansion in copts of objc_library + 50274a9: [5.x] Remote: Add support for compression on gRPC cache (#14277) + 61bf2e5: Automated rollback of commit 34c7146. + 79888fe: Silence a zstd-jni GCC warning. + 063b5c9: Remote: Limit max number of gRPC connections by --remote_max_connections. + fd727ec: Do location expansion in copts of objc_library + 23d0969: Fix _is_shared_library_extension_valid + 5cf1d6e: Remove merging of java_outputs in JavaPluginInfo. + cea5f4f: Cherrypick Bzlmod documentation (#14301) + 227e49e: Format work requests according to ndjson spec + ae0a6c9: Enable user_link_flags_feature for macosx cc_toolchain_config + 8c2c78c: Remote: Use Action's salt field to differentiate cache across workspaces. + f948989: [5.x] Remote: Fix "file not found" error when remote cache is changed from enabled to disabled. (#14321) + 3069ac4: Delete marker file before fetching an external repository + c05c626: Remote: Fix file counting in merkletree.DirectoryTreeBuilder + d84f799: Fix remote spawn tests for remote_merkle_tree_cache=true + 59e16e9: Show skipped tests as a warning + 76b3c24: Build xcode-locator as a universal binary + aa52f2d: Exit collect_coverage.sh early if LCOV_MERGER is not set. + 4256d46: Automated rollback of commit d84f799. + dce2435: [apple] fix issues compiling C in objc_library for watchos/armv7k + bfc2413: 5.x: Remote: Ignore blobs referenced in BEP if the generating action cannot be cached remotely. (#14389) + 5aef53a: Remote: Don't blocking-get when acquiring gRPC connections. (#14420) + 005361c: Disable IncludeValidation for ObjC in bazel + d703b7b: Update java_tools v11.6 + 90965b0: Stop remote blob upload if upload is complete. (#14467) + dc59d9e: [5.x] Make remote BES uploader better (#14472) + 2edab73: Avoid too verbose warnings in terminal when cache issues + 1160485: Rename --project_id to --bes_instance_name + c63d9ec: Automated rollback of commit bfdfa6e. + b341802: [apple] support watchos_arm64 in toolchain + 43bcf80: Disable implicitly collecting baseline coverage for toolchain targets. + 302971e: Automated rollback of commit 7d09b4a. + 6200202: Bzlmod: Starlarkify default attr values for TypeCheckedTags + 38117d4: Fix build after rc4 cherrypicks (#14581) + 41feb61: Release 5.0.0 (2022-01-19) + 486d153: Find runfiles in directories that are themselves runfiles + 0de7bb9: Don't resolve symlinks for --sandbox_base + 8b60c90: Remove uses of -lstdc++ on darwin + 60f757c: Allow Label instances as keys in select (#14755) + 3836ad0: Remote: Only waits for background tasks from remote execution. + 8734ccf: Add the default solib dir to the rpath for cc_imports with transitions + 9e16a64: Flip --experimental_worker_allow_json_protocol + fce7ea8: Fix `ctx.fragments.apple.single_arch_cpu` returning incorrect cpu for tools when host cpu and exec cpu are different + 0c1d09e: Propagate --experimental_cc_implementation_deps to host config + 1c3a245: Support select() on constraint_value for aliases. + 67a133b: Improve documentation for select() + 5356fed: Cherrypicks for experimental cc_shared_library (#14773) + ffdd633: [apple] support tvos_sim_arm64 in toolchain (#14779) + a58ddea: Cherry pick win arm64 (#14794) + dc41a20: [5.1.0] cherrypick subpackages support (#14780) + 86e2db7: Add a helper method for rules to depend on the cpp toolchain type. + 6990c02: UrlRewriter should be able to load credentials from .netrc (#14834) + 32d1606: Add "arch" struct field to repository_os + 2cfdcea: [5.x] bzlmod: Add support for WORKSPACE.bzlmod (#14813) + c2ddbd1: Ignore missing include directory in JDK distribution. + 16de035: Fix bazel coverage false negative + 0c74741: Remote: Postpone the block waiting in `afterCommand` to `BlockWaitingModule` (#14833) + 3297d92: Switch to `ProcessHandle` for getting the PID (#14842) + a987b98: Fix uses of std++ on bsd + d184e48: Remote: handle early return of compressed blobs uploads + 0b09e9e: Add removeprefix/removesuffix to Starlark strings + d42ab0c: Fix default CPU for macOS and iOS (#14923) + cd24f39: Add paramfile support for def_parser, since in rare cases on Windows command line character limit was reached. + 0b1beef: Normalize rpath entries to guard against missing default solib dir + 24e8242: Fix aggressive params file assumption + c45838b: Fix precompiled libs not in runfiles of cc_shared_library (#14943) + 764614e: Bzlmod: Allow multiple `use_extension`s on the same extension (#14945) + fa761f8: Fix typo in `apple_common.platform` docs + f7d8288: Yield a Proxy for addresses without protocol + 8cefb8b: Avoid merging URLs in HttpUtils + b480480: Make protocOpts() public. (#14952) + 113eaca: Do not hide BulkTransferException messages when there were more than one exception + b1bf9d6: merkle_tree_cache: change default size to 1000 + f15e0c7: Add --experimental_repository_cache_urls_as_default_canonical_id to help detect broken repository URLs (#14989) + f421474: Expose the logic to read user netrc file + b858ec3: Correct cpu and os values of `local_config_cc_toolchains` targets + 5e79972: Expose CoverageOutputGenerator on a Fragment (#14997) + 78f0311: Correct error runfiles cc_shared_library (#14998) + 7937dd1: [5.1] Adding Starlark dependencies to the package //external (#14991) + a73aa12: Remote: Fix crashes with InterruptedException when using http cache. + f8707c0: Account for interface libraries in cc_shared_library + a570f5f: Fix coverage runfiles directory issue + 95de355: Do not validate input-only settings in transitions (#15048) + 71747cc: Filter out system headers on macOS. + cb6500a: Update Bazel bootstrap documentation and remove obsolete flags. (#15065) + 4c031d1: [5.1] Undocument --bes_best_effort (#15066) + 267142f: Fix conflicting actions error when specifying --host_macos_minimum_os (#15068) + f192362: [5.1] Remote: Action should not be successful and cached if outputs were not created (#15071) + 00d74ff: Support decompressing zstd tar archives for repository rules. + f585783: Remote: Don't check TreeArtifact output + efb2b80: osx_cc_wrapper: Only expand existing response files + c771c43: Remote: Fix crashes by InterruptedException when dynamic execution is enabled. (#15091) + 3785677: Use python3 on macOS + 815d9e4: Release 5.1.0 (2022-03-24) + 1fbb69e: Prepare 5.1.1 release + df153df: Fix CODEOWNERS syntax + 2b92a31: Remote: Don't check declared outputs for failed action + b47aa71: Upgrade abseil version to the latest + c49c45d: Revert default export all symbols on Windows + 7d3fb99: Support ZIP files with total number of disks = 0 + 0f5dc11: Release 5.1.1 (2022-04-08) + 2422cfb: Update CODEOWNERS + bbcff18: [5.2.0] Update java_tools 11.7.1 (#15231) + 9c98120: Add support for .ar archives (and .deb files) + d3435b0: Seperate GetSelfPath implementation for Blaze and Bazel + c94572b: Include jdk.crypto.mscapi in minimized Windows embedded JDK + 299022c: remote: Proactively close the ZstdInputStream in ZstdDecompressingOutputStream. + 2770799: Collect coverage from cc_binary data deps of java_test + 3442179: Configure Apple crosstool to return a complete target triple that includes minimum OS version and target environment + bb6f1a7: Collect C++ lcov coverage if runtime object not in runfiles + dbb6e99: Fixing dependencies of //external package + f0213bb: [5.2] Upgrade Google Auth Version (#15383) + a1a74c9: Fix chocolatey package - docsUrl must not 404 (#15395) + fe644be: Fix cache leak when applying transitions when only a rule's attributes change. + ad74d52: Fix checking remote cache for omitted files in buildevent file (#15405) + ac21910: fix(bzlmod): throw on json parse exception + 3d85b88: Add a flag to expose undeclared test outputs in unzipped form. (#15431) + abd7a9f: Remove -U_FORTIFY_SOURCE when thin_lto is enabled (#15433) + 53b9cb8: Catch NumberFormatException while trying to parse thread id. + 19740b5: Improve the --sandbox_debug error message + 0a2a43f: Set keywords on appropriate lifecycle events. + 394ddb8: Record additional profiling information for remotely executed actions. + 652b48e: Fix downloading remote execution output files inside output dirs. (#15444) + 73f1ecb: Fix android emulator darwin_arm64 select + 2649c7c: Fix --use_top_level_targets_for_symlinks with aliases (#15446) + fa1081c: Filter libtool warning about table of contents + 26f8783: Unify sandbox/remote handling of empty TreeArtifact inputs (#15449) + 6b21b77: Revert "Fixes incorrect install names on darwin platforms" + e133e66: config doesn't error on duplicate `--define` values (#15473) + 84d5917: Collect coverage from cc_binary data deps of py_test (#15298) + 519d2da: SolibSymlinkAction does not need exec platform or properties + 6e54699: Let Starlark tests inherit env variables (#15217) + 9610ae8: Update PythonZipper action to use CommandLineItem.CapturingMapFn + 2f1ff6f: Make `coverage --combined_report=lcov` skip incompatible tests + 9fad5a3: Disable ReturnValueIgnored checks to unblock java_tools release + 0120118: Bump the limit of Bazel install base size (#15585) + 668805a: Upgrade zlib to 1.2.12 + 4d900ce: [5.2] Remote: Fix a bug that outputs of actions tagged with no-remote are u... (#15453) + b703cb9: Add feature to produce serialized diagnostics files (#15600) + 2e8458b: Release 5.2.0 (2022-06-07) + 536f8d9: Fix fail message construction in cc_shared_library + 2d42925: Define cc-compiler-darwin in Xcode toolchain + a1d7d1f: Fix alwayslink in objc_import + d273cb6: Unify URL/URLs parameter code across http_archive, http_file, http_jar + fea32be: Preserve --experimental_allow_unresolved_symlinks in exec cfg + e4bc370: Ck/cherry pick cc shared library (#15754) + dbdfa07: Let Starlark executable rules specify their environment (#15766) + e2a6a2b: Fix string formatting when java_home path is missing. + d54a288: Optionally enable LLVM profile continuous mode + ad17b44: Print remote execution message when the action times out (#15772) + 240e3d1: Add missing line to cherrypick e4bc370 (#15784) + 804b474: Replace strdupa with strdup + 62be9ea: Bzlmod: Better canonical repo names for modules with overrides (#15793) + d4663a1: Add repo env test (#15768) + 594962c: Add is_root struct field to bazel_module (#15815) + 3dd2b93: Fix null pointer crash with `bazel coverage` on only incompatible tests + 4175018: Add util for finding credential helper to use + 3ea9eb2: Merge ManifestMergerAction-related commits into release-5.3.0 (#15824) + 64571a4: Ck/cherrypick 15669 (#15788) + 1404651: Create output directories for remote execution (#15818) + ae523f8: Use tree artifacts in bootclasspath rule + 37f181c: [credentialhelper] Add types to communicate with the subprocess + 06ca634: Add a flag to force Bazel to download certain artifacts when using --remote_download_minimal (#15870) + d35f923: RemoteExecutionService: fix outputs not being uploaded + 78af34f: Cherry-pick proto_lang_toolchain Starlarkfication and proto_common module (#15854) + afb434d: Fix behavior of `print()` in module extensions + 6714c30: [credentialhelper] Implement invoking credential helper as subprocess + 0f05904: Add register_{execution_platforms,toolchains} directives to MODULE.bazel files (#15852) + 33516e2: [remote] Improve .netrc test in RemoteModuleTest + aa2a1f3: Fix ZipDecompressor windows 0x80 (file attribute normal) + 30f16e5: Replace uses of `cfg = "host"` with `cfg = "exec"` (#15922) + 2a8d0ad: target pattern file: allow comments + 6f73205: Add factory for creating paths relative to well-known roots (#15931) + 32cc8e6: Update CODEOWNERS (#15910) + 63bc14b: Implement native analysis_test call. (#15940) + 4df77f7: Increase osx_cc_configure timeouts + cdf01a3: Allow string_list flags to be set via repeated flag uses + 05e758d: [credentialhelper] Add parser for flag syntax (#15929) + e4ee344: Docs should mention the new no-remote-cache-upload tag (#15965) + 96d23d3: Add netrc support to --bes_backend (#15970) + c5bc34e: Add CommandLinePathFactory to CommandEnvironment (#15971) + 508f185: Move newCredentialHelperProvider into GoogleAuthUtils (#15973) + 14c944a: Wire up credential helper to command-line flag(s) (#15976) + 04c373b: Add `--output=files` mode to cquery (#15979) + edfe2a1: Make cpp assembly file extensions case sensitive again + 4ae8538: Prevent aspects from executing on incompatible targets (#15984) + f440f8e: Remote: Fix performance regression in "upload missing inputs". (#15998) + 0109031: Updated Codeowners file (#16032) + 6102d33: Propagate the error message when a credential helper fails. (#16030) + a8dacc7: Migrate legacy desugar wrapper to new rlocation() (#16025) + 11368be: Correctly report errors thrown by CommandLinePathFactory#create. + 82452c7: Fix an issue that `incompatible_remote_build_event_upload_respect_no_… (#16045) + e745468: Fix rpath for binaries in external repositories (#16079) + 83041b1: Refactor combined cache. (#16110) + c62496f: C++: Add compound error linked statically but not exported (#16113) Important changes: - alias() can now select() directly on constraint_value() Fixes #13047. Closes #14310. - Fixed an issue where Bazel could erroneously report a test passes in coverage mode without actually running the test. - Make protocOpts() publicly accessible. - Add coverage configuration fragment, used to expose output_generator label. - Bazel now no longer includes system headers on macOS in coverage reports (#14969). Closes #14971. - Starlark test rules can use the new inherited_environment parameter of testing.TestEnvironment to specify environment variables whose values should be inherited from the shell environment. Closes #14849. - none RELNOTES:none - Enable merging permissions during Android manifest merging with the --merge_android_manifest_permissions flag. - Added new register_{execution_platforms,toolchains} directives to the MODULE.bazel file, to replace the {execution_platforms,toolchains}_to_register attributes on the module() directive. - Add support for fetching RPC credentials from credential helper. Progress on #15856 Closes #15947. - `cquery`'s new output mode [`--output=files`](https://bazel.build/docs/cquery#files-output) lists the output files of the targets matching the query. It takes the current value of `--output_groups` into account. Closes #15552. - Fix for desugaring failure on Bazel+Android+Windows build scenario. This release contains contributions from many people at Google, as well as amberdixon, Andreas Fuchs, Benjamin Peterson, Brentley Jones, Chenchu Kolli, Dan Fleming, Danny Wolf, Emil Kattainen, Fabian Meumertzheim, Gowroji Sunil, hvadehra, Juh-Roch, Keith Smiley, kshyanashree, Niyas Sait, Noa Resare, Oliver Eikemeier, oquenchil, Peter Mounce, Philipp Schrader, Thi Doãn, Xùdōng Yáng, Yannic, Zhongpeng Lin.
copybara-service bot
pushed a commit
that referenced
this pull request
Aug 22, 2022
Baseline: 8d66a41 Cherry picks: + becd149: Remote: Cache merkle trees + d7628e1: Update DEFAULT_IOS_CPU for M1 arm64 simulator support + 80c56ff: Compile Apple tools as fat binaries if possible + 3c09f34: Add protobuf as a well known module + 3a5b360: Remote: Merge target-level exec_properties with --remote_default_exec_properties + 917e15e: Add -no_uuid for hermetic macOS toolchain setup + f5cf8b0: Remote: Fixes an issue when --experimental_remote_cache_async encounter flaky tests. + 77a002c: Remove DigestUtils.getDigestInExclusiveMode() now that SsdModule has … + 557a7e7: Fixes for the Starlark transition hash computation (#14251) + 34c7146: Do location expansion in copts of objc_library + 50274a9: [5.x] Remote: Add support for compression on gRPC cache (#14277) + 61bf2e5: Automated rollback of commit 34c7146. + 79888fe: Silence a zstd-jni GCC warning. + 063b5c9: Remote: Limit max number of gRPC connections by --remote_max_connections. + fd727ec: Do location expansion in copts of objc_library + 23d0969: Fix _is_shared_library_extension_valid + 5cf1d6e: Remove merging of java_outputs in JavaPluginInfo. + cea5f4f: Cherrypick Bzlmod documentation (#14301) + 227e49e: Format work requests according to ndjson spec + ae0a6c9: Enable user_link_flags_feature for macosx cc_toolchain_config + 8c2c78c: Remote: Use Action's salt field to differentiate cache across workspaces. + f948989: [5.x] Remote: Fix "file not found" error when remote cache is changed from enabled to disabled. (#14321) + 3069ac4: Delete marker file before fetching an external repository + c05c626: Remote: Fix file counting in merkletree.DirectoryTreeBuilder + d84f799: Fix remote spawn tests for remote_merkle_tree_cache=true + 59e16e9: Show skipped tests as a warning + 76b3c24: Build xcode-locator as a universal binary + aa52f2d: Exit collect_coverage.sh early if LCOV_MERGER is not set. + 4256d46: Automated rollback of commit d84f799. + dce2435: [apple] fix issues compiling C in objc_library for watchos/armv7k + bfc2413: 5.x: Remote: Ignore blobs referenced in BEP if the generating action cannot be cached remotely. (#14389) + 5aef53a: Remote: Don't blocking-get when acquiring gRPC connections. (#14420) + 005361c: Disable IncludeValidation for ObjC in bazel + d703b7b: Update java_tools v11.6 + 90965b0: Stop remote blob upload if upload is complete. (#14467) + dc59d9e: [5.x] Make remote BES uploader better (#14472) + 2edab73: Avoid too verbose warnings in terminal when cache issues + 1160485: Rename --project_id to --bes_instance_name + c63d9ec: Automated rollback of commit bfdfa6e. + b341802: [apple] support watchos_arm64 in toolchain + 43bcf80: Disable implicitly collecting baseline coverage for toolchain targets. + 302971e: Automated rollback of commit 7d09b4a. + 6200202: Bzlmod: Starlarkify default attr values for TypeCheckedTags + 38117d4: Fix build after rc4 cherrypicks (#14581) + 41feb61: Release 5.0.0 (2022-01-19) + 486d153: Find runfiles in directories that are themselves runfiles + 0de7bb9: Don't resolve symlinks for --sandbox_base + 8b60c90: Remove uses of -lstdc++ on darwin + 60f757c: Allow Label instances as keys in select (#14755) + 3836ad0: Remote: Only waits for background tasks from remote execution. + 8734ccf: Add the default solib dir to the rpath for cc_imports with transitions + 9e16a64: Flip --experimental_worker_allow_json_protocol + fce7ea8: Fix `ctx.fragments.apple.single_arch_cpu` returning incorrect cpu for tools when host cpu and exec cpu are different + 0c1d09e: Propagate --experimental_cc_implementation_deps to host config + 1c3a245: Support select() on constraint_value for aliases. + 67a133b: Improve documentation for select() + 5356fed: Cherrypicks for experimental cc_shared_library (#14773) + ffdd633: [apple] support tvos_sim_arm64 in toolchain (#14779) + a58ddea: Cherry pick win arm64 (#14794) + dc41a20: [5.1.0] cherrypick subpackages support (#14780) + 86e2db7: Add a helper method for rules to depend on the cpp toolchain type. + 6990c02: UrlRewriter should be able to load credentials from .netrc (#14834) + 32d1606: Add "arch" struct field to repository_os + 2cfdcea: [5.x] bzlmod: Add support for WORKSPACE.bzlmod (#14813) + c2ddbd1: Ignore missing include directory in JDK distribution. + 16de035: Fix bazel coverage false negative + 0c74741: Remote: Postpone the block waiting in `afterCommand` to `BlockWaitingModule` (#14833) + 3297d92: Switch to `ProcessHandle` for getting the PID (#14842) + a987b98: Fix uses of std++ on bsd + d184e48: Remote: handle early return of compressed blobs uploads + 0b09e9e: Add removeprefix/removesuffix to Starlark strings + d42ab0c: Fix default CPU for macOS and iOS (#14923) + cd24f39: Add paramfile support for def_parser, since in rare cases on Windows command line character limit was reached. + 0b1beef: Normalize rpath entries to guard against missing default solib dir + 24e8242: Fix aggressive params file assumption + c45838b: Fix precompiled libs not in runfiles of cc_shared_library (#14943) + 764614e: Bzlmod: Allow multiple `use_extension`s on the same extension (#14945) + fa761f8: Fix typo in `apple_common.platform` docs + f7d8288: Yield a Proxy for addresses without protocol + 8cefb8b: Avoid merging URLs in HttpUtils + b480480: Make protocOpts() public. (#14952) + 113eaca: Do not hide BulkTransferException messages when there were more than one exception + b1bf9d6: merkle_tree_cache: change default size to 1000 + f15e0c7: Add --experimental_repository_cache_urls_as_default_canonical_id to help detect broken repository URLs (#14989) + f421474: Expose the logic to read user netrc file + b858ec3: Correct cpu and os values of `local_config_cc_toolchains` targets + 5e79972: Expose CoverageOutputGenerator on a Fragment (#14997) + 78f0311: Correct error runfiles cc_shared_library (#14998) + 7937dd1: [5.1] Adding Starlark dependencies to the package //external (#14991) + a73aa12: Remote: Fix crashes with InterruptedException when using http cache. + f8707c0: Account for interface libraries in cc_shared_library + a570f5f: Fix coverage runfiles directory issue + 95de355: Do not validate input-only settings in transitions (#15048) + 71747cc: Filter out system headers on macOS. + cb6500a: Update Bazel bootstrap documentation and remove obsolete flags. (#15065) + 4c031d1: [5.1] Undocument --bes_best_effort (#15066) + 267142f: Fix conflicting actions error when specifying --host_macos_minimum_os (#15068) + f192362: [5.1] Remote: Action should not be successful and cached if outputs were not created (#15071) + 00d74ff: Support decompressing zstd tar archives for repository rules. + f585783: Remote: Don't check TreeArtifact output + efb2b80: osx_cc_wrapper: Only expand existing response files + c771c43: Remote: Fix crashes by InterruptedException when dynamic execution is enabled. (#15091) + 3785677: Use python3 on macOS + 815d9e4: Release 5.1.0 (2022-03-24) + 1fbb69e: Prepare 5.1.1 release + df153df: Fix CODEOWNERS syntax + 2b92a31: Remote: Don't check declared outputs for failed action + b47aa71: Upgrade abseil version to the latest + c49c45d: Revert default export all symbols on Windows + 7d3fb99: Support ZIP files with total number of disks = 0 + 0f5dc11: Release 5.1.1 (2022-04-08) + 2422cfb: Update CODEOWNERS + bbcff18: [5.2.0] Update java_tools 11.7.1 (#15231) + 9c98120: Add support for .ar archives (and .deb files) + d3435b0: Seperate GetSelfPath implementation for Blaze and Bazel + c94572b: Include jdk.crypto.mscapi in minimized Windows embedded JDK + 299022c: remote: Proactively close the ZstdInputStream in ZstdDecompressingOutputStream. + 2770799: Collect coverage from cc_binary data deps of java_test + 3442179: Configure Apple crosstool to return a complete target triple that includes minimum OS version and target environment + bb6f1a7: Collect C++ lcov coverage if runtime object not in runfiles + dbb6e99: Fixing dependencies of //external package + f0213bb: [5.2] Upgrade Google Auth Version (#15383) + a1a74c9: Fix chocolatey package - docsUrl must not 404 (#15395) + fe644be: Fix cache leak when applying transitions when only a rule's attributes change. + ad74d52: Fix checking remote cache for omitted files in buildevent file (#15405) + ac21910: fix(bzlmod): throw on json parse exception + 3d85b88: Add a flag to expose undeclared test outputs in unzipped form. (#15431) + abd7a9f: Remove -U_FORTIFY_SOURCE when thin_lto is enabled (#15433) + 53b9cb8: Catch NumberFormatException while trying to parse thread id. + 19740b5: Improve the --sandbox_debug error message + 0a2a43f: Set keywords on appropriate lifecycle events. + 394ddb8: Record additional profiling information for remotely executed actions. + 652b48e: Fix downloading remote execution output files inside output dirs. (#15444) +
Externally visible changes:
proto_common_do_not_usemodule is exposed with implementation matching Bazel@HEADproto_lang_toolchainrule is extended withplugin_format_flag,progress_message,mnemonic,protocol_compilerattributesproto_lang_toolchainrules doesn't accept$(PLUGIN_OUT)placeholder (there are no uses on github)ProtoLangToolchainInfoare accessible in Starlark and the provider can be constructedctx.actions.runandctx.actions.run_shellacceptresource_setparameterOther invisible changes:
proto_lang_toolchainrule is replaced with Starlark implementationProtoLangToolchainInfois implemented in Starlarkproto_common,proto_lang_toolchainrules are imported.resource_setparameterWhat's not changed:
proto_library,*_proto_library.Cherry-picks for "Starlarkify proto_lang_toolchain rule":
ProtoCommon.checkPrivateStarlarkificationAllowlistand Starlark annotations inProtoInfoandProtoSource(mostly from 2a44dbd Implement proto_common.write_descriptor_set in Starlark.)Cherry-picks for "Implement proto_common":
proto_common.compileproto_libraryStarlarkificiation