Skip to content

Conversation

@ted-xie
Copy link
Contributor

@ted-xie ted-xie commented Jul 6, 2022

@ted-xie ted-xie requested a review from ckolli5 as a code owner July 6, 2022 17:26
@ted-xie ted-xie force-pushed the bazel_5.3.0_manifestmergeaction branch from eeb8d33 to 62ee255 Compare July 6, 2022 18:02
Bencodes added 2 commits July 6, 2022 14:04
Adding support for conditionally merging `uses-permissions`.

bazelbuild#10628
bazelbuild#5411

Closes bazelbuild#13445.

RELNOTES: Enable merging permissions during Android manifest merging with the --merge_android_manifest_permissions flag.
PiperOrigin-RevId: 439613035
Calling `System#exit` kills the worker during the build. Passing the exception up to the worker should be enough for it to end up in the worker or local execution output.

Closes bazelbuild#14427.

PiperOrigin-RevId: 447808701
@ted-xie ted-xie force-pushed the bazel_5.3.0_manifestmergeaction branch from 62ee255 to 593c2fd Compare July 6, 2022 18:05
`manifest.toString().replaceFirst("^/", "")` silently fails on windows machines causing `removePermissions` to write to the original test file. This pull request creates a new temp file that `removePermissions` can write the modified manifest to.

Pulling this change out of another PR so that it's easier to merge. Original PR here https://github.com/bazelbuild/bazel/pull/13445/files#r631575251

Closes bazelbuild#13760.

PiperOrigin-RevId: 438643774
@ted-xie ted-xie changed the title Merge two ManifestMergerAction-related commits into release-5.3.0 Merge ManifestMergerAction-related commits into release-5.3.0 Jul 6, 2022
@ted-xie
Copy link
Contributor Author

ted-xie commented Jul 6, 2022

@ckolli5 Looks like all of the presubmit tests pass with these 3 commits cherry-picked.

@ckolli5 ckolli5 merged commit 3ea9eb2 into bazelbuild:release-5.3.0 Jul 6, 2022
@ted-xie ted-xie deleted the bazel_5.3.0_manifestmergeaction branch July 6, 2022 20:37
copybara-service bot pushed a commit that referenced this pull request Aug 22, 2022
Baseline: 8d66a41

Cherry picks:

   + becd149:
     Remote: Cache merkle trees
   + d7628e1:
     Update DEFAULT_IOS_CPU for M1 arm64 simulator support
   + 80c56ff:
     Compile Apple tools as fat binaries if possible
   + 3c09f34:
     Add protobuf as a well known module
   + 3a5b360:
     Remote: Merge target-level exec_properties with
     --remote_default_exec_properties
   + 917e15e:
     Add -no_uuid for hermetic macOS toolchain setup
   + f5cf8b0:
     Remote: Fixes an issue when --experimental_remote_cache_async
     encounter flaky tests.
   + 77a002c:
     Remove DigestUtils.getDigestInExclusiveMode() now that SsdModule
     has …
   + 557a7e7:
     Fixes for the Starlark transition hash computation (#14251)
   + 34c7146:
     Do location expansion in copts of objc_library
   + 50274a9:
     [5.x] Remote: Add support for compression on gRPC cache (#14277)
   + 61bf2e5:
     Automated rollback of commit
     34c7146.
   + 79888fe:
     Silence a zstd-jni GCC warning.
   + 063b5c9:
     Remote: Limit max number of gRPC connections by
     --remote_max_connections.
   + fd727ec:
     Do location expansion in copts of objc_library
   + 23d0969:
     Fix _is_shared_library_extension_valid
   + 5cf1d6e:
     Remove merging of java_outputs in JavaPluginInfo.
   + cea5f4f:
     Cherrypick Bzlmod documentation (#14301)
   + 227e49e:
     Format work requests according to ndjson spec
   + ae0a6c9:
     Enable user_link_flags_feature for macosx cc_toolchain_config
   + 8c2c78c:
     Remote: Use Action's salt field to differentiate cache across
     workspaces.
   + f948989:
     [5.x] Remote: Fix "file not found" error when remote cache is
     changed from enabled to disabled.  (#14321)
   + 3069ac4:
     Delete marker file before fetching an external repository
   + c05c626:
     Remote: Fix file counting in merkletree.DirectoryTreeBuilder
   + d84f799:
     Fix remote spawn tests for remote_merkle_tree_cache=true
   + 59e16e9:
     Show skipped tests as a warning
   + 76b3c24:
     Build xcode-locator as a universal binary
   + aa52f2d:
     Exit collect_coverage.sh early if LCOV_MERGER is not set.
   + 4256d46:
     Automated rollback of commit
     d84f799.
   + dce2435:
     [apple] fix issues compiling C in objc_library for watchos/armv7k
   + bfc2413:
     5.x: Remote: Ignore blobs referenced in BEP if the generating
     action cannot be cached remotely. (#14389)
   + 5aef53a:
     Remote: Don't blocking-get when acquiring gRPC connections.
     (#14420)
   + 005361c:
     Disable IncludeValidation for ObjC in bazel
   + d703b7b:
     Update java_tools v11.6
   + 90965b0:
     Stop remote blob upload if upload is complete. (#14467)
   + dc59d9e:
     [5.x] Make remote BES uploader better (#14472)
   + 2edab73:
     Avoid too verbose warnings in terminal when cache issues
   + 1160485:
     Rename --project_id to --bes_instance_name
   + c63d9ec:
     Automated rollback of commit
     bfdfa6e.
   + b341802:
     [apple] support watchos_arm64 in toolchain
   + 43bcf80:
     Disable implicitly collecting baseline coverage for toolchain
     targets.
   + 302971e:
     Automated rollback of commit
     7d09b4a.
   + 6200202:
     Bzlmod: Starlarkify default attr values for TypeCheckedTags
   + 38117d4:
     Fix build after rc4 cherrypicks (#14581)
   + 41feb61:
     Release 5.0.0 (2022-01-19)
   + 486d153:
     Find runfiles in directories that are themselves runfiles
   + 0de7bb9:
     Don't resolve symlinks for --sandbox_base
   + 8b60c90:
     Remove uses of -lstdc++ on darwin
   + 60f757c:
     Allow Label instances as keys in select (#14755)
   + 3836ad0:
     Remote: Only waits for background tasks from remote execution.
   + 8734ccf:
     Add the default solib dir to the rpath for cc_imports with
     transitions
   + 9e16a64:
     Flip --experimental_worker_allow_json_protocol
   + fce7ea8:
     Fix `ctx.fragments.apple.single_arch_cpu` returning incorrect
     cpu for tools when host cpu and exec cpu are different
   + 0c1d09e:
     Propagate --experimental_cc_implementation_deps to host config
   + 1c3a245:
     Support select() on constraint_value for aliases.
   + 67a133b:
     Improve documentation for select()
   + 5356fed:
     Cherrypicks for experimental cc_shared_library (#14773)
   + ffdd633:
     [apple] support tvos_sim_arm64 in toolchain (#14779)
   + a58ddea:
     Cherry pick win arm64 (#14794)
   + dc41a20:
     [5.1.0] cherrypick subpackages support (#14780)
   + 86e2db7:
     Add a helper method for rules to depend on the cpp toolchain
     type.
   + 6990c02:
     UrlRewriter should be able to load credentials from .netrc
     (#14834)
   + 32d1606:
     Add "arch" struct field to repository_os
   + 2cfdcea:
     [5.x] bzlmod: Add support for WORKSPACE.bzlmod (#14813)
   + c2ddbd1:
     Ignore missing include directory in JDK distribution.
   + 16de035:
     Fix bazel coverage false negative
   + 0c74741:
     Remote: Postpone the block waiting in `afterCommand` to
     `BlockWaitingModule` (#14833)
   + 3297d92:
     Switch to `ProcessHandle` for getting the PID (#14842)
   + a987b98:
     Fix uses of std++ on bsd
   + d184e48:
     Remote: handle early return of compressed blobs uploads
   + 0b09e9e:
     Add removeprefix/removesuffix to Starlark strings
   + d42ab0c:
     Fix default CPU for macOS and iOS (#14923)
   + cd24f39:
     Add paramfile support for def_parser, since in rare cases on
     Windows command line character limit was reached.
   + 0b1beef:
     Normalize rpath entries to guard against missing default solib
     dir
   + 24e8242:
     Fix aggressive params file assumption
   + c45838b:
     Fix precompiled libs not in runfiles of cc_shared_library
     (#14943)
   + 764614e:
     Bzlmod: Allow multiple `use_extension`s on the same extension
     (#14945)
   + fa761f8:
     Fix typo in `apple_common.platform` docs
   + f7d8288:
     Yield a Proxy for addresses without protocol
   + 8cefb8b:
     Avoid merging URLs in HttpUtils
   + b480480:
     Make protocOpts() public. (#14952)
   + 113eaca:
     Do not hide BulkTransferException messages when there were more
     than one exception
   + b1bf9d6:
     merkle_tree_cache: change default size to 1000
   + f15e0c7:
     Add --experimental_repository_cache_urls_as_default_canonical_id
     to help detect broken repository URLs (#14989)
   + f421474:
     Expose the logic to read user netrc file
   + b858ec3:
     Correct cpu and os values of `local_config_cc_toolchains` targets
   + 5e79972:
     Expose CoverageOutputGenerator on a Fragment (#14997)
   + 78f0311:
     Correct error runfiles cc_shared_library (#14998)
   + 7937dd1:
     [5.1] Adding Starlark dependencies to the package //external
     (#14991)
   + a73aa12:
     Remote: Fix crashes with InterruptedException when using http
     cache.
   + f8707c0:
     Account for interface libraries in cc_shared_library
   + a570f5f:
     Fix coverage runfiles directory issue
   + 95de355:
     Do not validate input-only settings in transitions (#15048)
   + 71747cc:
     Filter out system headers on macOS.
   + cb6500a:
     Update Bazel bootstrap documentation and remove obsolete flags.
     (#15065)
   + 4c031d1:
     [5.1] Undocument --bes_best_effort (#15066)
   + 267142f:
     Fix conflicting actions error when specifying
     --host_macos_minimum_os (#15068)
   + f192362:
     [5.1] Remote: Action should not be successful and cached if
     outputs were not created (#15071)
   + 00d74ff:
     Support decompressing zstd tar archives for repository rules.
   + f585783:
     Remote: Don't check TreeArtifact output
   + efb2b80:
     osx_cc_wrapper: Only expand existing response files
   + c771c43:
     Remote: Fix crashes by InterruptedException when dynamic
     execution is enabled. (#15091)
   + 3785677:
     Use python3 on macOS
   + 815d9e4:
     Release 5.1.0 (2022-03-24)
   + 1fbb69e:
     Prepare 5.1.1 release
   + df153df:
     Fix CODEOWNERS syntax
   + 2b92a31:
     Remote: Don't check declared outputs for failed action
   + b47aa71:
     Upgrade abseil version to the latest
   + c49c45d:
     Revert default export all symbols on Windows
   + 7d3fb99:
     Support ZIP files with total number of disks = 0
   + 0f5dc11:
     Release 5.1.1 (2022-04-08)
   + 2422cfb:
     Update CODEOWNERS
   + bbcff18:
     [5.2.0] Update java_tools 11.7.1 (#15231)
   + 9c98120:
     Add support for .ar archives (and .deb files)
   + d3435b0:
     Seperate GetSelfPath implementation for Blaze and Bazel
   + c94572b:
     Include jdk.crypto.mscapi in minimized Windows embedded JDK
   + 299022c:
     remote: Proactively close the ZstdInputStream in
     ZstdDecompressingOutputStream.
   + 2770799:
     Collect coverage from cc_binary data deps of java_test
   + 3442179:
     Configure Apple crosstool to return a complete target triple
     that includes minimum OS version and target environment
   + bb6f1a7:
     Collect C++ lcov coverage if runtime object not in runfiles
   + dbb6e99:
     Fixing dependencies of //external package
   + f0213bb:
     [5.2] Upgrade Google Auth Version (#15383)
   + a1a74c9:
     Fix chocolatey package - docsUrl must not 404 (#15395)
   + fe644be:
     Fix cache leak when applying transitions when only a rule's
     attributes change.
   + ad74d52:
     Fix checking remote cache for omitted files in buildevent file
     (#15405)
   + ac21910:
     fix(bzlmod): throw on json parse exception
   + 3d85b88:
     Add a flag to expose undeclared test outputs in unzipped form.
     (#15431)
   + abd7a9f:
     Remove -U_FORTIFY_SOURCE when thin_lto is enabled (#15433)
   + 53b9cb8:
     Catch NumberFormatException while trying to parse thread id.
   + 19740b5:
     Improve the --sandbox_debug error message
   + 0a2a43f:
     Set keywords on appropriate lifecycle events.
   + 394ddb8:
     Record additional profiling information for remotely executed
     actions.
   + 652b48e:
     Fix downloading remote execution output files inside output
     dirs. (#15444)
   + 73f1ecb:
     Fix android emulator darwin_arm64 select
   + 2649c7c:
     Fix --use_top_level_targets_for_symlinks with aliases (#15446)
   + fa1081c:
     Filter libtool warning about table of contents
   + 26f8783:
     Unify sandbox/remote handling of empty TreeArtifact inputs
     (#15449)
   + 6b21b77:
     Revert "Fixes incorrect install names on darwin platforms"
   + e133e66:
     config doesn't error on duplicate `--define` values (#15473)
   + 84d5917:
     Collect coverage from cc_binary data deps of py_test (#15298)
   + 519d2da:
     SolibSymlinkAction does not need exec platform or properties
   + 6e54699:
     Let Starlark tests inherit env variables (#15217)
   + 9610ae8:
     Update PythonZipper action to use CommandLineItem.CapturingMapFn
   + 2f1ff6f:
     Make `coverage --combined_report=lcov` skip incompatible tests
   + 9fad5a3:
     Disable ReturnValueIgnored checks to unblock java_tools release
   + 0120118:
     Bump the limit of Bazel install base size (#15585)
   + 668805a:
     Upgrade zlib to 1.2.12
   + 4d900ce:
     [5.2] Remote: Fix a bug that outputs of actions tagged with
     no-remote are u... (#15453)
   + b703cb9:
     Add feature to produce serialized diagnostics files (#15600)
   + 2e8458b:
     Release 5.2.0 (2022-06-07)
   + 536f8d9:
     Fix fail message construction in cc_shared_library
   + 2d42925:
     Define cc-compiler-darwin in Xcode toolchain
   + a1d7d1f:
     Fix alwayslink in objc_import
   + d273cb6:
     Unify URL/URLs parameter code across http_archive, http_file,
     http_jar
   + fea32be:
     Preserve --experimental_allow_unresolved_symlinks in exec cfg
   + e4bc370:
     Ck/cherry pick cc shared library (#15754)
   + dbdfa07:
     Let Starlark executable rules specify their environment (#15766)
   + e2a6a2b:
     Fix string formatting when java_home path is missing.
   + d54a288:
     Optionally enable LLVM profile continuous mode
   + ad17b44:
     Print remote execution message when the action times out (#15772)
   + 240e3d1:
     Add missing line to cherrypick
     e4bc370 (#15784)
   + 804b474:
     Replace strdupa with strdup
   + 62be9ea:
     Bzlmod: Better canonical repo names for modules with overrides
     (#15793)
   + d4663a1:
     Add repo env test (#15768)
   + 594962c:
     Add is_root struct field to bazel_module (#15815)
   + 3dd2b93:
     Fix null pointer crash with `bazel coverage` on only
     incompatible tests
   + 4175018:
     Add util for finding credential helper to use
   + 3ea9eb2:
     Merge ManifestMergerAction-related commits into release-5.3.0
     (#15824)
   + 64571a4:
     Ck/cherrypick 15669 (#15788)
   + 1404651:
     Create output directories for remote execution (#15818)
   + ae523f8:
     Use tree artifacts in bootclasspath rule
   + 37f181c:
     [credentialhelper] Add types to communicate with the subprocess
   + 06ca634:
     Add a flag to force Bazel to download certain artifacts when
     using --remote_download_minimal (#15870)
   + d35f923:
     RemoteExecutionService: fix outputs not being uploaded
   + 78af34f:
     Cherry-pick proto_lang_toolchain Starlarkfication and
     proto_common module (#15854)
   + afb434d:
     Fix behavior of `print()` in module extensions
   + 6714c30:
     [credentialhelper] Implement invoking credential helper as
     subprocess
   + 0f05904:
     Add register_{execution_platforms,toolchains} directives to
     MODULE.bazel files (#15852)
   + 33516e2:
     [remote] Improve .netrc test in RemoteModuleTest
   + aa2a1f3:
     Fix ZipDecompressor windows 0x80 (file attribute normal)
   + 30f16e5:
     Replace uses of `cfg = "host"` with `cfg = "exec"` (#15922)
   + 2a8d0ad:
     target pattern file: allow comments
   + 6f73205:
     Add factory for creating paths relative to well-known roots
     (#15931)
   + 32cc8e6:
     Update CODEOWNERS (#15910)
   + 63bc14b:
     Implement native analysis_test call. (#15940)
   + 4df77f7:
     Increase osx_cc_configure timeouts
   + cdf01a3:
     Allow string_list flags to be set via repeated flag uses
   + 05e758d:
     [credentialhelper] Add parser for flag syntax (#15929)
   + e4ee344:
     Docs should mention the new no-remote-cache-upload tag (#15965)
   + 96d23d3:
     Add netrc support to --bes_backend (#15970)
   + c5bc34e:
     Add CommandLinePathFactory to CommandEnvironment (#15971)
   + 508f185:
     Move newCredentialHelperProvider into GoogleAuthUtils (#15973)
   + 14c944a:
     Wire up credential helper to command-line flag(s) (#15976)
   + 04c373b:
     Add `--output=files` mode to cquery (#15979)
   + edfe2a1:
     Make cpp assembly file extensions case sensitive again
   + 4ae8538:
     Prevent aspects from executing on incompatible targets (#15984)
   + f440f8e:
     Remote: Fix performance regression in "upload missing inputs".
     (#15998)
   + 0109031:
     Updated Codeowners file (#16032)
   + 6102d33:
     Propagate the error message when a credential helper fails.
     (#16030)
   + a8dacc7:
     Migrate legacy desugar wrapper to new rlocation() (#16025)
   + 11368be:
     Correctly report errors thrown by CommandLinePathFactory#create.
   + 82452c7:
     Fix an issue that
     `incompatible_remote_build_event_upload_respect_no_… (#16045)
   + e745468:
     Fix rpath for binaries in external repositories (#16079)
   + 83041b1:
     Refactor combined cache. (#16110)
   + c62496f:
     C++: Add compound error linked statically but not exported
     (#16113)

Important changes:

  - alias() can now select() directly on constraint_value()

    Fixes #13047.

    Closes #14310.
  - Fixed an issue where Bazel could erroneously report a test passes
    in coverage mode without actually running the test.
  - Make protocOpts() publicly accessible.
  - Add coverage configuration fragment, used to expose
    output_generator label.
  - Bazel now no longer includes system headers on macOS in coverage
    reports (#14969).

    Closes #14971.
  - Starlark test rules can use the new inherited_environment
    parameter of testing.TestEnvironment to specify environment
    variables
    whose values should be inherited from the shell environment.

    Closes #14849.
  - none
    RELNOTES:none
  - Enable merging permissions during Android manifest merging with
    the --merge_android_manifest_permissions flag.
  - Added new register_{execution_platforms,toolchains} directives to
    the MODULE.bazel file, to replace the
    {execution_platforms,toolchains}_to_register attributes on the
    module() directive.
  - Add support for fetching RPC credentials from credential helper.

    Progress on #15856

    Closes #15947.
  - `cquery`'s new output mode
    [`--output=files`](https://bazel.build/docs/cquery#files-output)
    lists the output files of the targets matching the query. It
    takes the current value of `--output_groups` into account.

    Closes