Add `--output=files` mode to cquery by fmeum · Pull Request #15552 · bazelbuild/bazel

fmeum · 2022-05-23T12:36:16Z

With the new output mode --output=files, cquery lists all files advertised by the matched targets in the currently requested output groups.

This new mode has the following advantages over --output=starlark combined with an appropriate handcrafted --starlark:expr:

provides a canonical answer to the very common "Where are my build outputs?" question
is more friendly to new users as it doesn't require knowing about providers and non-BUILD dialect Starlark
takes the value of --output_groups into account
stays as close to the logic for build summaries printed by bazel build as possible

Fixes #8739

RELNOTES: cquery's new output mode --output=files lists the output files of the targets matching the query. It takes the current value of --output_groups into account.

aiuto · 2022-05-23T14:10:56Z

I am very hesitant about adding this since it seems to be just a convenience for what can be done with starlark mode. We are trying to reduce the size of bazel and move features to starlark not the other way around. I can look at it later this week.

fmeum · 2022-05-23T14:24:12Z

I am very hesitant about adding this since it seems to be just a convenience for what can be done with starlark mode. We are trying to reduce the size of bazel and move features to starlark not the other way around. I can look at it later this week.

I generally agree with this position, but I think this case is a bit special: "Where is my build output?" is a very common question that is also quite relevant for developers that are not core Bazel users and just happen to interface with it. Think of an OSS project that needs to build another OSS project that uses Bazel and has to figure out the set of files it needs to import. If we have the barrier to entry somewhere at "knows about DefaultInfo", that may already be too high for people to comfortably build their OSS project with Bazel knowing that others less familiar with Bazel will have to answer such questions.

gregestren · 2022-05-23T19:53:50Z

Disclaimer: I haven't reviewed this in any meaningful way yet.

Generally speaking I agree with @aiuto 's comment and default to a position of needing strong justification for new user-facing features.

I see @fmeum that you appreciate these points and are trying to make such a justification for this change. Whatever we ultimately decide I think it's fair for us to evaluate your points on their merit.

pcjanzen · 2022-05-23T20:18:13Z

A wise man once said:

The major weakness of this model is that it makes paths completely opaque. This is a calculated compromise: it assumes new APIs can be developed to help users find the subset of outputs they care about, and that these APIs will be important no matter what path model Bazel uses.

gregestren · 2022-05-23T20:26:45Z

Still support that statement 100%!

I'll hold back from saying more until I've properly read through this, and how all the themes connect.

aiuto · 2022-05-23T20:43:41Z

I think the value is not just from the output paths, but attaching a meaningful labeling to each. The output group name is generally good, but not everything produces that. Sometimes you have to go into the rule specific providers to understand the meaning of a file. Once that happens, we are back at writing rule specific starlark code.

gregestren · 2022-05-24T21:14:00Z

Reviewed further. I support this, with some caveats.

As @pcjanzen wisely pointed out, I've long supported a proper API for finding build outputs.
That's because it's strongly in the dev team's interest to have flexibility to change output path patterns (bazel-out/arm-fastbuild-<hash>-...). This lets us focus on API and performance improvements freely. So builds can get faster and more cacheable at a faster pace.
Historically, users rely on exact path patterns (like looking at what bazel build produced and copying the exact path into downstream tools). That's the complete opposite of what we want. The only alternative is a canonical API.
This is important enough to be a) universal, b) easy to use, c) works out of the box
I think that overcomes the "keep the API simple" concerns mentioned
I never knew quite what that API would look like. This is a nice approach to it: leveraging existing concepts (cquery, output groups) clearly and I think effectively. And with room for further expansion
I still want to be careful to support backwards compatible API expansion
I agree IdentifyingName: output would be a more powerful API
That requires API expansion (I suppose extending the Starlark calls that declare outputs). Sounds manageable but would require its own design
For now, is it reasonable to output a raw list? Or a dictionary where every key is empty to offer forward-facing structure for consuming tools?
I also agree more complex cases still require custom Starlark. But I think it's reasonable to offer a trivial API for the common case and custom Starlark for advanced cases
Conceptually we could still implement the logic in Starlark, if there was some way to bundle that behind the canonical API. But I don't think we have the infrastructure for that. And, again, I think working cleanly, out of the box, without requiring build expertise is a powerful argument.

gregestren · 2022-05-24T21:36:59Z

One more comment:

There's also some awkwardness with multi-target builds. I can imagine someone wanting to just replace their bazel build ... command with bazel cquery .... But:

Multiple targets all produce output in a big blob: you can't tell which output was from which target
Syntax isn't interchangeable: bazel build //:foo //:bar -> bazel cquery '//:foo+//:bar'

fmeum · 2022-05-25T07:04:48Z

Syntax isn't interchangeable: bazel build //:foo //:bar -> bazel cquery '//:foo+//:bar'

In fact, even before working on this change, I have always wondered about why bazel (c)query doesn't support a strict superset of the target specification grammar supported by bazel build. How about supporting bazel (c)query <expr 1> <expr 2> -- -<expr 3> and simply turning it into (<expr 1>) + (<expr 2>) - (<expr 3>) in the command logic before handing it to the query engine? bazel (c)query currently errors when it receives more than one positional argument, so this change would be backwards compatible and - at least in my opinion - useful beyond the situation of the current PR.

fmeum · 2022-05-25T07:50:55Z

That's because it's strongly in the dev team's interest to have flexibility to change output path patterns (bazel-out/arm-fastbuild-<hash>-...). This lets us focus on API and performance improvements freely. So builds can get faster and more cacheable at a faster pace.

With that aim, it is important to keep the following in mind: If content-based output paths are used to make builds more cacheable and if these paths are used throughout Bazel and not just on the level of Spawns, then cquery would no longer be the right tool to show output paths since knowing the paths would require executing (or recalling) a partial build.

Historically, users rely on exact path patterns (like looking at what bazel build produced and copying the exact path into downstream tools). That's the complete opposite of what we want. The only alternative is a canonical API.

Fully agreed. It's not completely uncommon to see ST hashes on the build outputs of top-level targets, at which point relying on exact paths is no longer reliable.

I agree IdentifyingName: output would be a more powerful API

Could you explain what you envision as IdentifyingName here? Would it contain more information than just the target (or possibly owner) label?

For now, is it reasonable to output a raw list? Or a dictionary where every key is empty to offer forward-facing structure for consuming tools?

I don't have a strong opinion here. When in doubt, I would prefer an output format that plays well with other Unix tools, given that this command will be a common interface to the non-Bazel world. There is jq however and it wouldn't be too uncommon to require it (or commit to emitting canonically indented JSON, which could also be parsed with grep in a pinch).

Conceptually we could still implement the logic in Starlark, if there was some way to bundle that behind the canonical API. But I don't think we have the infrastructure for that. And, again, I think working cleanly, out of the box, without requiring build expertise is a powerful argument.

I could implement --output=files as a thin wrapper around --output=starlark pointing at a shipped Starlark file, but for that would have to:

Parse the value of the --output_groups option in Starlark, duplicating the parsing logic.
Duplicate the filtering logic used in BuildResultPrinter.

That would probably defeat the purpose of decoupling this logic from core Bazel.

fmeum · 2022-05-25T07:51:18Z

That's because it's strongly in the dev team's interest to have flexibility to change output path patterns (bazel-out/arm-fastbuild-<hash>-...). This lets us focus on API and performance improvements freely. So builds can get faster and more cacheable at a faster pace.

With that aim, it is important to keep the following in mind: If content-based output paths are used to make builds more cacheable and if these paths are used throughout Bazel and not just on the level of Spawns, then cquery would no longer be the right tool to show output paths since knowing the paths would require executing (or recalling) a partial build.

Historically, users rely on exact path patterns (like looking at what bazel build produced and copying the exact path into downstream tools). That's the complete opposite of what we want. The only alternative is a canonical API.

Fully agreed. It's not completely uncommon to see ST hashes on the build outputs of top-level targets, at which point relying on exact paths is no longer reliable.

I agree IdentifyingName: output would be a more powerful API

Could you explain what you envision as IdentifyingName here? Would it contain more information than just the target (or possibly owner) label?

For now, is it reasonable to output a raw list? Or a dictionary where every key is empty to offer forward-facing structure for consuming tools?

I don't have a strong opinion here. When in doubt, I would prefer an output format that plays well with other Unix tools, given that this command will be a common interface to the non-Bazel world. There is jq however and it wouldn't be too uncommon to require it (or commit to emitting canonically indented JSON, which could also be parsed with grep in a pinch).

Conceptually we could still implement the logic in Starlark, if there was some way to bundle that behind the canonical API. But I don't think we have the infrastructure for that. And, again, I think working cleanly, out of the box, without requiring build expertise is a powerful argument.

I could implement --output=files as a thin wrapper around --output=starlark pointing at a shipped Starlark file, but for that would have to:

Parse the value of the --output_groups option in Starlark, duplicating the parsing logic.
Duplicate the filtering logic used in BuildResultPrinter.

That would probably defeat the purpose of decoupling this logic from core Bazel.

gregestren · 2022-05-25T18:59:52Z

I agree IdentifyingName: output would be a more powerful API

Could you explain what you envision as IdentifyingName here? Would it contain more information than just the target (or possibly owner) label?

Instead of:

$ bazel cquery --output=files '//tt:arm' 
bazel-out/arm-fastbuild-ST-7d684b6a9cec/bin/tt/libcb.a
bazel-out/arm--fastbuild-ST-7d684b6a9cec/bin/tt/libcb.so
bazel-out/arm--fastbuild-ST-7d684b6a9cec/bin/tt/libcb.ifso
bazel-out/arm--fastbuild-ST-7d684b6a9cec/bin/tt/arm.manifest

Something like:

$ bazel cquery --output=files '//tt:arm' 
StaticLibrary: bazel-out/arm-fastbuild-ST-7d684b6a9cec/bin/tt/libcb.a
DynamicLibrary: bazel-out/arm--fastbuild-ST-7d684b6a9cec/bin/tt/libcb.so
InterfaceSharedObjects: bazel-out/arm--fastbuild-ST-7d684b6a9cec/bin/tt/libcb.ifso
ManifestFile: bazel-out/arm--fastbuild-ST-7d684b6a9cec/bin/tt/arm.manifest

To be clear, I'm not asking you to implement this in this PR. I'm just thinking about a forward-facing API.

gregestren · 2022-06-13T23:07:21Z

Broadly speaking, and after more discussion with @brandjon , I support the simplest syntax:

$ bazel cquery --output=files '//tt:arm' 
bazel-out/arm-fastbuild-ST-7d684b6a9cec/bin/tt/libcb.a
...

which you're already doing.

The more complex structures quickly get speculative, where the core need is just a simple, easy API. Any extra syntax we add now would make future API changes that much more complicated if we get it wrong. @brandjon had some interesting ideas about separating the content request (outputs vs. BUILD vs. ...) with the format (JSON vs. plain vs. ...).

Now's not the time to get caught up in the subtleties.

gregestren · 2022-06-13T23:09:36Z

Stated differently, lacking more experience with this, I think it's fair to assume users know which targets they're requesting outputs for. So we don't have to preemptively add API features to contextualize.

With the new output mode `--output=files`, cquery lists all files advertised by the matched targets in the currently requested output groups. This new mode has the following advantages over `--output=starlark` combined with an appropriate handcrafted `--starlark:expr`: * provides a canonical answer to the very common "Where are my build outputs?" question * is more friendly to new users as it doesn't require knowing about providers and non-BUILD dialect Starlark * takes the value of `--output_groups` into account * stays as close to the logic for build summaries printed by `bazel build` as possible Fixes bazelbuild#8739 RELNOTES: `cquery`'s new output mode [`--output=files`](https://bazel.build/docs/cquery#files-output) lists the output files of the targets matching the query. It takes the current value of `--output_groups` into account. Closes bazelbuild#15552. PiperOrigin-RevId: 462630629 Change-Id: Ic648f22aa160ee57b476180561b444f08799ebb6

ckolli5 · 2022-07-25T19:56:59Z

Hey @fmeum, we are trying to cherry pick this change to release-5.3.0. But presubmit checks are failing. Could you please help us in cherry-picking this PR to release-5.3.0? Thanks!

gregestren · 2022-07-25T19:59:40Z

That might have been a merge discrepancy. I think it's just a matter of the right dependency in a BUILD file being added.

gregestren · 2022-07-25T20:27:05Z

There's some kind of baseline discrepancy.

For example, for ProtoOutputFormatterCallback, this commit only adjusts the line

super(eventHandler, options, out, skyframeExecutor, accessor, /*uniquifyResults=*/ false);

But the cherrypick also includes

 OutputType outputType,
 @Nullable TransitionFactory<RuleTransitionData> trimmingTransitionFactory) {

which was added by 9994c32.

aiuto · 2022-07-26T00:18:51Z

It should usually be the job of the original PR author to create a PR that backports a feature to the previous release. The release team is not responsible for figuring out a complex PR when it is on a single patch.

…

On Mon, Jul 25, 2022 at 4:27 PM Greg ***@***.***> wrote: There's some kind of baseline discrepancy. For example, for ProtoOutputFormatterCallback, this commit <681f534#diff-4f1f5a5f095fc900af3c7a6ce9217da68db4c1d2b3b23644a0a38ff6acf2adaf> only adjusts the line super(eventHandler, options, out, skyframeExecutor, accessor, /*uniquifyResults=*/ false); But the cherrypick <https://github.com/bazelbuild/bazel/pull/15955/files#diff-4f1f5a5f095fc900af3c7a6ce9217da68db4c1d2b3b23644a0a38ff6acf2adaf> also includes OutputType outputType, @nullable TransitionFactory<RuleTransitionData> trimmingTransitionFactory) { which was added by 9994c32 <9994c32> . — Reply to this email directly, view it on GitHub <#15552 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAXHHHFEXGYBLOGBEB6DSJDVV32CJANCNFSM5WVWQ6XA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

With the new output mode `--output=files`, cquery lists all files advertised by the matched targets in the currently requested output groups. This new mode has the following advantages over `--output=starlark` combined with an appropriate handcrafted `--starlark:expr`: * provides a canonical answer to the very common "Where are my build outputs?" question * is more friendly to new users as it doesn't require knowing about providers and non-BUILD dialect Starlark * takes the value of `--output_groups` into account * stays as close to the logic for build summaries printed by `bazel build` as possible Fixes bazelbuild#8739 RELNOTES: `cquery`'s new output mode [`--output=files`](https://bazel.build/docs/cquery#files-output) lists the output files of the targets matching the query. It takes the current value of `--output_groups` into account. Closes bazelbuild#15552. PiperOrigin-RevId: 462630629 Change-Id: Ic648f22aa160ee57b476180561b444f08799ebb6

fmeum · 2022-07-26T06:33:08Z

I submitted #15979 with the merge conflicts resolved.

With the new output mode `--output=files`, cquery lists all files advertised by the matched targets in the currently requested output groups. This new mode has the following advantages over `--output=starlark` combined with an appropriate handcrafted `--starlark:expr`: * provides a canonical answer to the very common "Where are my build outputs?" question * is more friendly to new users as it doesn't require knowing about providers and non-BUILD dialect Starlark * takes the value of `--output_groups` into account * stays as close to the logic for build summaries printed by `bazel build` as possible Fixes #8739 RELNOTES: `cquery`'s new output mode [`--output=files`](https://bazel.build/docs/cquery#files-output) lists the output files of the targets matching the query. It takes the current value of `--output_groups` into account. Closes #15552. PiperOrigin-RevId: 462630629 Change-Id: Ic648f22aa160ee57b476180561b444f08799ebb6

Baseline: 8d66a41 Cherry picks: + becd149: Remote: Cache merkle trees + d7628e1: Update DEFAULT_IOS_CPU for M1 arm64 simulator support + 80c56ff: Compile Apple tools as fat binaries if possible + 3c09f34: Add protobuf as a well known module + 3a5b360: Remote: Merge target-level exec_properties with --remote_default_exec_properties + 917e15e: Add -no_uuid for hermetic macOS toolchain setup + f5cf8b0: Remote: Fixes an issue when --experimental_remote_cache_async encounter flaky tests. + 77a002c: Remove DigestUtils.getDigestInExclusiveMode() now that SsdModule has … + 557a7e7: Fixes for the Starlark transition hash computation (#14251) + 34c7146: Do location expansion in copts of objc_library + 50274a9: [5.x] Remote: Add support for compression on gRPC cache (#14277) + 61bf2e5: Automated rollback of commit 34c7146. + 79888fe: Silence a zstd-jni GCC warning. + 063b5c9: Remote: Limit max number of gRPC connections by --remote_max_connections. + fd727ec: Do location expansion in copts of objc_library + 23d0969: Fix _is_shared_library_extension_valid + 5cf1d6e: Remove merging of java_outputs in JavaPluginInfo. + cea5f4f: Cherrypick Bzlmod documentation (#14301) + 227e49e: Format work requests according to ndjson spec + ae0a6c9: Enable user_link_flags_feature for macosx cc_toolchain_config + 8c2c78c: Remote: Use Action's salt field to differentiate cache across workspaces. + f948989: [5.x] Remote: Fix "file not found" error when remote cache is changed from enabled to disabled. (#14321) + 3069ac4: Delete marker file before fetching an external repository + c05c626: Remote: Fix file counting in merkletree.DirectoryTreeBuilder + d84f799: Fix remote spawn tests for remote_merkle_tree_cache=true + 59e16e9: Show skipped tests as a warning + 76b3c24: Build xcode-locator as a universal binary + aa52f2d: Exit collect_coverage.sh early if LCOV_MERGER is not set. + 4256d46: Automated rollback of commit d84f799. + dce2435: [apple] fix issues compiling C in objc_library for watchos/armv7k + bfc2413: 5.x: Remote: Ignore blobs referenced in BEP if the generating action cannot be cached remotely. (#14389) + 5aef53a: Remote: Don't blocking-get when acquiring gRPC connections. (#14420) + 005361c: Disable IncludeValidation for ObjC in bazel + d703b7b: Update java_tools v11.6 + 90965b0: Stop remote blob upload if upload is complete. (#14467) + dc59d9e: [5.x] Make remote BES uploader better (#14472) + 2edab73: Avoid too verbose warnings in terminal when cache issues + 1160485: Rename --project_id to --bes_instance_name + c63d9ec: Automated rollback of commit bfdfa6e. + b341802: [apple] support watchos_arm64 in toolchain + 43bcf80: Disable implicitly collecting baseline coverage for toolchain targets. + 302971e: Automated rollback of commit 7d09b4a. + 6200202: Bzlmod: Starlarkify default attr values for TypeCheckedTags + 38117d4: Fix build after rc4 cherrypicks (#14581) + 41feb61: Release 5.0.0 (2022-01-19) + 486d153: Find runfiles in directories that are themselves runfiles + 0de7bb9: Don't resolve symlinks for --sandbox_base + 8b60c90: Remove uses of -lstdc++ on darwin + 60f757c: Allow Label instances as keys in select (#14755) + 3836ad0: Remote: Only waits for background tasks from remote execution. + 8734ccf: Add the default solib dir to the rpath for cc_imports with transitions + 9e16a64: Flip --experimental_worker_allow_json_protocol + fce7ea8: Fix `ctx.fragments.apple.single_arch_cpu` returning incorrect cpu for tools when host cpu and exec cpu are different + 0c1d09e: Propagate --experimental_cc_implementation_deps to host config + 1c3a245: Support select() on constraint_value for aliases. + 67a133b: Improve documentation for select() + 5356fed: Cherrypicks for experimental cc_shared_library (#14773) + ffdd633: [apple] support tvos_sim_arm64 in toolchain (#14779) + a58ddea: Cherry pick win arm64 (#14794) + dc41a20: [5.1.0] cherrypick subpackages support (#14780) + 86e2db7: Add a helper method for rules to depend on the cpp toolchain type. + 6990c02: UrlRewriter should be able to load credentials from .netrc (#14834) + 32d1606: Add "arch" struct field to repository_os + 2cfdcea: [5.x] bzlmod: Add support for WORKSPACE.bzlmod (#14813) + c2ddbd1: Ignore missing include directory in JDK distribution. + 16de035: Fix bazel coverage false negative + 0c74741: Remote: Postpone the block waiting in `afterCommand` to `BlockWaitingModule` (#14833) + 3297d92: Switch to `ProcessHandle` for getting the PID (#14842) + a987b98: Fix uses of std++ on bsd + d184e48: Remote: handle early return of compressed blobs uploads + 0b09e9e: Add removeprefix/removesuffix to Starlark strings + d42ab0c: Fix default CPU for macOS and iOS (#14923) + cd24f39: Add paramfile support for def_parser, since in rare cases on Windows command line character limit was reached. + 0b1beef: Normalize rpath entries to guard against missing default solib dir + 24e8242: Fix aggressive params file assumption + c45838b: Fix precompiled libs not in runfiles of cc_shared_library (#14943) + 764614e: Bzlmod: Allow multiple `use_extension`s on the same extension (#14945) + fa761f8: Fix typo in `apple_common.platform` docs + f7d8288: Yield a Proxy for addresses without protocol + 8cefb8b: Avoid merging URLs in HttpUtils + b480480: Make protocOpts() public. (#14952) + 113eaca: Do not hide BulkTransferException messages when there were more than one exception + b1bf9d6: merkle_tree_cache: change default size to 1000 + f15e0c7: Add --experimental_repository_cache_urls_as_default_canonical_id to help detect broken repository URLs (#14989) + f421474: Expose the logic to read user netrc file + b858ec3: Correct cpu and os values of `local_config_cc_toolchains` targets + 5e79972: Expose CoverageOutputGenerator on a Fragment (#14997) + 78f0311: Correct error runfiles cc_shared_library (#14998) + 7937dd1: [5.1] Adding Starlark dependencies to the package //external (#14991) + a73aa12: Remote: Fix crashes with InterruptedException when using http cache. + f8707c0: Account for interface libraries in cc_shared_library + a570f5f: Fix coverage runfiles directory issue + 95de355: Do not validate input-only settings in transitions (#15048) + 71747cc: Filter out system headers on macOS. + cb6500a: Update Bazel bootstrap documentation and remove obsolete flags. (#15065) + 4c031d1: [5.1] Undocument --bes_best_effort (#15066) + 267142f: Fix conflicting actions error when specifying --host_macos_minimum_os (#15068) + f192362: [5.1] Remote: Action should not be successful and cached if outputs were not created (#15071) + 00d74ff: Support decompressing zstd tar archives for repository rules. + f585783: Remote: Don't check TreeArtifact output + efb2b80: osx_cc_wrapper: Only expand existing response files + c771c43: Remote: Fix crashes by InterruptedException when dynamic execution is enabled. (#15091) + 3785677: Use python3 on macOS + 815d9e4: Release 5.1.0 (2022-03-24) + 1fbb69e: Prepare 5.1.1 release + df153df: Fix CODEOWNERS syntax + 2b92a31: Remote: Don't check declared outputs for failed action + b47aa71: Upgrade abseil version to the latest + c49c45d: Revert default export all symbols on Windows + 7d3fb99: Support ZIP files with total number of disks = 0 + 0f5dc11: Release 5.1.1 (2022-04-08) + 2422cfb: Update CODEOWNERS + bbcff18: [5.2.0] Update java_tools 11.7.1 (#15231) + 9c98120: Add support for .ar archives (and .deb files) + d3435b0: Seperate GetSelfPath implementation for Blaze and Bazel + c94572b: Include jdk.crypto.mscapi in minimized Windows embedded JDK + 299022c: remote: Proactively close the ZstdInputStream in ZstdDecompressingOutputStream. + 2770799: Collect coverage from cc_binary data deps of java_test + 3442179: Configure Apple crosstool to return a complete target triple that includes minimum OS version and target environment + bb6f1a7: Collect C++ lcov coverage if runtime object not in runfiles + dbb6e99: Fixing dependencies of //external package + f0213bb: [5.2] Upgrade Google Auth Version (#15383) + a1a74c9: Fix chocolatey package - docsUrl must not 404 (#15395) + fe644be: Fix cache leak when applying transitions when only a rule's attributes change. + ad74d52: Fix checking remote cache for omitted files in buildevent file (#15405) + ac21910: fix(bzlmod): throw on json parse exception + 3d85b88: Add a flag to expose undeclared test outputs in unzipped form. (#15431) + abd7a9f: Remove -U_FORTIFY_SOURCE when thin_lto is enabled (#15433) + 53b9cb8: Catch NumberFormatException while trying to parse thread id. + 19740b5: Improve the --sandbox_debug error message + 0a2a43f: Set keywords on appropriate lifecycle events. + 394ddb8: Record additional profiling information for remotely executed actions. + 652b48e: Fix downloading remote execution output files inside output dirs. (#15444) + 73f1ecb: Fix android emulator darwin_arm64 select + 2649c7c: Fix --use_top_level_targets_for_symlinks with aliases (#15446) + fa1081c: Filter libtool warning about table of contents + 26f8783: Unify sandbox/remote handling of empty TreeArtifact inputs (#15449) + 6b21b77: Revert "Fixes incorrect install names on darwin platforms" + e133e66: config doesn't error on duplicate `--define` values (#15473) + 84d5917: Collect coverage from cc_binary data deps of py_test (#15298) + 519d2da: SolibSymlinkAction does not need exec platform or properties + 6e54699: Let Starlark tests inherit env variables (#15217) + 9610ae8: Update PythonZipper action to use CommandLineItem.CapturingMapFn + 2f1ff6f: Make `coverage --combined_report=lcov` skip incompatible tests + 9fad5a3: Disable ReturnValueIgnored checks to unblock java_tools release + 0120118: Bump the limit of Bazel install base size (#15585) + 668805a: Upgrade zlib to 1.2.12 + 4d900ce: [5.2] Remote: Fix a bug that outputs of actions tagged with no-remote are u... (#15453) + b703cb9: Add feature to produce serialized diagnostics files (#15600) + 2e8458b: Release 5.2.0 (2022-06-07) + 536f8d9: Fix fail message construction in cc_shared_library + 2d42925: Define cc-compiler-darwin in Xcode toolchain + a1d7d1f: Fix alwayslink in objc_import + d273cb6: Unify URL/URLs parameter code across http_archive, http_file, http_jar + fea32be: Preserve --experimental_allow_unresolved_symlinks in exec cfg + e4bc370: Ck/cherry pick cc shared library (#15754) + dbdfa07: Let Starlark executable rules specify their environment (#15766) + e2a6a2b: Fix string formatting when java_home path is missing. + d54a288: Optionally enable LLVM profile continuous mode + ad17b44: Print remote execution message when the action times out (#15772) + 240e3d1: Add missing line to cherrypick e4bc370 (#15784) + 804b474: Replace strdupa with strdup + 62be9ea: Bzlmod: Better canonical repo names for modules with overrides (#15793) + d4663a1: Add repo env test (#15768) + 594962c: Add is_root struct field to bazel_module (#15815) + 3dd2b93: Fix null pointer crash with `bazel coverage` on only incompatible tests + 4175018: Add util for finding credential helper to use + 3ea9eb2: Merge ManifestMergerAction-related commits into release-5.3.0 (#15824) + 64571a4: Ck/cherrypick 15669 (#15788) + 1404651: Create output directories for remote execution (#15818) + ae523f8: Use tree artifacts in bootclasspath rule + 37f181c: [credentialhelper] Add types to communicate with the subprocess + 06ca634: Add a flag to force Bazel to download certain artifacts when using --remote_download_minimal (#15870) + d35f923: RemoteExecutionService: fix outputs not being uploaded + 78af34f: Cherry-pick proto_lang_toolchain Starlarkfication and proto_common module (#15854) + afb434d: Fix behavior of `print()` in module extensions + 6714c30: [credentialhelper] Implement invoking credential helper as subprocess + 0f05904: Add register_{execution_platforms,toolchains} directives to MODULE.bazel files (#15852) + 33516e2: [remote] Improve .netrc test in RemoteModuleTest + aa2a1f3: Fix ZipDecompressor windows 0x80 (file attribute normal) + 30f16e5: Replace uses of `cfg = "host"` with `cfg = "exec"` (#15922) + 2a8d0ad: target pattern file: allow comments + 6f73205: Add factory for creating paths relative to well-known roots (#15931) + 32cc8e6: Update CODEOWNERS (#15910) + 63bc14b: Implement native analysis_test call. (#15940) + 4df77f7: Increase osx_cc_configure timeouts + cdf01a3: Allow string_list flags to be set via repeated flag uses + 05e758d: [credentialhelper] Add parser for flag syntax (#15929) + e4ee344: Docs should mention the new no-remote-cache-upload tag (#15965) + 96d23d3: Add netrc support to --bes_backend (#15970) + c5bc34e: Add CommandLinePathFactory to CommandEnvironment (#15971) + 508f185: Move newCredentialHelperProvider into GoogleAuthUtils (#15973) + 14c944a: Wire up credential helper to command-line flag(s) (#15976) + 04c373b: Add `--output=files` mode to cquery (#15979) + edfe2a1: Make cpp assembly file extensions case sensitive again + 4ae8538: Prevent aspects from executing on incompatible targets (#15984) + f440f8e: Remote: Fix performance regression in "upload missing inputs". (#15998) + 0109031: Updated Codeowners file (#16032) + 6102d33: Propagate the error message when a credential helper fails. (#16030) + a8dacc7: Migrate legacy desugar wrapper to new rlocation() (#16025) + 11368be: Correctly report errors thrown by CommandLinePathFactory#create. + 82452c7: Fix an issue that `incompatible_remote_build_event_upload_respect_no_… (#16045) + e745468: Fix rpath for binaries in external repositories (#16079) + 83041b1: Refactor combined cache. (#16110) + c62496f: C++: Add compound error linked statically but not exported (#16113) Important changes: - alias() can now select() directly on constraint_value() Fixes #13047. Closes #14310. - Fixed an issue where Bazel could erroneously report a test passes in coverage mode without actually running the test. - Make protocOpts() publicly accessible. - Add coverage configuration fragment, used to expose output_generator label. - Bazel now no longer includes system headers on macOS in coverage reports (#14969). Closes #14971. - Starlark test rules can use the new inherited_environment parameter of testing.TestEnvironment to specify environment variables whose values should be inherited from the shell environment. Closes #14849. - none RELNOTES:none - Enable merging permissions during Android manifest merging with the --merge_android_manifest_permissions flag. - Added new register_{execution_platforms,toolchains} directives to the MODULE.bazel file, to replace the {execution_platforms,toolchains}_to_register attributes on the module() directive. - Add support for fetching RPC credentials from credential helper. Progress on #15856 Closes #15947. - `cquery`'s new output mode [`--output=files`](https://bazel.build/docs/cquery#files-output) lists the output files of the targets matching the query. It takes the current value of `--output_groups` into account. Closes #15552. - Fix for desugaring failure on Bazel+Android+Windows build scenario. This release contains contributions from many people at Google, as well as amberdixon, Andreas Fuchs, Benjamin Peterson, Brentley Jones, Chenchu Kolli, Dan Fleming, Danny Wolf, Emil Kattainen, Fabian Meumertzheim, Gowroji Sunil, hvadehra, Juh-Roch, Keith Smiley, kshyanashree, Niyas Sait, Noa Resare, Oliver Eikemeier, oquenchil, Peter Mounce, Philipp Schrader, Thi Doãn, Xùdōng Yáng, Yannic, Zhongpeng Lin.

Extract build output display logic out of BuildResultPrinter

00b293f

fmeum requested review from fweikert and gregestren as code owners May 23, 2022 12:36

fmeum mentioned this pull request May 23, 2022

Obtaining output paths of the build artifacts #8739

Closed

fmeum changed the title ~~Add "files" output mode to cquery~~ Add --output=files mode to cquery May 23, 2022

fmeum added 2 commits May 23, 2022 15:23

Add output mode "files" to cquery

cf95f0e

Uniquify results of FilesOutputFormatterCallback

a986ca1

fmeum force-pushed the cquery-output-files branch from f37b578 to a986ca1 Compare May 23, 2022 13:23

aiuto requested review from aiuto and removed request for fweikert May 23, 2022 14:09

aiuto added the team-Configurability platforms, toolchains, cquery, select(), config transitions label May 23, 2022

gregestren reviewed May 24, 2022

View reviewed changes

Comment thread site/en/docs/cquery.md

gregestren self-assigned this Jun 7, 2022

gregestren reviewed Jun 13, 2022

View reviewed changes

Address review comments

81da138

fmeum force-pushed the cquery-output-files branch from 546a20a to 81da138 Compare June 30, 2022 12:37

ckolli5 mentioned this pull request Jul 22, 2022

Add --output=files mode to cquery #15955

Closed

fmeum deleted the cquery-output-files branch July 26, 2022 06:07

fmeum mentioned this pull request Jul 26, 2022

Add --output=files mode to cquery #15979

Merged

copybara-service Bot pushed a commit that referenced this pull request Aug 22, 2022

Conversation

fmeum commented May 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aiuto commented May 23, 2022

Uh oh!

fmeum commented May 23, 2022

Uh oh!

gregestren commented May 23, 2022

Uh oh!

pcjanzen commented May 23, 2022

Uh oh!

gregestren commented May 23, 2022

Uh oh!

aiuto commented May 23, 2022

Uh oh!

gregestren commented May 24, 2022

Uh oh!

gregestren commented May 24, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

fmeum commented May 25, 2022

Uh oh!

fmeum commented May 25, 2022

Uh oh!

fmeum commented May 25, 2022

Uh oh!

gregestren commented May 25, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gregestren commented Jun 13, 2022

Uh oh!

gregestren commented Jun 13, 2022

Uh oh!

ckolli5 commented Jul 25, 2022

Uh oh!

gregestren commented Jul 25, 2022

Uh oh!

gregestren commented Jul 25, 2022

Uh oh!

aiuto commented Jul 26, 2022 via email

Uh oh!

fmeum commented Jul 26, 2022

Uh oh!

fmeum commented May 23, 2022 •

edited

Loading

gregestren commented May 24, 2022 •

edited

Loading