Skip to content

Add short_uncached and detailed_uncached options to --test_summary#28290

Closed
Silic0nS0ldier wants to merge 3 commits intobazelbuild:masterfrom
Silic0nS0ldier:uncached-test-summary-options
Closed

Add short_uncached and detailed_uncached options to --test_summary#28290
Silic0nS0ldier wants to merge 3 commits intobazelbuild:masterfrom
Silic0nS0ldier:uncached-test-summary-options

Conversation

@Silic0nS0ldier
Copy link
Contributor

@Silic0nS0ldier Silic0nS0ldier commented Jan 14, 2026

These options exclude cached test results from the summary.

With short (default)

vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.770s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
//src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (cached) PASSED in 2.8s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

With short_uncached

vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed --test_summary=short_uncached
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.264s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.

Resolves #28062

RELNOTES: Reporting of cached test results can now be suppressed with --test_summary=short_uncached or --test_summary=detailed_uncached.

@Silic0nS0ldier Silic0nS0ldier requested a review from a team as a code owner January 14, 2026 04:06
@Silic0nS0ldier Silic0nS0ldier requested review from mai93 and removed request for a team January 14, 2026 04:06
@github-actions github-actions bot added team-Configurability platforms, toolchains, cquery, select(), config transitions awaiting-review PR is awaiting review from an assigned reviewer labels Jan 14, 2026
@Silic0nS0ldier Silic0nS0ldier mentioned this pull request Jan 14, 2026
Closed
gemini-code-assist[bot]
gemini-code-assist bot reviewed Jan 14, 2026
View reviewed changes
Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces short_uncached and detailed_uncached options for --test_summary to exclude cached test results, which is a useful feature. The implementation is solid, and I particularly appreciate the refactoring in TerminalTestResultNotifier to use EnumSets for clarity and the significant improvements to TerminalTestResultNotifierTest for better test isolation and readability. The new shell tests are also a great addition. I've left a couple of suggestions to improve the new unit tests to make them more robust.

@Silic0nS0ldier
Copy link
Contributor Author

Silic0nS0ldier commented Jan 14, 2026 •
edited
Loading

Motivation for this change is to better support using bazel test //... for CI checks. In large repositories the summary can be enormous (almost 2MB in our case).

Excessively large output means more scrolling and in the case of Buildkite, can result in the top of logs being cutoff due to 2MB limit for terminal rendering (there is a separate log-only page with a higher limit).

@Silic0nS0ldier Silic0nS0ldier changed the title Adds short_uncached and detailed_uncached options to --test_summary Add short_uncached and detailed_uncached options to --test_summary Jan 14, 2026
@brentleyjones
Copy link
Contributor

brentleyjones commented Jan 14, 2026

I would love this change! It would be nice to get into 8.6 as well if possible.

@fmeum fmeum added team-Core Skyframe, bazel query, BEP, options parsing, bazelrc and removed team-Configurability platforms, toolchains, cquery, select(), config transitions labels Jan 14, 2026
@fmeum fmeum requested review from meisterT and removed request for mai93 January 14, 2026 16:48
meisterT
meisterT reviewed Jan 15, 2026
View reviewed changes
src/test/java/com/google/devtools/build/lib/runtime/TerminalTestResultNotifierTest.java Outdated

private void printFailedToBuildSummaries() throws LabelSyntaxException {
// A record that is used to generate a TestSummary mock object for testing.
private static record TestSummarySpec(
Copy link
Member

@meisterT meisterT Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a builder pattern (basically promoting comments from above to actual methods)

Copy link
Contributor Author

@Silic0nS0ldier Silic0nS0ldier Jan 15, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored. It's a shame records don't implement this kind of pattern (or something similar like named arguments).

@Silic0nS0ldier Silic0nS0ldier force-pushed the uncached-test-summary-options branch from a91854e to 31516c5 Compare January 15, 2026 13:46
@meisterT meisterT added awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally and removed awaiting-review PR is awaiting review from an assigned reviewer labels Jan 15, 2026
@github-actions github-actions bot removed the awaiting-PR-merge PR has been approved by a reviewer and is ready to be merge internally label Jan 16, 2026
@brentleyjones
Copy link
Contributor

brentleyjones commented Jan 16, 2026

@Silic0nS0ldier How easy would this be to get into 8.x and 9.0?

@Silic0nS0ldier
Copy link
Contributor Author

Silic0nS0ldier commented Jan 17, 2026

I don't think much has changed in the areas touched across theose versions, so backporting should be easy.

@Silic0nS0ldier
Copy link
Contributor Author

Silic0nS0ldier commented Jan 20, 2026

@meisterT Any reservations about backporting this to v9 and v8.6?

@meisterT
Copy link
Member

meisterT commented Jan 20, 2026

No reservations, please go ahead and assign me as reviewer

@Silic0nS0ldier
Copy link
Contributor Author

Silic0nS0ldier commented Jan 20, 2026

@bazel-io fork 9.0.0

@Silic0nS0ldier
Copy link
Contributor Author

Silic0nS0ldier commented Jan 20, 2026

Yeah, I figured that required maintainer permission minimum. I'll get it done manually.

Silic0nS0ldier added a commit to Silic0nS0ldier/bazel that referenced this pull request Jan 20, 2026
@Silic0nS0ldier
Add short_uncached and detailed_uncached options to `--test_summa…
66ef65d 
…ry` (bazelbuild#28290)

These options exclude cached test results from the summary.

With `short` (default)
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.770s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
//src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (cached) PASSED in 2.8s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

With `short_uncached`
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed --test_summary=short_uncached
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.264s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

Resolves bazelbuild#28062

RELNOTES: Reporting of cached test results can now be suppressed with `--test_summary=short_uncached` or `--test_summary=detailed_uncached`.

Closes bazelbuild#28290.

PiperOrigin-RevId: 857282700
Change-Id: Iaea823462344a6118bd5a112d734df5c35a1e152
@Silic0nS0ldier Silic0nS0ldier mentioned this pull request Jan 20, 2026
Merged
Silic0nS0ldier added a commit to Silic0nS0ldier/bazel that referenced this pull request Jan 20, 2026
@Silic0nS0ldier
Add short_uncached and detailed_uncached options to `--test_summa…
633e3b1 
…ry` (bazelbuild#28290)

These options exclude cached test results from the summary.

With `short` (default)
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.770s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
//src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (cached) PASSED in 2.8s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

With `short_uncached`
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed --test_summary=short_uncached
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.264s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

Resolves bazelbuild#28062

RELNOTES: Reporting of cached test results can now be suppressed with `--test_summary=short_uncached` or `--test_summary=detailed_uncached`.

Closes bazelbuild#28290.

PiperOrigin-RevId: 857282700
Change-Id: Iaea823462344a6118bd5a112d734df5c35a1e152
@Silic0nS0ldier Silic0nS0ldier mentioned this pull request Jan 20, 2026
Merged
@Silic0nS0ldier
Copy link
Contributor Author

Silic0nS0ldier commented Jan 20, 2026

@meisterT PRs raised, but I lack the permission necessary to assign you as a reviewer.

github-merge-queue bot pushed a commit that referenced this pull request Jan 20, 2026
@Silic0nS0ldier
[8.6.0] Add short_uncached and detailed_uncached options to `--te…
88832ab 
…st_summary` (#28343)

These options exclude cached test results from the summary.

With `short` (default)
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.770s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
//src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (cached) PASSED in 2.8s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

With `short_uncached`
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed --test_summary=short_uncached
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.264s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

Resolves #28062

RELNOTES: Reporting of cached test results can now be suppressed with
`--test_summary=short_uncached` or `--test_summary=detailed_uncached`.

Backport of #28290.
iancha1992 pushed a commit that referenced this pull request Jan 22, 2026
@Silic0nS0ldier
[9.1.0] Add short_uncached and detailed_uncached options to `--te…
87aad18 
…st_summary` (#28341)

These options exclude cached test results from the summary.

With `short` (default)
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.770s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
//src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (cached) PASSED in 2.8s

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

With `short_uncached`
```
vscode ➜ /workspaces/bazel/bazel (uncached-test-summary-options) $ bazel-bin/src/bazel test //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests --test_filter=shortUncachedOption_allPassed --test_summary=short_uncached
INFO: Analyzed target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/java/com/google/devtools/build/lib/runtime:RuntimeTests up-to-date:
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests
  bazel-bin/src/test/java/com/google/devtools/build/lib/runtime/RuntimeTests.jar
INFO: Elapsed time: 0.264s, Critical Path: 0.00s
INFO: 1 process: 1 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action

Executed 0 out of 1 test: 1 test passes.
There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
```

Resolves #28062

RELNOTES: Reporting of cached test results can now be suppressed with
`--test_summary=short_uncached` or `--test_summary=detailed_uncached`.

Backport of #28290.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@meisterT meisterT meisterT approved these changes

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

team-Core Skyframe, bazel query, BEP, options parsing, bazelrc

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uncached-only test_summary option

4 participants

@Silic0nS0ldier @brentleyjones @meisterT @fmeum