Skip to content

[checks] Collapse nested subjects in condition descriptions - #2784

Merged
natebosch merged 4 commits into
masterfrom
collapse-describe
Oct 12, 2026
Merged

natebosch merged 4 commits into
masterfrom
collapse-describe

Conversation

@natebosch

@natebosch natebosch commented Oct 8, 2026 •

Copy link
Copy Markdown
Member

Failure messages collapse a nested subject with a single expectation to one
line, for example has length: <1>, but Condition.describe always used the
expanded multi-line form. Expectations which embed the description of a
condition, such as IterableChecks.any, showed every nested subject
expanded even where the same check made directly would collapse:

Expected: a List<Baz> that:
  contains a value that:
    has qux that:
      equals 'expected'

There were two causes in _TestContext._fullDetails:

  • Passing nested clauses were only collapsed when formatting a rejection, and
    describe never has one. Collapse them without a rejection as well.
  • describe passes the root context as the failing context, so the last root
    clause was treated as the failing clause and always expanded. Only treat a
    clause as failing when there is a rejection.

Collapsing without a rejection also affects ordinary failure messages. A
passing sibling with multiple expectations is formatted without a rejection,
so its own nested subjects now collapse like those at any other depth, for
example has length: <3> instead of has length that: followed by
equals <3>.

Collapsing skips the guard clauses added by atSameLevel nesting, such as
isA<T>() and isNotNull(), because a guard that let its value through is a
satisfied precondition. Within describe there is no value, so a guard is
never evaluated. Record on each guard whether its value was extracted, and
only skip satisfied guards. An unevaluated or rejected guard counts as an
expectation, so a described condition keeps the precondition:

Expected: a List<Wrapper> that:
  contains a value that:
    has value that:
      is a Foo
      has someField: 'bar'

With descriptions collapsing, add a predicate noun to any, every,
containsKeyThat, and containsValueThat when the condition describes in a
single line, so these failures also collapse at the root:

Expected: an iterable containing a value that has qux: 'expected'
Actual: [Instance of 'Baz']
Which: contains no matching element

The predicate nouns use a new internal describedPredicateNoun helper in
describe.dart. It returns null when the condition describes over multiple
lines, or when the combined noun is longer than 80 characters, matching the
limit used when a collapsed description is joined to a root label. Those
expectations then keep the multi-line format.

@github-actions github-actions Bot added the package:checks Issues related to pkg:checks label Oct 8, 2026
@natebosch
natebosch added this pull request to stack #2785 October 8, 2026 22:57
@natebosch
natebosch marked this pull request as ready for review October 8, 2026 22:59
@natebosch
natebosch requested a review from a team as a code owner October 8, 2026 22:59
@natebosch
natebosch requested a review from davidmorgan October 8, 2026 22:59

@davidmorgan davidmorgan left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks.

FYI, the agents noticed two pre-existing issues, for you to do with as you like :)

  1. every crashes with Failed assertion: 'foundDepth == -1' (checks.dart _fullDetails) when a guard inside the condition rejects:
    check([Wrapper(123)]).every(.it()..value.isA().someField.equals('bar'))
    softCheckSync keeps adding clauses after isA rejects, so _clauses.last (someField) is treated as the failing clause on a context that itself failed.

  2. every's Which: drops the property name when the nested failure detail collapsed, because it reads failure.detail.actual.skip(1):
    check([Foo('foo')]).every(.it()..someField.equals('bar'))
    gives "Which: has an element at index 0 that: / Actual: 'foo'", without "has someField". StreamChecks.inOrder/anyOf check collapsedDetails instead.

Comment thread pkgs/checks/test/describe_test.dart Resolved
Comment thread pkgs/checks/lib/src/checks.dart Resolved
Comment thread pkgs/checks/lib/src/describe.dart Resolved
Base automatically changed from collapse-fix-has to master October 9, 2026 19:00
Failure messages collapse a nested subject with a single expectation to one
line, for example `has length: <1>`, but `Condition.describe` always used the
expanded multi-line form. Expectations which embed the description of a
condition, such as `IterableChecks.any`, showed every nested subject
expanded even where the same check made directly would collapse:

    Expected: a List<Baz> that:
      contains a value that:
        has qux that:
          equals 'expected'

There were two causes in `_TestContext._fullDetails`:

- Passing nested clauses were only collapsed when formatting a rejection, and
  `describe` never has one. Collapse them without a rejection as well.
- `describe` passes the root context as the failing context, so the last root
  clause was treated as the failing clause and always expanded. Only treat a
  clause as failing when there is a rejection.

With descriptions collapsing, add a predicate noun to `any`, `every`,
`containsKeyThat`, and `containsValueThat` when the condition describes in a
single line, so these failures also collapse at the root:

    Expected: an iterable containing a value that has qux: 'expected'
    Actual: [Instance of 'Baz']
    Which: contains no matching element

The predicate nouns use a new internal `singleLineDescription` helper in
`describe.dart`. Conditions which describe over multiple lines keep the
multi-line format.
@github-actions

github-actions Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

PR Health

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

This check can be disabled by tagging the PR with skip-changelog-check (for example, for non-user-visible changes when the package version is already at -wip).

Collapsing skips the guard clauses added by `atSameLevel` nesting, such as
`isA<T>()` or `isNotNull()`, because a guard that let its value through is a
satisfied precondition. Within `Condition.describe` there is no value, so a
guard is never evaluated and nothing is known to be satisfied. Skipping it
anyway dropped the precondition from the description:

    check([Wrapper(123)])
        .any(.it()..value.isA<Foo>().someField.equals('bar'));

    Expected: an iterable containing a value that has value that has
    someField: 'bar'

The failure is that 123 is not a `Foo`, but the description gave no hint of
the `isA` check.

Record on each guard clause whether its value was extracted, and only skip
satisfied guards when looking for a single clause to collapse. An
unevaluated or rejected guard counts as an expectation like any other, so
the description above keeps the full form:

    Expected: a List<Wrapper> that:
      contains a value that:
        has value that:
          is a Foo
          has someField: 'bar'

Collapsing through a guard that was satisfied when checking a real value is
unchanged.
Collapsing passing nested clauses without a rejection, so that
`Condition.describe` output collapses, also changes ordinary failure
messages. Previously only the clauses directly under a subject on the path
to the failure collapsed. A passing sibling with multiple expectations is
formatted without a rejection, so its own nested subjects kept the expanded
form:

    Expected: a Foo that:
      has someField that:
        has length that:
          equals <3>
        has length that:
          is less than <5>
      has someField: 'bar'

They now collapse like nested subjects at any other depth:

    Expected: a Foo that:
      has someField that:
        has length: <3>
        has length: a value < <5>
      has someField: 'bar'

Add a CHANGELOG entry for this change in failure output, and a test which
fails if passing clauses only collapse when there is a rejection.
When a collapsed description is joined to the label of a root subject, it
only stays on one line if the result fits in 80 columns. Otherwise the
message keeps the multi-line form. The predicate nouns for `any`, `every`,
`containsKeyThat`, and `containsValueThat` stand in for that join, but they
are leaf clauses, so the root limit never applied. A chain that splits after
`a Wrapper that:` when checked directly became a single 133 character
`Expected:` line when checked inside `any`.

Replace `singleLineDescription` with `describedPredicateNoun`, which builds
the whole noun from a prefix and the condition description, and returns
`null` when the result is longer than 80 characters. Those expectations then
fall back to the multi-line form:

    Expected: a List<Foo> that:
      contains a value that:
        has a long property name: 'a long expected value'
@natebosch
natebosch merged commit 0b6efee into master Oct 12, 2026
71 checks passed
@natebosch
natebosch deleted the collapse-describe branch October 12, 2026 19:16
copybara-service Bot pushed a commit to dart-lang/sdk that referenced this pull request Oct 13, 2026
Revisions updated by `dart tools/rev_sdk_deps.dart`.

dartdoc (https://github.com/dart-lang/dartdoc/compare/5eb0f2c..e1859cf):
  e1859cfc  Fri Oct 9 14:53:01 2026 +0200  Sarah Zakarias  Link a bare prefix only when it maps to exactly one library (dart-lang/dartdoc#4295)
  c48f6fb9  Fri Oct 9 14:25:42 2026 +0200  Sarah Zakarias  Resolve prefixes through the analyzer's PrefixElement.scopeLibraries (dart-lang/dartdoc#4294)
  3a6297e2  Wed Oct 7 12:56:14 2026 +0200  Sarah Zakarias  Require analyzer 14.5.0 and bump to 9.0.10 (dart-lang/dartdoc#4292)

ecosystem (https://github.com/dart-lang/ecosystem/compare/16dca7a..622ef61):
  622ef61  Mon Oct 12 00:23:27 2026 -0700  Kevin Moore  fix(firehose): enforce -wip on published versions and tighten changelog checks (dart-lang/ecosystem#467)
  5576a85  Wed Oct 7 09:29:44 2026 +0200  Moritz  Support use-flutter option in groundskeeper workflow (dart-lang/ecosystem#444)

http (https://github.com/dart-lang/http/compare/585d433..23f8574):
  23f8574  Thu Oct 8 23:27:25 2026 +0700  Cong Bang  feat(http2): configurable receive windows and frame size (dart-lang/http#1999)
  18051a6  Thu Oct 8 10:39:38 2026 +0200  Moritz  Enable Flutter in Groundskeeper workflow for Flutter packages (dart-lang/http#2004)

native (https://github.com/dart-lang/native/compare/a2a6179..10515af):
  10515afca  Mon Oct 12 12:21:52 2026 +0200  Daco Harkes  [native_toolchain_c] Do not pass linker flags when compiling static libraries (dart-lang/native#3731)

native is held back at 10515af, before dart-lang/native#3728, because it
breaks the dartdev `dart install` native assets tests. See
dart-lang/native#3743.

test (https://github.com/dart-lang/test/compare/d749b10..30acb69):
  30acb69e  Mon Oct 12 12:42:54 2026 -0700  Nate Bosch  Time out browser loads just before the suite load timeout (dart-lang/test#2786)
  597a0326  Mon Oct 12 12:22:53 2026 -0700  Kevin Moore  Fix misplaced changelog entries and remove no-op health check (dart-lang/test#2789)
  0b6efee8  Mon Oct 12 12:16:29 2026 -0700  Nate Bosch  [checks] Collapse nested subjects in condition descriptions (dart-lang/test#2784)
  17c98493  Mon Oct 12 12:08:23 2026 -0700  Nate Bosch  Follow symlinks in precompiled browser test directories (dart-lang/test#2787)
  1cc813ec  Fri Oct 9 11:59:56 2026 -0700  Nate Bosch  [checks] Join nested collapsed subjects with "that" (dart-lang/test#2783)
  ada7b950  Thu Oct 8 12:03:42 2026 -0700  Nate Bosch  Prepare to publish (dart-lang/test#2780)
  c4b165b3  Wed Oct 7 11:29:08 2026 -0700  Kevin Moore  [checks] Require package:meta ^1.17.0 (dart-lang/test#2779)
  4a363d70  Tue Oct 6 10:46:31 2026 -0700  Nate Bosch  Let Chrome choose its remote debugging port (dart-lang/test#2778)
  465d66ab  Tue Oct 6 09:35:22 2026 -0700  Nate Bosch  Keep Chrome stderr in the browser output when debugging (dart-lang/test#2777)

tools (https://github.com/dart-lang/tools/compare/a66d517..2b54c71):
  2b54c71af  Mon Oct 12 10:16:45 2026 +0200  dart-ecosystem-bot[bot]  [boring] Update BoringSSL to 3fed4f20 (dart-lang/tools#2655)
  2337f995a  Sat Oct 10 03:11:40 2026 +0300  Hasan Armoush  [json_rpc_2] Add strictProtocolChecks parameter to Client (dart-lang/tools#2555)
  c65c69ed1  Fri Oct 9 08:54:25 2026 +0200  Moritz  [boring] Move package:boring to dart-lang/tools (2/4): FFI helpers from package:webcrypto (dart-lang/tools#2615)
  a04e45491  Thu Oct 8 21:07:18 2026 -0700  Kevin Moore  chore: wire package:api_summary and package:string_scanner PR auto-labeler (dart-lang/tools#2652)
  8bdd4b92c  Thu Oct 8 09:20:48 2026 +0200  Morgan :)  [code_builder] Allow built_collection 6.x. (dart-lang/tools#2645)
  c16040d35  Wed Oct 7 17:30:08 2026 -0700  Kevin Moore  [api_summary] Sort primary barrel first and experimental entry points last (v1.2.0) (dart-lang/tools#2619)
  b2d0780db  Wed Oct 7 11:27:16 2026 -0700  Kevin Moore  [stream_channel] Require package:async ^2.6.0 (dart-lang/tools#2644)
  3ab7bf9d0  Wed Oct 7 11:35:27 2026 +0200  Moritz  [boring] Move package:boring to dart-lang/tools (1/4): vendored BoringSSL, bindings and source build (dart-lang/tools#2614)
  29df6348b  Tue Oct 6 15:21:44 2026 -0700  Kevin Moore  feat(coverage): support web coverage reports in test_with_coverage (dart-lang/tools#2448)

Regenerate pkg/{analyzer,analyzer_plugin,analyzer_testing}/api.txt. The
rolled package:api_summary v1.2.0 (dart-lang/tools#2619) sorts the primary
barrel first and other entry points after it, so entries in these files
are reordered. The APIs themselves are unchanged.

Change-Id: Ie8ff19c5a981ea8ecee8939c75457986a947e0a7
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/563220
Reviewed-by: Samuel Rawlins <[email protected]>
Auto-Submit: Nate Bosch <[email protected]>
Reviewed-by: Paul Berry <[email protected]>
Commit-Queue: Paul Berry <[email protected]>
natebosch added a commit that referenced this pull request Oct 29, 2026
`IterableChecks.any`, `IterableChecks.every`,
`MapChecks.containsKeyThat`, and `MapChecks.containsValueThat` embed the
description of a condition. They collapse to a single line at the root,
through a predicate noun such as `an iterable containing a value that is
greater than <7>`, but their clause always used the multi-line form of a
label followed by the indented description. Where the clause itself is
shown, such as when one of these expectations is nested inside the
condition of another, the description never collapsed, while the same
check reached through `has` did. Reported in
#2784 (comment):

    check([[1]]).any(.it()..any(.it()..isGreaterThan(1)));

    Expected: a List<List<int>> that:
      contains a value that:
        contains a value that:
          is greater than <1>

Add an internal `describedClause` helper next to
`describedPredicateNoun`. It joins the label to the description of the
condition on a single line when the condition describes in a single line
and the result fits within 80 columns, and otherwise keeps the label
followed by the indented description. Both helpers share the logic for
joining and the length limit. Use it for the clauses of all four
expectations. The inner `any` above now describes itself as a single
line, so the outer `any` collapses to a single `Expected:` line at the
root:

    an iterable containing a value that contains a value that is
    greater than <1>

Joining the label of `every`, `only has values that`, to a description
such as `is less than <10>` reads as `only has values that is less than
<10>`. Describe the condition as `only has elements where each` instead,
which reads as `only has elements where each is less than <10>` on one
line, and `only has elements where each:` followed by the description
otherwise. The label is a predicate over the iterable ending with a
singular noun, so it also reads as a sentence when `every` is nested in
another condition, such as `an iterable where every element only has
elements where each is less than <10>`. Change the predicate noun to
match the wording: `an iterable where every element is less than <10>`.
natebosch added a commit that referenced this pull request Oct 30, 2026
Expectations which soft check a condition against part of their value,
such as `IterableChecks.every`, describe the failure within the `which` of
their own rejection. Each of `every`, `IterableChecks.deepEquals`,
`StreamChecks.inOrder`, `StreamChecks.anyOf`, and
`IterableChecks.pairwiseMatches` formatted the nested failure by hand, each
slightly differently, and each dropped detail in some cases. Two bugs in
`every` were reported in
#2784 (review).

Add `CheckFailure.describeNested`, which describes a failure in the same
format as a failure message from `check`. A soft checked condition has no
label for its root subject, so the caller provides its own line describing
the value, such as `has an element at index 0 that:`. When the actual does
not collapse to a single line, a line with only `Actual:` takes the place
of the `Actual: <label> that:` line from `check`. `check` and
`describeNested` share a private method which produces the "Actual:" and
"Which:" lines, so the two formats agree by construction. `CheckFailure` is
exported only from `package:checks/context.dart`, so the method is
available to authors of expectations without being part of the API for
writing checks.

Use `describeNested` from `every`, `deepEquals`, `inOrder`, `anyOf`, and
`pairwiseMatches`.

- `every` and `deepEquals` lost the property name when the nested failure
  collapsed:

      Which: has an element at index 0 that:
        Actual: 'foo'

  It now reads `Actual: has field: 'foo'`. `deepEquals` also uses `Which:`
  in place of `which`.
- `inOrder` and `anyOf` dropped the actual description entirely when a
  condition failed without collapsing, including when the actual collapsed
  to the root. They now always include an `Actual:` line, and no longer
  repeat an `Expected:` line for a collapsed failure since the outer
  description already lists each condition.
- `pairwiseMatches` keeps the description of the condition for the element,
  followed by `Actual:` in place of `Actual element at index N:`, so the
  expected and actual descriptions are separated:

      does not have an element at index 0 that:
        has field: 'foo'
        has field that has length: <4>
        Actual:
          has field: 'foo'
          has field that has length: <3>
          Which: is not equal

Fix the failure details when a soft check rejects a precondition such as
`isA<T>()` and further expectations follow it, which also affected `every`:

    check([Wrapper(123)])
        .every(.it()..value.isA<Foo>().field.equals('bar'));

    Failed assertion: 'foundDepth == -1': is not true.

`_fullDetails` assumed that when a subject itself failed, the failing clause
was the last one added to it. That holds for `check`, where a failure
throws before anything else is added, but not for a soft check, where the
following nested clauses are still added, nor for an unawaited expectation,
which can fail after later clauses are added. Pass the rejected clause from
each failure through to `_fullDetails` instead of relying on
`_clauses.last`. A rejected guard is also no longer added a second time as
a `_GuardClause`, since the failing expectation clause already describes
it, so the precondition appears once in the description.

Omit the clauses after the failing one from the failure details. They were
described under `Actual:` as if the value satisfied them, and when the
failing clause came first the actual collapsed onto the root line with the
later clauses printed as its children.

Keep asynchronous expectations in the order they were made, and leave out
any which have not finished when another expectation fails. Nested
asynchronous expectations such as `completes` and `emits` added their clause
only once the value arrived, so an expectation made after them was described
before them. An expectation from
`Context.expectAsync` added its clause before its predicate finished, so it
was described as satisfied if a later expectation failed first. Both now
add a placeholder which keeps their place and is not described until they
finish. Expectations from `Context.expectUnawaited`, such as
`doesNotComplete`, are never awaited by the check and are still described
as satisfied until they fail.

Make the `CheckFailure` and `FailureDetail` constructors private. Failures
only come from soft checking a `Condition`, since `Condition` and `Context`
are `abstract final`, and no API accepts a failure constructed outside the
package. The public `FailureDetail` constructor also recorded that any
custom `actual` started with the root label line, so `describeNested`
dropped its first line.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

package:checks Issues related to pkg:checks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants