Should fail and TestFailure be exposed by scaffolding.dart? #2073

Activity

1. jamesderlin

   added

   package:checksIssues related to pkg:checksIssues related to pkg:checks

   on Aug 3, 2023
2. natebosch commented on Aug 11, 2023

   

   natebosch

   on Aug 11, 2023

   Member

   More actions

   I do prefer to push the check(functionExpectedToFail).throws<ExpectedException>() style.

   there's a significant amount of code in my catch block, so I think it's more readable with the explicit try-catch.

   Is there behavior other than checking expectations on e?

   Do you have a concrete example that we could compare against the .throws<ExpectedException>() pattern?
3. jamesderlin commented on Aug 12, 2023

   

   jamesderlin

   on Aug 12, 2023

   ContributorAuthor

   More actions

   Is there behavior other than checking expectations on e?

   I'd like to do something like:

   var stopwatch = Stopwatch()..start();
   try {
     functionExpectedToFail();
     fail('Failed to fail.');
   } on ExpectedException catch (e) {
     check(stopwatch.elapsed).isLessThan(timeToFailThreshold);
     check(e.someProperty).deepEquals(expectedValue);
     check(someTransformation(e.someOtherProperty)).deepEquals(someOtherExpectedValue);
   }
4. natebosch commented on Aug 15, 2023

   

   natebosch

   on Aug 15, 2023

   Last edited by natebosch

   Member

   More actions

   I'd like to do something like:

   Interesting, the stopwatch use case is not one I had predicted.

   If we were to naively force this into full checks pattern I think it would look like

       var stopwatch = Stopwatch()..start();
       check(functionExpectedToFail).throws<ExpectedException>()
         ..has((_) => stopwatch.elapsed, 'elapsed time')
             .isLessThan(timeToFailThreshold)
         ..has((e) => e.someProperty, 'someProperty').deepEquals(expectedValue)
         ..has((e) => someTransformation(e.someOtherProperty),
                 'some transformed property')
             .deepEquals(somOtherExpectedValue);

   The planned codegen would only cover a small part of that

        check(functionExpectedToFail).throws<ExpectedException>()
          ..has((_) => stopwatch.elapsed, 'elapsed time')
              .isLessThan(timeToFailThreshold)
   -      ..has((e) => e.someProperty, 'someProperty').deepEquals(expectedValue)
   +      ..someProperty.deepEquals(expectedValue)
          ..has((e) => someTransformation(e.someOtherProperty),
                  'some transformed property')
              .deepEquals(somOtherExpectedValue);

   We could write a small utility to cover the someTransformation case which works nicely with the codegen. The implementation overhead is lower if someTransformation is already a test utility and can be directly rewritten as the extension.

          ..has((_) => stopwatch.elapsed, 'elapsed time')
              .isLessThan(timeToFailThreshold)
          ..someProperty.deepEquals(expectedValue)
   -      ..has((e) => someTransformation(e.someOtherProperty),
   -              'some transformed property')
   -          .deepEquals(somOtherExpectedValue);
   +      ..someOtherProperty.transformed.deepEquals(someOtherExpectedValue);
      });
    }
   +
   +extension on Subject<PropertyType> {
   +  Subject<TransformedType> get transformed => context.nest(
   +      'transformed', (value) => Extracted.value(someTransformation(value)));
   +}

   I'm less sure what to do about the stopwatch though. It's a bit of an unusual test since we don't expect code running as a test to perform the same as code running in production. If it can tolerate the perf difference anyway, it should probably handle the small overhead that the .throws check may have. I think it's probably OK for an unusual test pattern to have an unusual checks usage, so we could hold the Subject in a variable before doing the expensive deepEquals. That would make the rewrite:

       var stopwatch = Stopwatch()..start();
       var exceptionSubject = check(functionExpectedToFail).throws<ExpectedException>();
       check(stopwatch).elapsed.isLessThan(timeToFailThreshold);
       exceptionSubject
         ..someProperty.deepEquals(expectedValue)
         ..someOtherProperty.transformed.deepEquals(someOtherExpectedValue);
         
       // ...
   extension on Subject<PropertyType> {
     Subject<TransformedType> get transformed => context.nest(
         'transformed', (value) => Extracted.value(someTransformation(value)));
   }

   I can see the appeal of writing it with the try/fail()/catch over using codegen and the extra extension on Subject utility. I've seen that pattern get implemented wrong a few times though, and I'm wary of encouraging it.

   I do think that checks has a better story for this situation than expect() does. I don't think that expect would let you split up parts of the expectations into different points like checks allows with the Subject variable. You must either get the exception as a variable yourself (with a try/catch probably), or do all of the expectations in the same call to expect by chaining some .having on the TypeMatcher.

   cc @jakemac53 @kevmoo for any thoughts on these patterns
5. natebosch

   added a parent issue on Jun 16, 2026

   • package:checks stable release burndown #2672
6. auto-submit

   closed this as completedin #2723on Aug 18, 2026
7. pull

   added a commit that references this issue on Aug 18, 2026

   Export TestFailure from package:test_api/scaffolding.dart (dart-lang#…

   ...

   This issue will close once commit bbe8f9b is merged into the 'master' branch.

   bbe8f9b