Skip to content

Include stdout/stderr in failure messages for devicelab's tests #18990

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
DanTup merged 3 commits into from
Jul 3, 2018
Merged

Include stdout/stderr in failure messages for devicelab's tests #18990

DanTup merged 3 commits into from
Jul 3, 2018

Conversation

@DanTup
Copy link
Contributor

@DanTup DanTup commented Jul 2, 2018

Sometimes some of these tests unexpectedly fail with a non-zero exit code. This ensures stdout/stderr is included in the test failure message when this happens so that we can track down the issue.

Before:

00:01 +0 -2: run.dart script accepts file paths [E]                                                                                                            
  Expected: <0>
    Actual: <1>
  
  package:test              expect
  test/run_test.dart 33:7   main.<fn>.expectScriptResult
  ===== asynchronous gap ===========================
  dart:async                _AsyncAwaitCompleter.completeError
  test/run_test.dart        main.<fn>.expectScriptResult
  ===== asynchronous gap ===========================
  dart:async                _asyncThenWrapperHelper
  test/run_test.dart 28:89  main.<fn>.expectScriptResult
  test/run_test.dart 43:31  main.<fn>.<fn>

After:

00:01 +0 -2: run.dart script accepts file paths [E]                                                                                                            
  Expected: <0>
    Actual: <1>
  [ stderr from test process ]

  [ end of stderr ]
  
  [ stdout from test process ]
  
      ═════════════════╡ ••• Running task "smoke_test_success" ••• ╞══════════════════
      
      
      Executing: /Users/dantup/Dev/Google/flutter/dev/devicelab/../../bin/cache/dart-sdk/bin/dart --enable-vm-service=0 --no-pause-isolates-on-exit bin/tasks/smoke_test_success.dart
      RegExp: pattern=Observatory listening on (\S+:(\d+)/\S*)$ flags=
      Instance of '_RegExpMatch'
      [smoke_test_success] [STDOUT] Observatory listening on http://127.0.0.1:65284/
      exitcode: 0
      Task result:
      {
        "success": false,
        "reason": "fake error"
      }
      
      
      ═════════════════╡ ••• Finished task "smoke_test_success" ••• ╞═════════════════
  
  [ end of stdout ]
  
  package:test              expect
  test/run_test.dart 33:7   main.<fn>.expectScriptResult
  ===== asynchronous gap ===========================
  dart:async                _AsyncAwaitCompleter.completeError
  test/run_test.dart        main.<fn>.expectScriptResult
  ===== asynchronous gap ===========================
  dart:async                _asyncThenWrapperHelper
  test/run_test.dart 28:89  main.<fn>.expectScriptResult
  test/run_test.dart 43:31  main.<fn>.<fn>

DanTup added 2 commits July 2, 2018 12:42
@DanTup
Include stdout/stderr in failure messages
ab206e9 
Sometimes some of these tests unexpectedly fail with a non-zero exit code. This ensures stdout/stderr is included in the test failure message when this happens so that we can track down the issue.
@DanTup
Remove redundant info about exit code
d0dd683
@DanTup
Remove unnecessary indenting
550820a 
The [markers] make it fairly clear so this just makes the test code noisy.
mravn-google
mravn-google approved these changes Jul 2, 2018
View reviewed changes
Copy link
Contributor

@mravn-google mravn-google left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DanTup DanTup mentioned this pull request Jul 2, 2018
Closed
@DanTup
Copy link
Contributor Author

DanTup commented Jul 2, 2018

(Windows build failure is #18287 so I've kicked it off again..)

@DanTup
Copy link
Contributor Author

DanTup commented Jul 2, 2018

And this time it seems to have been terminated after an hour while running tests :/ Trying again!

@DanTup DanTup merged commit 0329f0c into flutter:master Jul 3, 2018
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Aug 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Reviewers

1 more reviewer

@mravn-google mravn-google mravn-google approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DanTup @mravn-google @googlebot