Conversation
Reference: #398 Reference: hashicorp/terraform#33454 Adds support for the `terraform test` command, which currently supports JSON machine-readable output and one flag for configuring the tests directory away from the command default of `tests`. The command will return a non-zero status if any of the tests fail, which returns an error back to callers of the `Test` function. If consumers need access to the pass/fail test results, the terraform-json Go module will need to be enhanced to support the test summary JSON, e.g. ``` {"@Level":"info","@message":"Failure! 0 passed, 1 failed.","@module":"terraform.ui","@timestamp":"2023-07-25T10:03:42.980799-04:00","test_summary":{"status":"fail","passed":0,"failed":1,"errored":0,"skipped":0},"type":"test_summary"} ``` Output of new end-to-end testing: ``` $ TFEXEC_E2ETEST_VERSIONS=1.5.3,1.6.0-alpha20230719 go test -count=1 -run='TestTest' -v ./tfexec/internal/e2etest ... --- PASS: TestTest (9.50s) --- SKIP: TestTest/test_command_passing-1.5.3 (4.06s) --- PASS: TestTest/test_command_passing-1.6.0-alpha20230719 (5.44s) ... --- PASS: TestTestError (0.48s) --- SKIP: TestTestError/test_command_failing-1.5.3 (0.27s) --- PASS: TestTestError/test_command_failing-1.6.0-alpha20230719 (0.21s) ```
|
Lovely! @liamcervante will be adding terraform-json support for the return types, so those will definitely be usable here. |
So, terraform-json doesn't actually support the streamed JSON logs for any of the other commands. I'm not sure if I should just go and add them for the testing command only. I'm gathering other opinions on this, is this something that would be required for this PR? If yes, that'll help me make my decision! I'm only hesitant as I'm worried about why the streamed logs for the other commands aren't available. |
|
We've agreed to be reactive here, so happy to merge this as-is. We can come and add more fine-grained info to the outputs as and when we have a concrete use-case. |
|
Lovely, thank you bflad |
3 participants
Closes #398
Reference: hashicorp/terraform#33454
Adds support for the
terraform testcommand, which currently supports JSON machine-readable output and one flag for configuring the tests directory away from the command default oftests. The command will return a non-zero status if any of the tests fail, which returns an error back to callers of theTestfunction. If consumers need access to the pass/fail test results, the terraform-json Go module will need to be enhanced to support the test summary JSON, e.g.Output of new end-to-end testing: