Maybe there's more info we could get to the user, but a forceful cancellation can (and usually will) always leave untracked resources around because the provider has not been able to return from some number of ApplyResourceChange calls. This is a more frequent hazard with user-executed testing, where they too often decide they don't want to wait for whatever reason and panic hit ctrl+c to try and abort.

So, I think the implementation right now won't cause this because the forceful cancellation actually only tells it to skip the tidy up, but it still always waits for whatever operation is ongoing to finish (ie. it doesn't pass the forceful cancellation into the terraform context, only the first nice cancel request). I did this because, as you say, we'll almost definitely be left with resources live that Terraform doesn't know about.

If I was to come back and implement the timer here I think we'd start to see the problems, which is why I've just left it as a TODO.

I'm not sure what the best way to deal with this would be. Potentially we could add a validate/refresh/import option to the test command, and it runs through all the tests and attempts to see if any infrastructure that matches the config exists? That might be very complicated. We already print out all the resources we know exist and haven't been tidied up, I don't know how much more Terraform can actually say.

Yeah, without aborting immediately, or at least within a short delay, this isn't going to do much of anything. Most providers don't fully implement cancellation, so this will prevent Terraform from exiting at all without an external signal until the provider times out internally. This has already passed the "there may be data loss" warning, so users should be willing to accept the risk, and now we are probably more likely to see users killing the process entirely to get unstuck.

Okay, I'll merge this as is now and will revisit this TODO in a separate PR. I can make the language a bit stronger in regards to warnings about what will happen during a hard interrupt. Could also print out the plan that was being executed when it was cancelled, so users can also see what resources might have been left behind.

Not sure if it matters here, but Empty also looks for root module outputs, because in a normal context those need to be removed explicitly by a plan, but for tests maybe it's useful to skip those?

Thanks! That's good context for me in the future. I think in this case it's okay, as it'll just go and do a very quick destroy operation and seeing that fail/succeed might be something the user wants to include in the tests.

If we are trying to handle panics in the tests, we may want to look into overriding the behavior of the global panic handler. Most codepaths in core are covered by logging.PanicHandler, which does not recover and exits with 11 after writing the custom error message. (this also might be a rare enough occurrence that it's not really worth the effort)

ah yeah, I see that logging.PanicHandler is also referenced directly in the graph walk cycle. Let me take this functionality out of this PR, and I can make a dedicated PR separately that can solve this. It's not as simple as I thought it would be.