Current behavior 😯
To distinguish known failures from any new failures while still presenting all test failures as such, the test-fixtures-windows CI job directs cargo-nextest to create an XML export of test results, which it then uses PowerShell to parse. Usually this works, but sometimes--and, it seems, increasingly, over the past couple of months--it fails with a message saying the XML file hasn't been created. For example, from this recent run:
Get-Content: C:\a\_temp\354806a2-1977-4dfc-b236-6f9dbd41d8a6.ps1:2
Line |
2 | … ml]$junit_xml = Get-Content -Path 'target/nextest/with-xml/junit.xml'
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| Cannot find path 'C:\a\gitoxide\gitoxide\target\nextest\with-xml\junit.xml' because it does not exist.
I assume that, when this happens, it really doesn't create the XML file, but I haven't yet investigated. Most of the time, it works fine. Rerunning the failed job is the workaround.
Expected behavior 🤔
The XML file should always be created and should always be accessible. in the test that uses the profile calling for it. Currently this is only in test-fixtures-windows tests, and it will probably remain only in those tests.
If the underlying bug turns out to be in cargo-nextest and we can identify it, then we should report it there. However, I think that, since that CI job was created, there are newer cargo-nextest features related to logging, exporting, etc., that can be used instead of this technique, and that may be preferred to this technique. That could be a misrecollection, however--unfortunately I don't have time at the moment to look into it, but it's something I've been meaning to get to, even separately from these failures.
(In the longer term, we shouldn't need to do anything like this at all, because we should make the test suite fully compatible with Windows including when not using any committed fixture archives. But that's outside the scope of this issue.)
Git behavior
Not applicable.
Steps to reproduce 🕹
I don't know how to trigger it, but it happens once in a while. It can be verified by checking the above-linked log. It can probably be reproduced by rerunning the test-fixture-windows jobs repeatedly on the same ref, but I don't actually know for sure that this is the case--it might have preconditions that vary subtly based on what is being tested.
Current behavior 😯
To distinguish known failures from any new failures while still presenting all test failures as such, the
test-fixtures-windowsCI job directscargo-nextestto create an XML export of test results, which it then uses PowerShell to parse. Usually this works, but sometimes--and, it seems, increasingly, over the past couple of months--it fails with a message saying the XML file hasn't been created. For example, from this recent run:I assume that, when this happens, it really doesn't create the XML file, but I haven't yet investigated. Most of the time, it works fine. Rerunning the failed job is the workaround.
Expected behavior 🤔
The XML file should always be created and should always be accessible. in the test that uses the profile calling for it. Currently this is only in
test-fixtures-windowstests, and it will probably remain only in those tests.If the underlying bug turns out to be in
cargo-nextestand we can identify it, then we should report it there. However, I think that, since that CI job was created, there are newercargo-nextestfeatures related to logging, exporting, etc., that can be used instead of this technique, and that may be preferred to this technique. That could be a misrecollection, however--unfortunately I don't have time at the moment to look into it, but it's something I've been meaning to get to, even separately from these failures.(In the longer term, we shouldn't need to do anything like this at all, because we should make the test suite fully compatible with Windows including when not using any committed fixture archives. But that's outside the scope of this issue.)
Git behavior
Not applicable.
Steps to reproduce 🕹
I don't know how to trigger it, but it happens once in a while. It can be verified by checking the above-linked log. It can probably be reproduced by rerunning the
test-fixture-windowsjobs repeatedly on the same ref, but I don't actually know for sure that this is the case--it might have preconditions that vary subtly based on what is being tested.