ci: move tests than can be run in parallel now#3038
Conversation
|
|
It should average around 5 minutes (300 sec) in my experience, at least on my 4 core/thread i5-6500 😅 Considerations for reducing the test time furtherI have tried a variety of things to speed it up, and noticed that single tests could be sped up by 50%, but once running in parallel ( Changing to a different base image for testssl showed about 40 seconds or more improvement for me, although I've also been observing a smaller improvement once the base image upgrades to a newer version of Alpine, I'm not 100% sure but it appears to be due to We can potentially look at splitting the test case up into multiple files grouped in a folder instead (with the methods extracted out into a helper) for bats to run in parallel (each test case runs For CI at least, we can always leverage more runners to spread out those test cases 🤔 Another option is to only run those tests only when relevant files would affect it for PRs, while master branch could run on each merge and raise an issue if the test failed. Oh, well currently test sets are looking like they'll average around 4 minutes anyway? So it's not that big of a deal, better to reduce the arm build, which I've been looking into and will suss out a PR for this week probably. |
polarathene
left a comment
There was a problem hiding this comment.
Are we making snake_case for file names a convention? I noticed we were using kebab-case before.
Thanks for tackling this :)
I'd go for snake case, indeed :) About the current state of the single 6min test: I actually don't mind because it's working fine. Just noting that there is room for future optimisations. I'd go for splitting the single file into multiple too :) And I totally agree that bringing down ARM build time should be a priority :D |
Description
Move & renaming of tests that can now run in parallel. I tried to group tests a bit, and I tried to evenly balance the test across the sets. All sets should now take more or less the same amount of time to complete.
Type of change
Checklist:
docs/)