Context

With the increasing amount of test cases testing is becoming more and more the bottleneck in development. Even a fast cached (make all-fast) build on a current PC currently takes around 17 minutes to complete.

My proposal would be to split the tests into separate test files according to the containers they are executed from.
The user could then decide to execute all containers and therefore all tests or just the particular container and its associated tests.
The Makefile would be getting a new optional argument CONTAINER='container1 container2'

Furthermore I would like to introduce a optional argument PERF=(int) setting, to respect different performance capabilities of different setups.
The setting would basically be a factor applied to any sleep command in the Makefile.

Pros

• shorter debugging times(Performance factor and selective testing)
• smaller and therefore more readable test files
• more granular debugging
• less ressource heavy. Containers would be removed after their tests exited successfully.
• downward compatible

Cons

• not one single searchable tests.bats file