Test suite with multiple tests only executes last-defined test #394
Description
When defining multiple tests in a test suite, only the last one is used/printed. For example:
tests.yaml:
elastic_tube_1d_test:
tutorials:
- path: elastic-tube-1d
case_combination:
- fluid-cpp
- solid-cpp
reference_result: ./elastic-tube-1d/reference-data/fluid-cpp_solid-cpp.tar.gz
- path: elastic-tube-1d
case_combination:
- fluid-python
- solid-python
reference_result: ./elastic-tube-1d/reference-data/fluid-python_solid-python.tar.gz
- path: elastic-tube-1d
case_combination:
- fluid-cpp
- solid-python
reference_result: ./elastic-tube-1d/reference-data/fluid-cpp_solid-python.tar.gzprint_test_suites.py:
~/repos/precice/tutorials/tools/tests [develop]$ python3 print_test_suites.py
Using log-level: INFO
Test suite: openfoam_adapter_pr contains:
Flow over heated plate
cases: [(fluid-openfoam, solid-openfoam)]
reference_results: [/home/gc/repos/precice/tutorials/flow-over-heated-plate/reference-data/fluid-openfoam_solid-openfoam.tar.gz]
Test suite: openfoam_adapter_release contains:
Flow over heated plate
cases: [(fluid-openfoam, solid-openfoam)]
reference_results: [/home/gc/repos/precice/tutorials/flow-over-heated-plate/reference-data/fluid-openfoam_solid-openfoam.tar.gz]
Test suite: fenics_test contains:
Flow over heated plate
cases: [(fluid-openfoam, solid-fenics)]
reference_results: [/home/gc/repos/precice/tutorials/flow-over-heated-plate/reference-data/fluid-openfoam_solid-fenics.tar.gz]
Test suite: nutils_test contains:
Flow over heated plate
cases: [(fluid-openfoam, solid-nutils)]
reference_results: [/home/gc/repos/precice/tutorials/flow-over-heated-plate/reference-data/fluid-openfoam_solid-nutils.tar.gz]
Test suite: calculix_test contains:
Perpendicular flap
cases: [(fluid-openfoam, solid-calculix)]
reference_results: [/home/gc/repos/precice/tutorials/perpendicular-flap/reference-data/fluid-openfoam_solid-calculix.tar.gz]
Test suite: elastic_tube_1d_test contains:
Elastic tube 1D
cases: [(fluid-cpp, solid-python)]
reference_results: [/home/gc/repos/precice/tutorials/elastic-tube-1d/reference-data/fluid-cpp_solid-python.tar.gz]systemtests.py:
~/repos/precice/tutorials/tools/tests [develop]$ python3 systemtests.py --suites=elastic_tube_1d_test
Using log-level: INFO
INFO: About to run the following systemtest in the directory /home/gc/repos/precice/tutorials/runs:
[Elastic tube 1D (fluid-cpp, solid-python)]
INFO: Started running Elastic tube 1D (fluid-cpp, solid-python), 0/1