Issue
When running a single environment, tox correctly reports the exit code back to the calling shell cmd.exe:
> tox run -e py310-fail 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
FAIL
>
> tox run -e py310-ok 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
>
However, when multiple environments are run, independent of individual results, the error code always indicates "OK"
> tox run -e py310-ok,py310-fail 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
>
> tox run -e py310-fail,py310-fail2 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
>
Expected behaviour:
If at least one run fails, that should be reflected in the return code:
> tox run -e py310-ok,py310-fail 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
FAIL
>
> tox run -e py310-fail,py310-fail2 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
FAIL
>
Of course, if all succeed, that should also be correctly transported:
> tox run -e py310-ok,py310-ok2 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL
>
Environment
- OS: Windows 10
pip list of the host Python where tox is installed:
Package Version
---------- -------
pip 23.0.1
setuptools 67.3.3
wheel 0.38.4
(tox is installed via pipx and is not in that environment).
>tox --version
4.4.6 from <USERHOME>\.local\pipx\venvs\tox\lib\site-packages\tox\__init__.py
Minimal example
I made the effort to set up a minimal example. Feel free to peruse it:
See this gist
Issue
When running a single environment, tox correctly reports the exit code back to the calling shell
cmd.exe:> tox run -e py310-ok 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL >However, when multiple environments are run, independent of individual results, the error code always indicates "OK"
> tox run -e py310-ok,py310-fail 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL >> tox run -e py310-fail,py310-fail2 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL >Expected behaviour:
If at least one run fails, that should be reflected in the return code:
Of course, if all succeed, that should also be correctly transported:
> tox run -e py310-ok,py310-ok2 1>NUL 2>NUL & if ERRORLEVEL 1 echo FAIL >Environment
pip listof the host Python wheretoxis installed:(tox is installed via
pipxand is not in that environment).Minimal example
I made the effort to set up a minimal example. Feel free to peruse it:
See this gist