Skip to content

make_parallel_tests_available method is broken and raising warnings #1018

Description

@Justin-W
  • Include how you run your tests and which testing framework or frameworks you are running.
    • we run via rake spec and rspec
  • Include the SimpleCov version you are running in your report.
    • simplecov 0.21.2
  • Include your ruby -e "puts RUBY_DESCRIPTION".
    • ruby 3.1.2p20 (2022-04-12 revision 4491bb740a) [x86_64-darwin20]
  • Please also specify the gem versions of Rails (if applicable).
    • not using Rails
  • Include any other coverage gems you may be using and their versions.
    • N/A

After upgrading to simplecov (0.21.2) and ruby 3.1.2, we are seeing a lot of stderr output generated by this line, because the make_parallel_tests_available method is behaving incorrectly due to incorrect assumptions/implementation within the probably_running_parallel_tests? method.

We use both of the ENV vars that the probably_running_parallel_tests? method checks, but not for parallel testing. We do sequential testing, but our tests span processes because we are collecting coverage from CLI commands executed from within the spec examples, and then merging the separate coverage reports after all tests have been run.

The above process worked perfectly with simplecov 0.17.1 and ruby 2.x, but broke in simplecov 0.18.0 (see #853 and #854).

The warnings on stderr from the probably_running_parallel_tests? method are actually causing some of our tests to fail (because there shouldn't be any STDERR output during those tests), in addition to being very noisy. And the warnings are based on bad assumptions which are incorrect in our test context (i.e. that we're using the parallel_tests gem).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions