BUILD: move all test dependencies to ./test_requirements.txt - #14378

The Check_Dependencies test is correctly failing:

> Package    Version Latest Type 
> ---------- ------- ------ -----
> pytest     5.0.1   5.1.1  wheel
> pytest-cov 2.6.1   2.7.1  wheel
> pytz       2018.9  2019.2 wheel

Now we need to decide what to do.

• Leave it just like this. When any of the dependencies release an update, our CI will complain but we can ignore it until we update the dependencies in a separate PR
  • for: simple
  • against: makes it confusing for drive-by contributors when CI fails
• Use a service instead of having the check in our CI.
  • for: outsourcing the problem
  • against: github has no fine-grained permissions, any service will need complete control of the repo. Also, can we set it up to use this non-standard test_requirements.txt?
• Automate instead of fail the test: automatically issue a PR to update the dependencies
  • for: cleanly supports upgrading in a non-intrusive way
  • against: complicates the script, will need some API support from github which might break at some point

Is it possible to mark that task as failable, so that we get an X from azure on just one CI integration, but that x does not aggregate to an overall GitHub ci failure?

I'd much prefer to use a service for this.

against: github has no fine-grained permissions, any service will need complete control of the repo

that's not correct, GitHub definitely has fine-grained permissions, and a well-written service should need nothing more than perhaps a "read email addresses" to be installed.

Please put nose back, we have public API that relies on it, as well as tests in numpy/testing/tests/

Note that nose imports the imp module (IIRC) and that module is scheduled to be removed in 3.10. Nose can be fixed pretty easily, and some distros do that, but unless someone updates nose on pip it could be a problem at some point.

Is it possible to mark that task as failable, so that we get an X from azure on just one CI integration, but that x does not aggregate to an overall GitHub ci failure?

I don't see how, the Job spec does not have any "failable" option I can see

I'd much prefer to use a service for this.

I am exploring options for sending an email when the test fails. I think we can add an extension for SendMail, do we have an organizational email entity that can sign up with them to get a token?

Isn't issuing PRs to update dependencies what Dependabot is for?

It seems dependabot may work out. In order to get the ball rolling, can we merge this PR as is, so that we can set up dependabot (or another service) and point them to the new file?

Merging so we can get started. This will break #14404, so I will update that. Thanks Matti.