We need a new Continuous Integration configuration to launch the tests of some popular downstream projects.

Candidate projects:

• xgboost and or lightgbm as very popular third party libraries
• skorch
• imbalanced-learn not necessarily as popular as others but with high use of sklearn's API
• autosklearn?
• any other suggestion?

Those test would not be run in PRs unless we use a specific tag in the commit message for instance. This test setup would be run by maintainers in the release branches.

We do something similar in the cloudpickle project with the [ci downstream] commit tag:

https://github.com/cloudpipe/cloudpickle/blob/master/.travis.yml#L37-L70

For scikit-learn this would probably need a bit more scripting as the build is more complex than for a pure python project such as cloudpickle but the general idea would be the same.