This is interesting... For test generators, one is able to set test_function.description to override the docstring. However as here, you need the function to be copied (perhaps by partial) for each description to be correctly displayed. This could be done by a decorator of some kind, but isn't particularly friendly...

This adds a dependency for testing. Therefore I am -1

How about we simply convert all docstrings to comments instead...?

I guess I'm also +1 on just using comments everywhere.

FYI when I introduced the generators into the common tests, I was thinking about a nice way to set the names. A decorator for the check functions would probably be a nice and easy solution.

@amueller @jnothman Do you feel decorators for generators are needed?

For this test and a generator without description set to it...

def test_discretenb_partial_fit():
    for cls in [MultinomialNB, BernoulliNB]:
        yield check_partial_fit, cls

The output when it passes is -

sklearn.tests.test_naive_bayes.test_discretenb_partial_fit(<class 'sklearn.naive_bayes.MultinomialNB'>,) ... ok
sklearn.tests.test_naive_bayes.test_discretenb_partial_fit(<class 'sklearn.naive_bayes.BernoulliNB'>,) ... ok

The output when it fails is -

sklearn.tests.test_naive_bayes.test_discretenb_partial_fit(<class 'sklearn.naive_bayes.MultinomialNB'>,) ... FAIL
sklearn.tests.test_naive_bayes.test_discretenb_partial_fit(<class 'sklearn.naive_bayes.BernoulliNB'>,) ... ok
.
.
.
======================================================================
FAIL: sklearn.tests.test_naive_bayes.test_discretenb_partial_fit(<class 'sklearn.naive_bayes.MultinomialNB'>,)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File ".../scikit-learn/sklearn/tests/test_naive_bayes.py", line 120, in check_partial_fit
    assert False
AssertionError

----------------------------------------------------------------------

decorators are for another PR.

But do you think we need them? given that, with a decorator, we simply get the description unlike the more detailed sklearn.tests.test_naive_bayes.test_discretenb_partial_fit(<class 'sklearn.naive_bayes.BernoulliNB'>,) ? ( The line of error is anyway going to be shown in the Traceback... The arguments passed to the generator could be useful right? )

yes, I think they would be good, if we can keep the name of the estimator in the string. If not, you are right. Maybe that was the reason why I didn't go for the decorator approach.

The thing is that you don't know which check is running from the output, and for example which one gives a warning or takes long etc.

@amueller Could you take a look at this one? I know this has a huge diff, but should be easy to review since this just changes docstring to comments...

(Kindly review the last two commits separately as they are not related to the docstring --> comment conversion)

Also as you had suggested, I'll make a separate PR for the decorator for test gens.!

apart from inline comment lgtm

Done! Please take a final look! @ogrisel could I trouble you for a 2nd review?

looks fine now.

@ogrisel Any reviews? (sorry to trouble you with multiple PR review requests :p)

can you rebase please?

Done :)