Skip to content

[BUG] "__doc__" attribute of static method is not set. #2403

Closed

Description

@pkerichang

The staticmethod wrapper class makes pybind11-generated static methods behave similarly to Python's static methods, which is a great improvement. However, for some reason the docstring of the underlying function is not passed to the staticmethod.

To reproduce, simply define a static method with any docstring (such as "Hello World!"), build the pybind11 library and import in python, then you'll see the docstring is no longer there when you call the help() method on the static method object.

After some trial and error, modifying the def_static() method in pybind11.h as follows solves the issue:

    template <typename Func, typename... Extra> class_ &
    def_static(const char *name_, Func &&f, const Extra&... extra) {
        static_assert(!std::is_member_function_pointer<Func>::value,
                "def_static(...) called with a non-static member function pointer");
        cpp_function cf(std::forward<Func>(f), name(name_), scope(*this),
                        sibling(getattr(*this, name_, none())), extra...);

        auto tmp = staticmethod(cf);
        PyCFunctionObject *fun_ptr = (PyCFunctionObject *) cf.ptr();
        if (fun_ptr->m_ml->ml_doc) {
            tmp.attr("__doc__") = strdup(const_cast<char *>(fun_ptr->m_ml->ml_doc));
        }

        attr(cf.name()) = tmp;
        return *this;
    }

I'm hesitant to submit a PR as I don't know if this is the proper way to set the doc string of a staticmethod (I can't find any documentations on manipulating Python objects in C/C++ online).

If this turns out to be a good fix then I can submit a PR. Otherwise, just letting you know this bug exists. Having good docstring is important to me as I use a stub generation script (similar to the one in Mypy object) to generator python stub files for pybind11 libraries to make IDE programming easier.

Activity

  1. bstaletic commented on Aug 17, 2020

    @bstaletic
    Collaborator

    I'm not at my computer right now, but I think this is a duplicate of a very old bug report. I might be wrong.

  2. YannickJadoul commented on Aug 17, 2020

    @YannickJadoul
    Collaborator

    This seems quite complex too me. Wouldn't this work just as well?

        template <typename Func, typename... Extra> class_ &
        def_static(const char *name_, Func &&f, const Extra&... extra) {
            static_assert(!std::is_member_function_pointer<Func>::value,
                    "def_static(...) called with a non-static member function pointer");
            cpp_function cf(std::forward<Func>(f), name(name_), scope(*this),
                            sibling(getattr(*this, name_, none())), extra...);
            auto static_cf = staticmethod(cf);
            if (py::hasattr(cf, "__doc__"))
                cf.attr("__doc__") = static_cf.attr("__doc__");
            attr(cf.name()) = static_cf;
            return *this;
        }

    But I'd still like to figure out why PyStaticMethod_New doesn't do the same as @staticmethod in Python, before getting that PR in. Maybe this mistake was made in other places as well.

  3. YannickJadoul commented on Aug 17, 2020

    @YannickJadoul
    Collaborator

    @pkerichang, is this on Python 2 or Python 3 or both, btw?

  4. pkerichang commented on Aug 18, 2020

    @pkerichang
    Author

    Hello,

    I only tested on Python 3 (3.7 FYI).

    as for using attr() versus ptr->m_ml->ml_doc to get the docstring, that's where my uncertainty comes in. Other places that deal with cpp_function obtain the doc string with the second approach, where-as for static method the only way to set the doc string is through the first approach. I'm not knowledgeable enough about the code base to know what is preferred versus not.

  5. YannickJadoul commented on Aug 18, 2020

    @YannickJadoul
    Collaborator

    Hi @pkerichang. I can't reproduce this, though. What version of pybind11 are you on?

    I just tried this:

    #include <pybind11/pybind11.h>
    
    namespace py = pybind11;
    
    struct X {
        static bool f(int x, float b) { return true; }
    };
    
    PYBIND11_MODULE(example, m)
    {
        py::class_<X>(m, "X")
            .def_static("f", &X::f, "This is an amazing docstring");
    }
    $ python3.7
    Python 3.7.6 (default, Dec 19 2019, 23:50:13) 
    [GCC 7.4.0] on linux
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import example
    >>> example.X
    <class 'example.X'>
    >>> example.X.f
    <built-in method f of PyCapsule object at 0x7f97e8638f30>
    >>> example.X.f.__doc__
    'f(arg0: int, arg1: float) -> bool\n\nThis is an amazing docstring\n'

    And help(example.X.f) shows me:

    Help on built-in function f in module example:
    
    f(...) method of builtins.PyCapsule instance
        f(arg0: int, arg1: float) -> bool
        
        This is an amazing docstring
    (END)
    

    Am I doing something different than you are?

  6. pkerichang commented on Aug 18, 2020

    @pkerichang
    Author

    Ah I figure out the problem, my bad, this is not a bug after all.

    In python, if f_inst is an instance method of class Foo, then Foo.f_inst is equivalent to Foo.__dict__['f_inst']. However, for f_static, a static method of class Foo, Foo.__dict__['f_static'] points to the static method decorator around the function object instead, not the function object itself. You need to use Foo.__dict__['f_static'].__func__ to access the underlying function object, Foo.f_static. I do not know this before, and my docstring parsing script is finding the docstring of the staticmethod decorator, not the docstring of the underlying object.

    I'm closing this issue now, thanks for all the quick responses!

  7. YannickJadoul commented on Aug 18, 2020

    @YannickJadoul
    Collaborator

    You need to use Foo.__dict__['f_static'].__func__ to access the underlying function object, Foo.f_static. I do not know this before, and my docstring parsing script is finding the docstring of the staticmethod decorator, not the docstring of the underlying object.

    Yes, the staticmethod class implements the descriptor protocol (see here for the details), so you won't notice if you call Foo.f_static (or getattr(Foo, 'f_static')), but if you go through __dict__, it doesn't resolve the descriptor protocol for you.

    But glad to hear I'm not going crazy and it's not a bug in pybind11; thanks! :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions