Skip to content

[autorefs] Support interlinking to non-object references #508

Closed as not planned

Description

@masasin

Hi

I have two projects (project_a and project_b) whose documentation I want to link together. Sphinx has intersphinx, and I found mkdocstrings which can do the same thing. However, the docs were written already, and there was no autodoc used at all. I would like to link to specific pages/headings in project_a from project_b.

What the mkdocs.yml looked like before:

site_name: Project A
theme:
  name: material

markdown_extensions:
  - pymdownx.highlight:
      use_pygments: true
      linenums: true
  - mkautodoc


plugins:
  - search
  - autolinks
  - mermaid2
  - awesome-pages

Under plugins, I added mkdocstrings, but the inventory file raised a 404. I thought that it would automatically generated if a python handler is used, and that the default handler was python. I added enable_inventory and set it to true. The objects.inv file existed, but was only 131 bytes big (almost empty compared to the size of the docs).

Since autodocs isn't used, I also added autorefs as recommended in the docs. This got me the same result, even when I turned of autolinks in case it conflicts.

site_name: Project A
theme:
  name: material

markdown_extensions:
  - pymdownx.highlight:
      use_pygments: true
      linenums: true
  - mkautodoc


plugins:
  - search
  - autolinks
  - autorefs
  - mermaid2
  - awesome-pages
  - mkdocstrings:
      enable_inventory: true

What else should I change in order to create an inventory file with the right references?

(Side question: How do you find out what the name of the project in mkdocstrings is? e.g., how did we know that inventory was supposed to be inventory and not Inventory or inventory_project etc on the Usage page? Does it have to be defined somewhere?)

Activity

  1. pawamoy commented on Jan 10, 2023

    @pawamoy
    Member

    Hello!

    Sorry, it's not clear to me what you're asking.
    You're trying to reference objects of project A in project B's documentation, right?

    Did you use the recipe for automatically creating a reference page for each module? Or do you manually add autodoc instructions to your Markdown pages?

  2. masasin commented on Jan 11, 2023

    @masasin
    Author
  3. pawamoy commented on Jan 12, 2023

    @pawamoy
    Member

    I see, thanks, I understand now.

    Currently, autorefs does not add non-objects references to the inventory.
    We could consider implementing it, though I'm not sure yet if this is even feasible.

  4. masasin commented on Jan 12, 2023

    @masasin
    Author

    That's too bad. Thanks for the confirmation. I'll link the live page directly in the meantime.

  5. added
    featureNew feature or request
    and removed on Mar 5, 2023
  6. chrysle commented on Jun 8, 2023

    @chrysle

    @pawamoy I'd like to try to work on this, but I'm unsure how to do this yet.

    Therefore It'd help me to test several parts of the code (outputting contents) simulating that mkdocstrings is used as an extension.
    Is there any way to do this?

  7. pawamoy commented on Jun 8, 2023

    @pawamoy
    Member

    To be clear, you'd like to add support for adding regular headings (not mkdocstrings-generated ones) into the mkdocstrings-generated objects.inv file? This change would happen in the mkdocstrings/autorefs repository. Basically, you'd want to "register" every encountered heading (though I don't think the code is made to return these headings to you) within the std domain (not py). EDIT: see comment below.

    To start testing things, you can just work on a regular project which uses mkdocstrings, clone autorefs somewhere and install it in editable mode:

    cd project_using_mkdocstrings
    # create venv, activate it, install deps, etc.
    git clone https://github.com/mkdocstrings/autorefs
    pip install -e autorefs
    

    Let me know if that's enough for you to start 🙂

  8. chrysle commented on Jun 8, 2023

    @chrysle

    Thank you for this information, it helps a lot!

    To be clear, you'd like to add support for adding regular headings (not mkdocstrings-generated ones) into the mkdocstrings-generated objects.inv file?

    I plan to start with adding all the regular documentation pages (in std:doc) to the inventory.
    I think this could be achieved here? It should be worth a configuration option.

  9. pawamoy commented on Jun 8, 2023

    @pawamoy
    Member

    If by "all the documentation pages" you mean mapping headings from all pages, then yeah. autorefs maps heading ids to their absolute location within the site.

  10. chrysle commented on Jun 9, 2023

    @chrysle

    autorefs doesn't yet support handling object inventories, does it?

    So I think the first step should be transferring inventory.py there.

  11. pawamoy commented on Jun 9, 2023

    @pawamoy
    Member

    Ha, I was mistaken actually. Let me reformulate everything.

    So. When adding the mkdocstrings plugin to MkDocs, it adds its own extension to the Markdown parser. We'll call it the main, or outer extension. This extension does two things:

    • it uses Jinja templates to render objects. These templates use a heading filter that both register the heading with autorefs and add the object to the inventory. The domain is the domain of the handler that took care of the object, and the role is passed using the data-role HTML attribute. The value of this data-role attribute is set within the Jinja templates.
    • it adds a second, inner Markdown extension (amongst other), only used when rendering docstrings, that detects subheadings (headings in docstrings), and bubbles them up to the outer extension. The outer extension then register these headings to autorefs.

    The reason why docstring subheadings are not added to the inventory, is because they don't have a data-role HTML attribute. To implement the feature you're interested in, you would first need to modify mkdocstrings to also add docstring subheadings to the inventory. Since these headings don't have HTML data, we can assume the domain is std and the role is doc (IIUC, but maybe there's another, more fitting default role we could use). The code in question:

    for heading in headings:
    anchor = heading.attrib["id"]
    self._autorefs.register_anchor(page, anchor)
    if "data-role" in heading.attrib:
    self._handlers.inventory.register(
    name=anchor,
    domain=handler.domain,
    role=heading.attrib["data-role"],
    uri=f"{page}#{anchor}",
    )

    An elif clause could be added:

    elif option_to_add_std_headings_in_inventory_is_enabled:
         self._handlers.inventory.register( 
             name=anchor, 
             domain=heading.attrib.get("data-domain", "std"), 
             role=heading.attrib.get("data-role", "doc"), 
             uri=f"{page}#{anchor}", 
         ) 
    
    # heading.attrib.get("data-domain", "std") and heading.attrib.get("data-role", "doc")
    # would allow docs writers to specify the domain and role themselves thanks to the attr_list extension:
    # ## My heading { data-domain="my-domain" data-role="my-role" }

    Secondly, doing that would mean we only handle subheadings in docstrings. So to handle all headings, we must find a way to detect these headings in mkdocstrings (adding them to the inventory is then easy, and done as described above). Maybe we could reuse the inner Markdown extension that reports headings, and set it as an outer extension as well. Not sure yet of the impact it could have. Or maybe this is doable with the MkDocs plugin hooks directly.

    @oprypin, if you could confirm what I say is correct, and makes sense, that'd be great 🙂

  12. chrysle commented on Jun 9, 2023

    @chrysle

    That must have been a lot of work, thank you!

    So. When adding the mkdocstrings plugin to MkDocs, it adds its own extension to the Markdown parser. We'll call it the main, or outer extension. This extension does two things:

    • it uses Jinja templates to render objects. These templates use a heading filter that both register the heading with autorefs and add the object to the inventory. The domain is the domain of the handler that took care of the object, and the role is passed using the data-role HTML attribute. The value of this data-role attribute is set within the Jinja templates.
    • it adds a second, inner Markdown extension (amongst other), only used when rendering docstrings, that detects subheadings (headings in docstrings), and bubbles them up to the outer extension. The outer extension then register these headings to autorefs.

    I guess these extensions are stored in extension.py and rendering.py.

    The reason why docstring subheadings are not added to the inventory, is because they don't have a data-role HTML attribute. To implement the feature you're interested in, you would first need to modify mkdocstrings to also add docstring subheadings to the inventory.

    I would also be interested in adding the whole page urls to the inventory, without any heading anchors. I'm sure this can be achieved, too.

    An example for that can be found in the inventory at https://webknjaz.github.io/intersphinx-untangled/towncrier.rtfd.io/.

    Since these headings don't have HTML data, we can assume the domain is std and the role is doc (IIUC, but maybe there's another, more fitting default role we could use).

    We can consider using std:label.

    So to handle all headings, we must find a way to detect these headings in mkdocstrings (adding them to the inventory is then easy, and done as described above). Maybe we could reuse the inner Markdown extension that reports headings, and set it as an outer extension as well.

    Not sure what you mean with that... Can't we just extend the functionality of the inner extension?

  13. pawamoy commented on Jun 9, 2023

    @pawamoy
    Member

    I would also be interested in adding the whole page urls to the inventory, without any heading anchors. I'm sure this can be achieved, too.

    Fine by me 🙂

    Not sure what you mean with that... Can't we just extend the functionality of the inner extension?

    The inner extension only runs when converting docstrings. It will never see the rest of the headings written in Markdown pages.

  14. chrysle commented on Jun 11, 2023

    @chrysle

    Some technical question while working:

    I've installed mkdocstrings in editable mode in a project, and would like to access the logs after the documentation has been built. How can I do that?

  15. chrysle commented on Jun 12, 2023

    @chrysle

    @pawamoy Sorry for pinging, but I could not find anything about the question and need to output the contents of some variables in a particular state, for example using log.debug(). Could you help me out?

    Also I'd like to ask if I can modify the inventory configuration, so it would look like this:

    plugins:
    - mkdocstrings:
        inventory:
          enabled: true
          register_headings: true

    I think this would ease configuring the inventory, and also leave room for further options.

  16. pawamoy commented on Jun 12, 2023

    @pawamoy
    Member

    I think you can use standard logging utilities to configure the mkdocs.plugins.mkdocstrings.templates logger, as to redirect/duplicate logs into a log file somewhere on the file system.

    Yes we can support a map of items for inventory as long as it's backward-compatible.

  17. changed the title [-]Empty inventory when linking across repositories in a project that added mkdocstrings after the docs have been written[/-] [+][autorefs] Support interlinking to non-object references[/+] on Aug 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions