Improve the accessibility of figures shown with _repr_html_

I was running into some accessibility audit errors with Jupyter Book, and it made me realize that the _repr_html_ method of Figures doesn't output img elements that have alt tags (something that screen readers like).

It seems that the relevant code is here:

matplotlib/lib/matplotlib/figure.py

Line 378 in 59acd5c

def _repr_html_(self):

and then uses the template that's referenced here:

matplotlib/lib/matplotlib/backends/backend_webagg.py

Line 289 in 690fd71

tpl = Path(core.FigureManagerWebAgg.get_static_file_path(),

Maybe using this?

https://github.com/matplotlib/matplotlib/blob/be978a35932a9f303f3d60f14f52ed9df89334c5/lib/matplotlib/backends/web_backend/js/mpl.js

I wonder if this could be a relatively simple fix by making the figure title the alt tag by default, and if it doesn't exist, just making it something like Figure?

Adding a simple alt sounds reasonable.

So i'd love to raise this issue from the 💀

ipython/ipython#12864 adds alt text to Image displays. This begs the question "where could the alt text live?" I'd love to move this issue along and would appreciate some advice.

I'm not familiar with the matplotlib abstractions, but is there a way we could do figure.set_alt_text("a bar chart.."). This way authors could construct/derive their own alt text for a figure. Some sample code with pandas might look like:

plt = df.plot.bar()
plt.figure.set_alt_text(f"A bar chart showing features {', '.join(df.columns)}")

In the output image, the alt text would be assigned to the image with a change in the IPython/core/pylabtools.py

It seems like this could significantly impact a lot of documentation using notebooks or sphinx gallery, and provide extra surfaces for new-comers to contribute improvements.

Is there a certain matplotlib object this feature would live at? Or is something like this out of scope for matplotlib objects in general, like alt text ain't in scope for the abstractions and the display should handle it?

Could we grab it from the label? I think that the label is not used for anything on the Figures.

If that feature doesn't get used for anything then I think it would be a solid place to infer the alt text from. i don't know enough about the _label_s usage, if y'all think its reasonable to appropriate it for alt text then we'd have a nice existing solution.

In #17893 there's a discussion about a colorblind friendly attribute & possibly a spec to go with it, and I think for similar reasons it'd be good to have an explicit .alt_text attribute that can start as a string and possibly grow into a spec as the community coalesces around standards.

I'm worried about using _label_s 'cause I think there's been some previous discussion on the calls about using the artist label attributes for some kind of artist management & now we have the subfigures too and I think it'd be better to keep that usage consistent.

if that is the case then maybe it is worth considering adding an aria attribute collection, alt is an alias for aria-label. this might be a more general approach that has a proper specification behind it.

plt = df.plot.bar()
alt = f"A bar chart showing features {', '.join(df.columns)}"
plt.figure.set_aria(label=alt)
# or
plt.figure.set_aria_label(alt)

i could imagine that with some thinking this abstraction could work its way into some of the interactive matplotlib views.

I think adding adding this as an Artist level state is probably the right place to put this. This will let us attach arbitrary aria data to anything in the draw tree. To start with we would probably only pull the one from the Figure to populate the data on the repr html, but I suspect that if we had information available we might be able to find a use for it.

Skimming https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/ARIA_Techniques it looks like most of them are not relevant to us (as our scope is much more limited that "arbitrary document / UI").

I propose a straw-man implementation of:

def set_aria(self, aria: Dict[str, obj]):
    # TODO validation
    self._aria = aria

def update_aria(self, **aria):
    # TODO validation
    for k, v in aria.items():
        self._aria[k] = v

def get_aria(self):
    return dict(self._aria)

I think we would want to keep obj.get_aria()['label'] and obj.get_label() separate as the text you would want for a legend and the text you would want to generate a text-description of the object may not be the same text.

As well as alt tags, it would also be useful to be able to add longdesc descriptions to <img> tags.

as it stands now, longdesc is deprecated, but it could be provided as an alias to aria-details. i think we can capture most accessibility affordances through the different combinations of the aria properties. so adding a longdesc with the current proposal would look like:

plt = df.plot.bar()
alt = f"A bar chart showing features {', '.join(df.columns)}"
plt.figure.set_aria(label=alt, details="figure_description.html")

Ah.. I should probably read up on accessibility best practice. My knowledge is probably a bit dated...

Resolving this issue and defining a method for setting figure alt text will also allow for the Sphinx-Gallery Matplotlib scraper to be upgraded to use it. This will allow alt text to be used for plots in the examples gallery section of the documentation.

Based on how Sphinx-Gallery is structured, there's no clear way to include alt text unless it's exposed via Matplotlib. Once an alt text method is defined, it also might make sense to default the Description tag in the metadata used by the AGG/PNG, Agg/TIFF, and SVG backends to the alt text if not otherwise defined.

Is this still an open issue?

This is still open. I think the requirements and thus the generality of the needed solution is not clear. Related:

If there is interest in reviving this, it's probably best to restart the discussion what is needed.

@timhoffm

@timhoffm there was also discussion on @QuLogic maybe working on writing out alt text as part of the metadata on a savefig (and I think this was part of the Small Dev Grant proposal) and the thinking is that it would then be something that could be injected into/read by the reprs.

Hello! I'd like to revive this thread again, as we would love to make Jupyter notebooks on GitHub more accessible, and matplotlib figures are some of the most common elements within notebooks. What would it look like it implement a very minimal version of this feature? From my perspective, the requirements would be:

Define a default alt text for all figures, so that all img tags automatically have an alt attribute.
- If necessary, this could simply be text like Figure 1 to indicate that the image is a plot/figure.
- Even better, the alt text should contain a general description of the figure to whatever extent is possible to be automated. Ideally, if we could include any information that is programmatically accessible that would be great, for example:
  - The title of the figure
  - The titles of the axes
  - The range of the axes
  - A description of the data (how many bars, how many lines, etc.)
Allow defining custom alt text for any figure. Using the API suggestion above, maybe this could look like:
```
plt = df.plot.bar()
plt.figure.set_alt_text(f"A bar chart showing features {', '.join(df.columns)}")
```
This isn't strictly an ARIA feature, so I'm not sure it makes sense to include in an API named like set_aria.

The ability to provide longer descriptions in some fashion would be a great improvement as well, but providing alt text is a must-have for web accessibility.

For matploltib plots, this gist, for example, suggests:

def get_plot_info(p):
    """ in this example p are the Axes objects pd.DataFrame.plot() returns """
    
    plot_info = {
    "title" : p.get_title(),
    "legendlabels": [x.get_text() for x in p.legend_.get_texts()],
    "x_label": p.xaxis.get_label_text(),
    "x_ticks": [x.get_text() for x in p.xaxis.get_majorticklabels()],
    "y_label": p.yaxis.get_label_text(),
    "y_ticks": [x.get_text() for x in p.yaxis.get_majorticklabels()]
    }
    
    return plot_info

Navigation Menu

Repository navigation

Improve the accessibility of figures shown with `_repr_html_` #15971

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Improve the accessibility of figures shown with _repr_html_ #15971

Description

Activity

timhoffm commented on Dec 20, 2019

tonyfast commented on Oct 6, 2021

QuLogic commented on Oct 6, 2021

tonyfast commented on Oct 6, 2021

story645 commented on Oct 7, 2021

tonyfast commented on Oct 7, 2021

tacaswell commented on Oct 7, 2021

psychemedia commented on Mar 9, 2022

tonyfast commented on Mar 9, 2022

psychemedia commented on Mar 9, 2022

mpetroff commented on Jul 16, 2022

psychemedia commented on Jul 15, 2024

timhoffm commented on Jul 15, 2024

story645 commented on Jul 15, 2024

camchenry commented on Jul 3, 2025

psychemedia commented on Jul 3, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Improve the accessibility of figures shown with `_repr_html_` #15971