Cannot pickle figure or axes (TypeError: instancemethod)

Couple of issues I've found.

def latex_float(f, pos=0):
    float_str = "{0:.2g}".format(f)
    if "e" in float_str:
        base, exponent = float_str.split("e")
        return r"${0} \times 10^{{{1}}}$".format(base, int(exponent))
    else:
        return r"${}$".format(float_str)

formatter = FuncFormatter(latex_float)

corr = np.corrcoef(data['x'], data['y'])[0,1]
fig, ax = pl.subplots(figsize=figsize)
hist, _, _, im = ax.hist2d(data['x'], data['y'], norm=LogNorm(), bins=(bins_x, bins_y) , alpha=0.75, cmap = cmap)

cbar = fig.colorbar(im, ticks=[0., 1., 2., 5., 10., 20., 50., 100., 200., 500., 1000., 2000., 5000.], format=formatter)
cbar.set_label('number density', fontsize=labelsize, labelpad=79)
cbar.ax.tick_params(labelsize=labelsize-6, width=3, length=15)
cbar.ax.minorticks_on()

ax.set_xlabel(label_x, fontsize=labelsize)
ax.set_ylabel(label_y, fontsize=labelsize)
ax.grid(True, which='both', linewidth=3, linestyle='--', alpha=0.5)
ax.text(0.95, 0.05, '$\mathrm{{Corr}} = {:0.4f}$'.format(corr), transform=ax.transAxes, fontsize=labelsize, verticalalignment='bottom', horizontalalignment='right', bbox=textprops)

ax.tick_params(axis='both', which='major', labelsize=labelsize-6)

pickle.dump(ax, file('ZH_ax.pkl', 'w+'))

fig.close()

As it is, the code gives the following error:

Traceback (most recent call last):
  File "make_plots_new.py", line 124, in <module>
    pickle.dump(fig, file('ZH_ax.pkl', 'w+'))
  File "/lib/python2.7/copy_reg.py", line 70, in _reduce_ex
    raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle instancemethod objects

If I exclude the label formatter function in the colorbar, I get an error of

Traceback (most recent call last):
  File "make_plots_new.py", line 124, in <module>
    pickle.dump(fig, file('ZH_ax.pkl', 'w+'))
cPickle.UnpickleableError: Cannot pickle <type 'Image'> objects

How can one pickle?

what version of mpl? Can you test with 1.4.0rc4?

mpl version 1.3.1 . I can check 1.4.0rc4 later, but I was kind of expecting that it would be more or less "working" with 1.3.x

There was a bunch of work done in the last year on pickling so it might already be fixed (just not released).

Once we release 1.4.0 (which should be very soon) we will no longer support the 1.3 series.

@kratsg

@kratsg - I've been triaging a few pickle bugs, but unfortunately I've not been able to test your code as it isn't a self contained example (imports and data needed). As a result, I will try to fix a few of the issues highlighted, but I may miss some of those that you are seeing. If you have some data and the necessary imports, please would you be willing to update the issue report with the necessaries?

Thanks,

@pelson

@pelson , you should be able to do something like:

import matplotlib.pyplot as pl
from matplotlib.colors import LogNorm
from matplotlib.ticker import FuncFormatter

figsize = (16,12)
data = {'x': np.random.rand(100), 'y': np.random.rand(100)}
bins_x = bins_y = 10
cmap = pl.get_cmap('hot')
label_x = 'x'
label_y = 'y'
labelsize = 28
textprops = dict(boxstyle='round', facecolor=self.light_grey, alpha=0.5, linewidth=0.0)

def latex_float(f, pos=0):
    float_str = "{0:.2g}".format(f)
    if "e" in float_str:
        base, exponent = float_str.split("e")
        return r"${0} \times 10^{{{1}}}$".format(base, int(exponent))
    else:
        return r"${}$".format(float_str)

formatter = FuncFormatter(latex_float)

corr = np.corrcoef(data['x'], data['y'])[0,1]
fig, ax = pl.subplots(figsize=figsize)
hist, _, _, im = ax.hist2d(data['x'], data['y'], norm=LogNorm(), bins=(bins_x, bins_y) , alpha=0.75, cmap = cmap)

cbar = fig.colorbar(im, ticks=[0., 1., 2., 5., 10., 20., 50., 100., 200., 500., 1000., 2000., 5000.], format=formatter)
cbar.set_label('number density', fontsize=labelsize, labelpad=79)
cbar.ax.tick_params(labelsize=labelsize-6, width=3, length=15)
cbar.ax.minorticks_on()

ax.set_xlabel(label_x, fontsize=labelsize)
ax.set_ylabel(label_y, fontsize=labelsize)
ax.grid(True, which='both', linewidth=3, linestyle='--', alpha=0.5)
ax.text(0.95, 0.05, '$\mathrm{{Corr}} = {:0.4f}$'.format(corr), transform=ax.transAxes, fontsize=labelsize, verticalalignment='bottom', horizontalalignment='right', bbox=textprops)

ax.tick_params(axis='both', which='major', labelsize=labelsize-6)

pickle.dump(ax, file('ZH_ax.pkl', 'w+'))

fig.close()

which is self-contained.

@pelson

@pelson , you should be able to do something like:

Great! Thanks for the quick response. My PR (#3627) appears to have fixed the problem without the grid, but the following will still fail:

ax = plt.axes()
ax.grid(True, which='both', linewidth=3, linestyle='--', alpha=0.5)
plt.draw()
pickle.dump(ax, BytesIO())

I'll take a look at this and add it to the PR.

Thanks.

For the record, I diagnosed the problem by calling recursive_pickle(ax) which can be found in matplotlib/tests/test_pickle.py.

The result:

<bound method Line2D._draw_dotted of <matplotlib.lines.Line2D object at 0x46a6690>>
Failed to pickle attribute "_lineFunc" in (attribute "gridline" in (list/tuple item #0 in (attribute "majorTicks" in (attribute "xaxis" in (top level object)))))

Next step is to look for where _lineFunc is added to the gridline - I went to the Axes class, found that def grid calls the grid method on an Axis, so went there, and the trail went cold. So instead I grepped for _lineFunc (nothing like bruteforce 😉) and found that it is attached to a line in lines.py. Fixing commit added to the PR (f1e43ed).

Pickling can be quite a pain. ;) I've had to write some of my own custom pickle handlers for various cython code I'm working on. This looks great, albeit slightly convoluted.

@pelson

@pelson: aside from the mention of dill.detect in uqfoundation/dill#4,
I should point you to the following files, which contains all the known types that dill can pickle, and the custom picklers:
https://github.com/uqfoundation/dill/blob/master/dill/_objects.py
https://github.com/uqfoundation/dill/blob/master/dill/dill.py

Several other of the most common packages have started allowing an override of pickle to enable people to use dill or another custom pickler to serialize their code… matplotlib could do the same. Please ping me if you have any questions about the above, or dill, or otherwise.

Several other of the most common packages have started allowing an override of pickle to enable people to use dill or another custom pickler to serialize their code…

Hmmm. Might be a good idea - mpl makes fairly regular use of instancemethods as attributes to be called at draw time, so automatic pickling of those would be quite pleasant.

The mpl codebase has needed a fair amount of custom getstate/setstate overrides in order to deal with those cases and a few other niceties (e.g. https://github.com/matplotlib/matplotlib/blob/master/lib/matplotlib/figure.py#L1299).

@pelson

@pelson Closing this as I think that #3627 fixes both issues

Navigation Menu

Repository navigation

Cannot pickle `figure` or `axes` (TypeError: instancemethod) #3392

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Cannot pickle figure or axes (TypeError: instancemethod) #3392

Description

Activity

tacaswell commented on Aug 21, 2014

kratsg commented on Aug 21, 2014

tacaswell commented on Aug 21, 2014

pelson commented on Oct 9, 2014

kratsg commented on Oct 9, 2014

pelson commented on Oct 9, 2014

pelson commented on Oct 9, 2014

kratsg commented on Oct 9, 2014

mmckerns commented on Oct 9, 2014

pelson commented on Oct 10, 2014

tacaswell commented on Oct 10, 2014

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

Cannot pickle `figure` or `axes` (TypeError: instancemethod) #3392