Can't pickle bar plots: Failed to pickle attribute "gridline"

I know pickling of figures (awesome!) is still experimental, but it seems bar plots have problems. recursive_pickle() seems to give a more detailed traceback than pickle.dump():

from matplotlib.tests.test_pickle import recursive_pickle
from pylab import figure, bar
fig = figure()
bar(left=range(10), height=range(10))
recursive_pickle(fig)

gives:

Line2D((0,0),(0,1))
Failed to pickle attribute "gridline" in (list/tuple item #0 in (attribute "majorTicks" in
(attribute "xaxis" in (list/tuple item #1 in (list/tuple item #1 in (list/tuple item #0 in
(attribute "_elements" in (attribute "_axstack" in (top level object))))))))).
 Type: <class 'matplotlib.lines.Line2D'>. Traceback follows:
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-9-b0c9db9f3303> in <module>()
----> 1 recursive_pickle(fig)

/home/mspacek/src/matplotlib/lib/matplotlib/tests/test_pickle.pyc in recursive_pickle(top_obj)
     88 #        print('trying %s' % location)
     89         try:
---> 90             pickle.dump(obj, BytesIO(), pickle.HIGHEST_PROTOCOL)
     91         except Exception, err:
     92             print(obj)

TypeError: expected string or Unicode object, NoneType found

Using pickle.dump() instead like this:

import pickle
from io import BytesIO
from pylab import figure, bar
fig = figure()
bar(left=range(10), height=range(10))
pickle.dump(fig, BytesIO(), pickle.HIGHEST_PROTOCOL)

gives a different error:

PicklingError: Can't pickle <built-in method copy_from_bbox of tuple object at 0x3406a70>:
it's not found as __main__.copy_from_bbox

This is in master on Python 2.7 on Ubuntu 12.10.

@pelson

@pelson, maybe you'd be the best suited to debug this one?

The key to this was changing the test to use pickle instead of cPickle. (The recursive_pickle output really wasn't helpful in this case - that could potentially be a bug... but I'm not 100%).

PicklingError: Can't pickle <function <lambda> at 0x3d15398>: it's not found as matplotlib.axes.<lambda>

Once I saw there was a lambda function I guessed it was on the bar itself:

    >>> b = plt.bar(left=range(10), height=range(10))
    >>> for k, v in b.__dict__.items():
    ...        print(k, type(v))
    _label <type 'str'>
    patches <type 'list'>
    errorbar <type 'NoneType'>
    _propobservers <type 'dict'>
    eventson <type 'bool'>
    _oid <type 'int'>
    _remove_method <type 'function'>

In short, there is no tidying up of the _remove_method on containers as there is on Artist subclasses.

Several tricky points to overcome, but at least the solution is nice and simple 😄 (#1988)

Fixed by #1988.

Thanks, but I just pulled from master and tried it, and unless I'm crazy, for the same input I still get the same errors listed in the first post. In both cases, I'm trying to pickle the figure. If instead I try to pickle the current axes with recursive_pickle like this:

from matplotlib.tests.test_pickle import recursive_pickle
from pylab import figure, bar, gca
fig = figure()
bar(left=range(10), height=range(10))
recursive_pickle(gca())

I get this slightly different error:

(1, <matplotlib.axes.AxesSubplot object at 0x2ef80d0>)
Failed to pickle list/tuple item #1 in (list/tuple item #0 in (attribute "_elements" in (attribute "_axstack" in (attribute "figure" in (top level object))))). 
 Type: <type 'tuple'>. Traceback follows:
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/mspacek/src/matplotlib/lib/matplotlib/tests/test_pickle.py", line 90, in recursive_pickle
    pickle.dump(obj, BytesIO(), pickle.HIGHEST_PROTOCOL)
TypeError: expected string or Unicode object, NoneType found

I made sure to recompile with sudo python setup.py develop, but that shouldn't have been necessary anyway, since the fix was pure Python. I seem to get the same errors whether I'm running in Python, IPython, or IPythonQt. Maybe some setting in my matplotlibrc could make the difference?

By the way, I tried forcing test_pickle.py to use pickle instead of cPickle, but that didn't seem to change the error messages.

I'll take another look...

@mspacek

@mspacek - the following code works for me on master:

>>> from pylab import figure, bar
>>> fig = figure()
>>> bar(left=range(10), height=range(10))
<Container object of 10 artists>
>>> import pickle
>>> pickle.dump(fig, open('delme.pkl', 'wb'))
>>>

Would you mind checking again, maybe something got merged which fixed it in the last fortnight....

I guess I must be doing something wrong. After updating from master, when I run the above, I get:

PicklingError: Can't pickle <built-in method copy_from_bbox of tuple object at 0x2d51368>: it's not found as __main__.copy_from_bbox

as I did in the first post.

I guess I must be doing something wrong

There could be version specific stuff going on here. ~~What version of python are you using?~~ (just saw it is 2.7) Which backend? Are you able to run:

$> python tests.py matplotlib.tests.test_pickle

Sorry for the delay. I'm running the Qt4Agg backend. I pulled from master and rebuilt today, still have the same problems.

Yes, the above command runs fine. Here's the output:

mspacek@Escher:~/src/matplotlib$ python tests.py matplotlib.tests.test_pickle
/usr/local/lib/python2.7/dist-packages/nose/plugins/manager.py:418: UserWarning: Module dateutil was already imported from /home/mspacek/src/matplotlib/lib/dateutil/__init__.pyc, but /usr/lib/python2.7/dist-packages is being added to sys.path
  import pkg_resources
...
----------------------------------------------------------------------
Ran 3 tests in 2.167s

OK

@mspacek

Just as a data point, I'm unable to reproduce with master on Python 2.7 (Fedora 18). I'm not sure why it's still failing for @mspacek.

@pelson

Unable to reproduce this with master. The code provided by @pelson works, and test_pickle.py passes for me and @mspacek.

I'm going to close this now as I think it's been resolved by #1988. Please feel free to reopen if I'm mistaken.

@pelson

I am able to run the code provided by @pelson however I am still experiencing the same error as @mspacek whenever I try to pickle using the TkAgg backend. I'm using Python 2.7.6 and matplotlib version 1.3.1.

I modified the TkAgg example code to demonstrate my issue (press any key to attempt a pickle):

#!/usr/bin/env python

import matplotlib
import pickle
matplotlib.use('TkAgg')

from numpy import arange, sin, pi
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg, NavigationToolbar2TkAgg
# implement the default mpl key bindings
from matplotlib.backend_bases import key_press_handler

from matplotlib.figure import Figure

import sys
import Tkinter as Tk


root = Tk.Tk()
root.wm_title("Embedding in TK")


f = Figure(figsize=(5,4), dpi=100)
a = f.add_subplot(111)
t = arange(0.0,3.0,0.01)
s = sin(2*pi*t)

a.plot(t,s)


# a tk.DrawingArea
canvas = FigureCanvasTkAgg(f, master=root)
canvas.show()
canvas.get_tk_widget().pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)

toolbar = NavigationToolbar2TkAgg( canvas, root )
toolbar.update()
canvas._tkcanvas.pack(side=Tk.TOP, fill=Tk.BOTH, expand=1)

def on_key_event(event):
    print('you pressed %s'%event.key)
    key_press_handler(event, canvas, toolbar)
    p = pickle.dumps(a)

canvas.mpl_connect('key_press_event', on_key_event)

def _quit():
    root.quit()     # stops mainloop
    root.destroy()  # this is necessary on Windows to prevent
                    # Fatal Python Error: PyEval_RestoreThread: NULL tstate

button = Tk.Button(master=root, text='Quit', command=_quit)
button.pack(side=Tk.BOTTOM)

Tk.mainloop()

Something has gone really sideways with this example as I get import errors on _winreg on a linux box:

you pressed o
Exception in Tkinter callback
Traceback (most recent call last):
  File "/usr/lib/python2.7/lib-tk/Tkinter.py", line 1489, in __call__
    return self.func(*args)
  File "/home/tcaswell/python_env/py2k/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/backends/backend_tkagg.py", line 485, in key_press
    FigureCanvasBase.key_press_event(self, key, guiEvent=event)
  File "/home/tcaswell/python_env/py2k/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/backend_bases.py", line 1814, in key_press_event
    self.callbacks.process(s, event)
  File "/home/tcaswell/python_env/py2k/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/cbook.py", line 533, in process
    proxy(*args, **kwargs)
  File "/home/tcaswell/python_env/py2k/local/lib/python2.7/site-packages/matplotlib-1.4.x-py2.7-linux-x86_64.egg/matplotlib/cbook.py", line 408, in __call__
    return mtd(*args, **kwargs)
  File "tmp.py", line 44, in on_key_event
    p = pickle.dumps(a)
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 739, in save_global
    module = whichmodule(obj, name)
  File "/usr/lib/python2.7/pickle.py", line 817, in whichmodule
    if name != '__main__' and getattr(module, funcname, None) is func:
  File "/usr/lib/python2.7/dist-packages/six.py", line 116, in __getattr__
    _module = self._resolve()
  File "/usr/lib/python2.7/dist-packages/six.py", line 105, in _resolve
    return _import_module(self.mod)
  File "/usr/lib/python2.7/dist-packages/six.py", line 76, in _import_module
    __import__(name)
ImportError: No module named _winreg

The issue here seems to be two fold, you should be pickling Figures, not axes, and second something about drawing the figure seems to break it

from __future__ import print_function
import matplotlib
matplotlib.use('agg')
import matplotlib.pyplot as plt
import pickle
print(matplotlib.__version__)
print(matplotlib.get_backend())
fig = plt.gcf()
plt.plot(range(5))
p = pickle.dumps(fig)
print('im good')
plt.draw()

p = pickle.dumps(fig)
print("won't see me")

gives

(py2k)tcaswell@eowyn:~$ python tmp.py
1.4.x
agg
im good
Traceback (most recent call last):
  File "tmp.py", line 14, in <module>
    p = pickle.dumps(fig)
  File "/usr/lib/python2.7/pickle.py", line 1374, in dumps
    Pickler(file, protocol).dump(obj)
  File "/usr/lib/python2.7/pickle.py", line 224, in dump
    self.save(obj)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 600, in save_list
    self._batch_appends(iter(obj))
  File "/usr/lib/python2.7/pickle.py", line 615, in _batch_appends
    save(x)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 562, in save_tuple
    save(element)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 562, in save_tuple
    save(element)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 331, in save
    self.save_reduce(obj=obj, *rv)
  File "/usr/lib/python2.7/pickle.py", line 419, in save_reduce
    save(state)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 649, in save_dict
    self._batch_setitems(obj.iteritems())
  File "/usr/lib/python2.7/pickle.py", line 663, in _batch_setitems
    save(v)
  File "/usr/lib/python2.7/pickle.py", line 286, in save
    f(self, obj) # Call unbound method with explicit self
  File "/usr/lib/python2.7/pickle.py", line 748, in save_global
    (obj, module, name))
pickle.PicklingError: Can't pickle <built-in method copy_from_bbox of tuple object at 0x3180878>: it's not found as __main__.copy_from_bbox

The first pickle works, the second one does not.

@tacaswell

@tacaswell , thanks for taking a look. I agree - it seems to be the call to draw that is mucking things up. Your code produces an identical traceback on my Windows machine. Should this be opened as a new issue?

Re opened and tagged as 1.4.x as I am not sure that we want to try and get this fixed for 1.4.0. I suspect that issue is that we need to be more careful about what we let it try to export.

Cannot reproduce with current master.

More details:

Python	Matplotlib	Status
3.4	1.4.0	bad
3.4	1.4.1	good

Platform: Windows. Packages from Conda.

I think this must have been fixed by #3627; I also cannot reproduce with any of the above scripts.

Navigation Menu

Repository navigation

Can't pickle bar plots: Failed to pickle attribute "gridline" #1719

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Can't pickle bar plots: Failed to pickle attribute "gridline" #1719

Description

Activity

mspacek commented on Jan 30, 2013

pelson commented on May 9, 2013

mdboom commented on May 10, 2013

mspacek commented on May 11, 2013

mspacek commented on May 11, 2013

pelson commented on May 11, 2013

pelson commented on May 24, 2013

mspacek commented on May 24, 2013

pelson commented on May 28, 2013

mspacek commented on May 29, 2013

mdboom commented on Jun 4, 2013

dmcdougall commented on Jul 1, 2013

PrivateStaticVoid commented on Jun 7, 2014

tacaswell commented on Jun 7, 2014

PrivateStaticVoid commented on Jun 26, 2014

tacaswell commented on Jun 26, 2014

Kojoley commented on Aug 3, 2016

QuLogic commented on Oct 22, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions