Skip to content

bundled six 1.9.0 causes ImportError: No module named 'winreg' in Pympler #6537

Closed

Description

@rathann

To help us understand and resolve your issue please check that you have provided
the information below.

  • Matplotlib version, Python version and Platform (Windows, OSX, Linux ...)

1.5.1, 2.7.11/3.5.1, Fedora rawhide

  • How did you install Matplotlib and Python (pip, anaconda, from source ...)

From Fedora package.

  • If possible please supply a Short, Self Contained, Correct, Example
    that demonstrates the issue i.e a small piece of code which reproduces the issue
    and can be run with out any other (or as few as possible) external dependencies.

Try building Pympler (http://pythonhosted.org/Pympler/) and running its tests (python3 setup.py test). On armv7hl, I get this exception:

Traceback (most recent call last):
  File "/builddir/build/BUILD/Pympler-0.4.3/test/tracker/test_classtracker.py", line 178, in test_snapshot_members
    self.tracker.create_snapshot(compute_total=True)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/classtracker.py", line 567, in create_snapshot
    snapshot.asizeof_total = asizeof.asizeof(all=True, code=True)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 2191, in asizeof
    s = _asizer.asizeof(*t)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1783, in asizeof
    s, _ = self._sizes(objs, None)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1753, in _sizes
    s[i] = self._sizer(o, 0, sized)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1729, in _sizer
    s += z(o, d, None)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1729, in _sizer
    s += z(o, d, None)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1729, in _sizer
    s += z(o, d, None)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1729, in _sizer
    s += z(o, d, None)
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 1728, in _sizer
    for o in r(obj, False):
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 536, in _refs
    for _, o in _dir2(obj, **kwds):
  File "/builddir/build/BUILDROOT/python-Pympler-0.4.3-1.fc25.noarch/usr/lib/python3.5/site-packages/pympler/asizeof.py", line 347, in _dir2
    if hasattr(obj, slots):
  File "/usr/lib/python3.5/site-packages/matplotlib/externals/six.py", line 116, in __getattr__
    _module = self._resolve()
  File "/usr/lib/python3.5/site-packages/matplotlib/externals/six.py", line 113, in _resolve
    return _import_module(self.mod)
  File "/usr/lib/python3.5/site-packages/matplotlib/externals/six.py", line 80, in _import_module
    __import__(name)
ImportError: No module named 'winreg'

The bundled six.py has this:

class _MovedItems(_LazyModule):
    """Lazy loading of moved objects"""
    __path__ = []  # mark as package


_moved_attributes = [
...
    MovedModule("winreg", "_winreg"),
]

while python-six-1.10.0 (latest version) has this:

# Add windows specific modules.
if sys.platform == "win32":
    _moved_attributes += [
        MovedModule("winreg", "_winreg"),
    ]

I applied this patch:

diff -up matplotlib-1.5.1/setupext.py.six matplotlib-1.5.1/setupext.py
--- matplotlib-1.5.1/setupext.py.six    2016-01-10 23:20:20.000000000 +0100
+++ matplotlib-1.5.1/setupext.py    2016-05-18 13:44:21.534494158 +0200
@@ -1026,6 +1026,18 @@ class Qhull(SetupPackage):
             ext.sources.extend(glob.glob('extern/qhull/*.c'))


+class Six(SetupPackage):
+    name = "six"
+
+    def check(self):
+        try:
+            import six
+        except ImportError:
+            return 'not found. pip may install it below.'
+
+        return 'version %s' % six.__version__
+
+
 class TTConv(SetupPackage):
     name = "ttconv"

diff -up matplotlib-1.5.1/setup.py.six matplotlib-1.5.1/setup.py
--- matplotlib-1.5.1/setup.py.six   2016-01-10 23:20:20.000000000 +0100
+++ matplotlib-1.5.1/setup.py   2016-05-18 13:44:21.528494040 +0200
@@ -84,7 +84,7 @@ mpl_packages = [
     setupext.Delaunay(),
     setupext.QhullWrap(),
     setupext.Tri(),
-    setupext.Externals(),
+    setupext.Six(),
     'Optional subpackages',
     setupext.SampleData(),
     setupext.Toolkits(),

and replaced all imports from bundled six to system-wide six without any adverse effects. Please consider unbundling six and using system-wide version.

Activity

  1. QuLogic commented on Jun 5, 2016

    @QuLogic
    Member

    Fedora packages correctly replace internal six with the external one; are you not using those? This directory /usr/lib/python3.5/site-packages/matplotlib/externals/ should not exist in Fedora packages.

  2. rathann commented on Jun 5, 2016

    @rathann
    Author

    Yes, I've just implemented that in the latest build. If you're fine with Fedora carrying a downstream patch, then I won't push any further, but it would be nice to have official support for using system-wide six.

  3. tacaswell commented on Jun 6, 2016

    @tacaswell
    Member

    Why doesn't this issue show up everywhere else?

    The discussion over bundling six (#4501) was protracted and I am not excited about re litigating that.

    Might be worth updating to 1.10, but the master branch is going to go python3 only ~18mo so this is not a pressing issue.

    A simpler patch downstream might be to replace the contents of external/six.py with

    from six import *
  4. matthew-brett commented on Jun 6, 2016

    @matthew-brett
    Contributor

    As the person who proposed and implemented the bundling - I don't feel strongly - I am happy to reverse it if you like.

  5. QuLogic commented on Jun 6, 2016

    @QuLogic
    Member

    from six import *

    No, six does some complicated monkey-patching; it doesn't work if you do that. I tried to do that on a different package.

  6. tacaswell commented on Jun 6, 2016

    @tacaswell
    Member

    @matthew-brett What has changed your mind from last year?

  7. rathann commented on Jun 6, 2016

    @rathann
    Author

    @tacaswell :

    Why doesn't this issue show up everywhere else?

    Probably because I've been able to reproduce it only on armv7hl so far and only while running Pympler testsuite. Debian doesn't package Pympler, for example.

  8. matthew-brett commented on Jun 6, 2016

    @matthew-brett
    Contributor

    @tacaswell - I think in retrospect I was wrong last year, pip was easily up to to the job, but in any case, after another year of reliable pip, it seems even harder to justify now.

  9. tacaswell commented on Jun 6, 2016

    @tacaswell
    Member

    if @matthew-brett is on board with reverting this change then 👍 on reverting for 2.0 (would rather not do it for 1.5.2 as that is mostly held for cleaning up the other issues from @rathann at this point).

  10. added this to the milestone on Jun 6, 2016
  11. jenshnielsen commented on Jun 13, 2016

    @jenshnielsen
    Member

    Closed by #6556

  12. megies commented on Sep 26, 2017

    @megies
    Contributor

    It seems this bug is affecting the matplotlib package in Ubuntu xenial which is a long term stable release, so this is something that can hurt people for years to come.
    I'm not sure how to best fix this, as I don't know if Ubuntu packaging team would consider a patch or would prepare an updated package (not sure about their policy for non-security-related bugfixes). Any ideas? See obspy/obspy/issues/1906.

  13. tacaswell commented on Sep 26, 2017

    @tacaswell
    Member

    I am not convinced that obspy/obspy#1906 is the same issue. This is a bug in the vendored version of six, that issue looks like the OP has crossed installations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions