Python 3.3 support (was: error in h5py on python 3.3.0)

Original author: [email protected] (November 23, 2012 13:25:42)

Python 3.3.0, Linux 64bit

The import mechanism has changed in python 3.3.0 and the local imports
in the cython files h5*.pyx do not work anymore.

These imports now read:

import _conv

But need to be:

from h5py import _conv

Note that the cimports still work fine

Original issue: http://code.google.com/p/h5py/issues/detail?id=251

From [email protected] on December 09, 2012 19:31:16
Thanks for the patch!

Other tasks for Python 3.3 support:

New Jenkins builders on UNIX and Windows buildslaves
Check compatible NumPy versions on Windows

Addressing h5py#251 by switching to absolute imports in .pyx files.

Merge pull request #273 from MatthewTurk/issue_251

Fixed in PR from Matthew Turk. Evidently this was an issue with versions of cython < 0.18.

This fixes it if I do it manually, but it would be good to have an updated version online for download. Version 2.1.3 does not have the fix, and it still causes trouble on Fedora 18 (cpython 0.18), in fact, the Fedora-18 package python3-h5py has this bug built in an does not work.

Yes, this will be part of h5py 2.2.0 (our next release). I'm sorry to hear the Fedora package is broken. The fix is relatively easy to backport; I encourage you to contact the Fedora maintainer.

Dear Andrew,

yes, I did file a bug report with the Fedora team. No reply yet. I was
surprised they did not test it, as it obviously fails. I just put in Matt
Turk's fix in my local copy of 2.1.3 for now but also submitted the fix tar
ball to fedora.

Best wishes,
Alexander

On Wed, May 15, 2013 at 10:34 AM, andrewcollette
[email protected]:

Yes, this will be part of h5py 2.2.0 (our next release). I'm sorry to hear
the Fedora package is broken. The fix is relatively easy to backport; I
encourage you to contact the Fedora maintainer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/251#issuecomment-17914025
.

Dear Andrew,

yes, I did file a bug report with the Fedora team. No reply yet. I
was surprised they did not test it, as it obviously fails. I just put
in Matt Turk's fix in my local copy of 2.1.3 for now but also
submitted the fix tar ball to fedora.

Best wishes,
Alexander

On Wed, May 15, 2013 at 10:34 AM, andrewcollette
[email protected] wrote:

Yes, this will be part of h5py 2.2.0 (our next release). I'm sorry to hear the Fedora package is broken. The fix is relatively easy to backport; I encourage you to contact the Fedora maintainer.

—
Reply to this email directly or view it on GitHub.

Dear Andrew,

not sure whether these emails get through to you ...
I found that 2.1.3 also has problems with np.ndarray dtype 'U' that the
current np 1.7.1 / python 3.3 generates from strings. Only when bytes are
assigned using str.encode() does it work, whereas Python 3.3 strings are
accepted in other places.

Can you add - or have already - support for this (transparently?) to the
next version (2.2.0?)?
Or is there a limitation in hdf-5 in the first place?

Best wishes,
Alexander

On Wed, May 15, 2013 at 10:34 AM, andrewcollette
[email protected]:

Yes, this will be part of h5py 2.2.0 (our next release). I'm sorry to hear
the Fedora package is broken. The fix is relatively easy to backport; I
encourage you to contact the Fedora maintainer.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/251#issuecomment-17914025
.

Yes; you can always get in touch with me at the h5py mailing list (h5py at google groups) as well.

The "U" issue is a limitation of HDF5 (it doesn't support wide characters). There is a proposal pending to automatically convert "U" strings to variable-length strings in object arrays, although it's a bit clunky.

Could a bugfix release in the 2.1.x series be made ?

Python 3.3 is now 3.3.2, the official latest stable Python, and this is rather problematic for packages depending on h5py and supporting Python 3.3.

A bug-fixed package for Fedora 18 was released, and should be included in
Fedora 19 as well.

On Thu, Jun 27, 2013 at 7:25 PM, Laurent Gautier
[email protected]:

Could a bugfix release in the 2.1.x series be made ?

Python 3.3 is now 3.3.2, the official latest stable Python, and this is
rather problematic for packages depending on h5py and supporting Python 3.3.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/251#issuecomment-20106290
.

That's nice, but the bugfix release should be upstream and on Pypi (and many people are not using Fedora)

I was told it was a problem that supposedly is resolved with use of the
current version of cython. I will see how things look with Fedora 19 next
week that should have all packages updated.

I can send you the changes/fix for 3.3 if you like .. but I think it is not
compatible with 2.7.
On Jun 27, 2013 8:39 PM, "Laurent Gautier" [email protected] wrote:

That's nice, but the bugfix release should be upstream and on Pypi (and
many people are not using Fedora)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/251#issuecomment-20110306
.

When building from a git clone, you will first have to do:

python api_gen.py

inside the "h5py" subdirectory. This generates defs.pyx, among other
things.

On Thu, Jun 27, 2013 at 2:21 PM, Laurent Gautier
[email protected]:

Thanks for looking into it.

I think that one file was not committed. I can't build the branch 2.1 with
the lastest release of cython and Python 3.3.1.
It ends with:

running build_ext
cythoning h5py/defs.pyx to h5py/defs.c
error:/foo/bar/h5py/h5py/defs.pyx: No such file or directory

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/251#issuecomment-20152255
.

I see.
Still not working though:

In [1]: import h5py
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-1-e81388eeaef8> in <module>()
----> 1 import h5py

/foo/bar/h5py/h5py/__init__.py in <module>()
----> 1 from h5py import _errors
      2 _errors.silence_errors()
      3 
      4 from h5py import _conv
      5 _conv.register_converters()

ImportError: cannot import name _errors

Can you post the full output? That error tends to be raised when the build process goes awry.

I found the problem: h5py can't be imported while the current working directory is the source tree for h5py (otherwise Python picks the directory h5py/ rather than the installed module).

Now the package is importing fine but I can't run the tests. h5py.run_tests() ends with:

ImportError: 'test_file' module incorrectly imported from '/foo/bar/py33_env/lib/python3.3/site-packages/h5py/_hl/tests'. Expected '/foo/bar/h5py/h5py/_hl/tests'. Is this module globally installed?

py33_env/ is a virtualenv, and h5py/ is the directory with the git clone for h5py. I guessing that this path is hard-coded at build time (and may be it should not).

Never seen that one before! What if you just do python setup.py test from the installation directory?

@lgautier

Hmm, how would I check that?

(probably you don't want to invest you time explaining all the details
of the pipeline) Just download the recent package and compile or need
to do a git/svn checkout of the current development version?

-Alexander

On 28/06/13 01:39, andrewcollette wrote:

Given that 2.2 is taking so long, I think it's reasonable to make one more release in the 2.1 series.

I've backported the fix to the 2.1 branch (35c2961); it imports correctly on my build of 3.3 and Cython. @lgautier, @2sn, if you have a moment to verify that it works as intended I will wrap it up with a few other fixes and make a release on Monday.

Reply to this email directly or view it on GitHub:
#251 (comment)

@2sn

@2sn You'll need a git clone and switch to the 2.1 branch.

@andrewcollette

@andrewcollette : things are not working from the install directory (see my comment right before yours).

I can't run the test suite, but h5py seems to be working with the code I am working on (I have unit tests checking what is coming out of HDF5 - I am only using a small part of the h5py API thought).

@lgautier

@lgautier, the tests should work from the installation directory; setup.py points Python at the files in build. Glad to hear it works with your code, though.

FYI, packaging for 2.1.4 will be delayed until sometime next week; I am still getting weird errors on my build of python3.3/h5py2.1 related to complex types.

I got the same error importing h5py on Ubuntu 13.04, python 3.3.1 eventhough the test suite completed as expected.

I ran the test suite and got:
~/temp/h5py-2.1.3$ python3 setup.py test
running test
running build_py
running build_ext
skipping 'h5py/defs.c' Cython extension (up-to-date)
skipping 'h5py/_errors.c' Cython extension (up-to-date)
skipping 'h5py/_objects.c' Cython extension (up-to-date)
skipping 'h5py/_proxy.c' Cython extension (up-to-date)
skipping 'h5py/h5fd.c' Cython extension (up-to-date)
skipping 'h5py/h5z.c' Cython extension (up-to-date)
skipping 'h5py/h5.c' Cython extension (up-to-date)
skipping 'h5py/h5i.c' Cython extension (up-to-date)
skipping 'h5py/h5r.c' Cython extension (up-to-date)
skipping 'h5py/utils.c' Cython extension (up-to-date)
skipping 'h5py/_conv.c' Cython extension (up-to-date)
skipping 'h5py/h5t.c' Cython extension (up-to-date)
skipping 'h5py/h5s.c' Cython extension (up-to-date)
skipping 'h5py/h5p.c' Cython extension (up-to-date)
skipping 'h5py/h5d.c' Cython extension (up-to-date)
skipping 'h5py/h5a.c' Cython extension (up-to-date)
skipping 'h5py/h5f.c' Cython extension (up-to-date)
skipping 'h5py/h5g.c' Cython extension (up-to-date)
skipping 'h5py/h5l.c' Cython extension (up-to-date)
skipping 'h5py/h5o.c' Cython extension (up-to-date)
skipping 'h5py/h5ds.c' Cython extension (up-to-date)

....................................................s.....................................ssssss.......................................................................................................................s...........................x..

Ran 246 tests in 0.587s

OK (skipped=8, expected failures=1)

latest build from git works now for me on Python 3.3.1.... thanks for fixing. h5py is a critical tool for me.

Python 3.3 is now officially tested & supported in 2.2.0. Thanks to everyone who helped get this working!

Dear Andrew,

thanks a lot for taking my feedback into account!

Best wishes,
Alexander

On Wed, Sep 4, 2013 at 2:00 AM, andrewcollette [email protected]:

Python 3.3 is now officially tested & supported in 2.2.0. Thanks to
everyone who helped get this working!

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/251#issuecomment-23724020
.

Navigation Menu

Repository navigation

Python 3.3 support (was: error in h5py on python 3.3.0) #251

5 remaining items

....................................................s.....................................ssssss.......................................................................................................................s...........................x..

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Python 3.3 support (was: error in h5py on python 3.3.0) #251

Description

Activity

andrewcollette commented on Dec 27, 2012

andrewcollette commented on Mar 14, 2013

2sn commented on May 12, 2013

andrewcollette commented on May 15, 2013

2sn commented on May 15, 2013

2sn commented on May 15, 2013

2sn commented on May 17, 2013

andrewcollette commented on May 17, 2013

lgautier commented on Jun 27, 2013

2sn commented on Jun 27, 2013

lgautier commented on Jun 27, 2013

2sn commented on Jun 27, 2013

5 remaining items

andrewcollette commented on Jun 27, 2013

lgautier commented on Jun 27, 2013

andrewcollette commented on Jun 27, 2013

lgautier commented on Jun 27, 2013

andrewcollette commented on Jun 27, 2013

2sn commented on Jul 1, 2013

lgautier commented on Jul 1, 2013

lgautier commented on Jul 1, 2013

andrewcollette commented on Jul 1, 2013

andrewcollette commented on Jul 2, 2013

1rocketdude commented on Jul 5, 2013

....................................................s.....................................ssssss.......................................................................................................................s...........................x..

1rocketdude commented on Jul 20, 2013

andrewcollette commented on Sep 3, 2013

2sn commented on Sep 3, 2013

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions