Segfault from using probably invalid dtype

On a 64 bit machine this leads to a segfault.

In [1]: a = np.array([1], dtype='O').astype(('O', [('name', 'Q')]))

In [2]: a['name']

The type on the right should probably be illegal.

Simpler version of problem:

In [1]: dt = ('O', [('name', 'Q')])

In [2]: a = np.array([1], dtype=dt)

In [3]: a['name']
-- segfault --

It segfaults for me too:

In [1]: import numpy as np

In [2]: dt = ('O', [('name', 'Q')])

In [3]: a = np.array([1], dtype=dt)

In [4]: a['name']
*** glibc detected *** /home/ondrej/py27/bin/python: free(): invalid pointer: 0x0000000001b8846f ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x7eb96)[0x7fe289b7db96]
/home/ondrej/py27/bin/python[0x415b48]
/home/ondrej/py27/bin/python(PyParser_ASTFromString+0x79)[0x4c0aa9]
/home/ondrej/py27/bin/python(Py_CompileStringFlags+0x52)[0x4c0c32]
/home/ondrej/py27/bin/python[0x494faf]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x540d)[0x49cacd]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python[0x4ffe2c]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x4224cf]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x421512]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x3bfc)[0x49b2bc]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x60a2)[0x49d762]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python[0x4fff23]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x4224cf]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python[0x421512]
/home/ondrej/py27/bin/python(PyObject_Call+0x47)[0x41eda7]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x3bfc)[0x49b2bc]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x60a2)[0x49d762]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x60a2)[0x49d762]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalFrameEx+0x4e95)[0x49c555]
/home/ondrej/py27/bin/python(PyEval_EvalCodeEx+0x826)[0x49e676]
/home/ondrej/py27/bin/python(PyEval_EvalCode+0x32)[0x49e7a2]
/home/ondrej/py27/bin/python(PyRun_FileExFlags+0xb0)[0x4c0fa0]
/home/ondrej/py27/bin/python(PyRun_SimpleFileExFlags+0xd7)[0x4c1af7]
/home/ondrej/py27/bin/python(Py_Main+0xadd)[0x4156fd]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xed)[0x7fe289b2076d]
/home/ondrej/py27/bin/python[0x414811]
======= Memory map: ========
00400000-0056a000 r-xp 00000000 08:01 15475025                           /home/ondrej/py27/bin/python
00769000-0076a000 r--p 00169000 08:01 15475025                           /home/ondrej/py27/bin/python
0076a000-007a6000 rw-p 0016a000 08:01 15475025                           /home/ondrej/py27/bin/python
007a6000-007b5000 rw-p 00000000 00:00 0 
00a81000-01b95000 rw-p 00000000 00:00 0                                  [heap]
7fe27c000000-7fe27c021000 rw-p 00000000 00:00 0 
7fe27c021000-7fe280000000 ---p 00000000 00:00 0 
7fe280fbd000-7fe280ffb000 r-xp 00000000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe280ffb000-7fe2811fb000 ---p 0003e000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe2811fb000-7fe2811fc000 r--p 0003e000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe2811fc000-7fe281230000 rw-p 0003f000 08:01 15603261                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/random/mtrand.so
7fe281230000-7fe281231000 rw-p 00000000 00:00 0 
7fe281231000-7fe28123a000 r-xp 00000000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe28123a000-7fe281439000 ---p 00009000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe281439000-7fe28143a000 r--p 00008000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe28143a000-7fe28143b000 rw-p 00009000 08:01 15603266                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/fft/fftpack_lite.so
7fe28143b000-7fe281470000 r-xp 00000000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe281470000-7fe28166f000 ---p 00035000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe28166f000-7fe281670000 r--p 00034000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe281670000-7fe281671000 rw-p 00035000 08:01 15344829                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7fe281671000-7fe281686000 r-xp 00000000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281686000-7fe281885000 ---p 00015000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281885000-7fe281886000 r--p 00014000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281886000-7fe281887000 rw-p 00015000 08:01 2228268                    /lib/x86_64-linux-gnu/libgcc_s.so.1
7fe281887000-7fe28199b000 r-xp 00000000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe28199b000-7fe281b9b000 ---p 00114000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe281b9b000-7fe281b9c000 r--p 00114000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe281b9c000-7fe281b9e000 rw-p 00115000 08:01 15350352                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7fe281b9e000-7fe281efe000 r-xp 00000000 08:01 15606652                   /usr/lib/atlas-base/atlas/libblas.so.3gf.0
7fe281efe000-7fe2820fe000 ---p 00360000 08:01 15606652                   /usr/lib/atlas-base/atlas/libblas.so.3gf.0
7fe2820fe000-7fe282107000 rw-p 00360000 08:01 15606652                   /usr/lib/atlas-base/atlas/libblas.so.3gf.0
7fe282107000-7fe282a12000 r-xp 00000000 08:01 15606653                   /usr/lib/atlas-base/atlas/liblapack.so.3gf.0
7fe282a12000-7fe282c12000 ---p 0090b000 08:01 15606653                   /usr/lib/atlas-base/atlas/liblapack.so.3gf.0
7fe282c12000-7fe282c17000 rw-p 0090b000 08:01 15606653                   /usr/lib/atlas-base/atlas/liblapack.so.3gf.0
7fe282c17000-7fe282d24000 rw-p 00000000 00:00 0 
7fe282d24000-7fe282d29000 r-xp 00000000 08:01 15732509                   /home/ondrej/py27/lib/python2.7/site-packages/numpy/linalg/lapack_lite.so
7fe282d29000-7fe282f29000 ---p 00005000 08:01 15732509                   /home/oAborted (core dumped)

Oops. We should fix this segfault before the final release.

I'm not convinced it needs to be fixed immediately. There are always going to be bugs and if you wait to fix all of them you will never release. I think it would be better to wait 1.7.1 or 1.8.. This segfault is also occurrs in numpy 1.5.

It makes me thing that there are several spots in numpy where using a standard parser generator and a formal language desciption might be the way to go. It might clarify some of these things and make maintenance easier. I'm thinking dtypes and maybe casting rules.

Ok, that sounds good to me. In any case, I am now releasing rc1.

@charris

Removing the 1.7 milestone because I agree w/ @charris. We've already shipped this bug, so releasing 1.7 with this bug won't make anything worse for anyone.

Still present in 1.9-devel.

I don't really understand the dtype syntax but what is happening is that somewhere during its parsing it loses the NPY_NEEDS_INIT flag of the object dtype so it works on uninitialized memory later.

Merge pull request #8235 from J-Sand/invalid-structured-dtypes-fix

Fixed by #8235

Navigation Menu

Repository navigation

Segfault from using probably invalid dtype #2865

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Segfault from using probably invalid dtype #2865

Description

Activity

charris commented on Dec 28, 2012

certik commented on Dec 28, 2012

charris commented on Dec 28, 2012

certik commented on Dec 28, 2012

njsmith commented on Jan 2, 2013

charris commented on Feb 15, 2014

juliantaylor commented on Feb 15, 2014

ahaldane commented on Nov 25, 2016

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions