record array is made up of numpy.void, not numpy.record

I raised this previously on the PyTables issue list, but it they have asked me to transfer to numpy. (PyTables/PyTables#271)

>>> import numpy as np
>>> a = (1, 2, 3)
>>> ra = np.rec.fromrecords([a])
>>> na = np.array([a], dtype='i8,i8,i8').view(type=np.recarray)
>>> na
rec.array([(1, 2, 3)], 
      dtype=[('f0', '<i8'), ('f1', '<i8'), ('f2', '<i8')])
>>> ra
rec.array([(1, 2, 3)], 
      dtype=[('f0', '<i8'), ('f1', '<i8'), ('f2', '<i8')])
>>> type(ra)
<class 'numpy.core.records.recarray'>
>>> type(na)
<class 'numpy.core.records.recarray'>
>>> ra[0]
(1, 2, 3)
>>> na[0]
(1, 2, 3)
>>> type(ra[0])
<class 'numpy.core.records.record'>
>>> type(na[0])
<class 'numpy.void'>

Any reason why .view can't return record arrays instead of sort of fake record arrays, or voidarrays? Is there a cast missing in the code somewhere?

Just wondering if anyone is in a position to confirm/deny this as an issue?

Bueller?
Bueller?

bump for this little annoyer - I can confirm this bug and it regularly affects me... accessing structure arrays with ['fieldname'] as the longtime workaround just looks so wrong to boot...

Anything that could be done to fix this issue would be appreciated... recarrays are powerful but this issue makes them tricky to use.

@mhvk

@mhvk when I wrote those PRs I didn't know about __array_finalize__, but now I do and I was just thinking about that this morning. It looks like adding

def __array_finalize__(self, obj):
    self.dtype = dtype(record, self.dtype)

to np.recarray works. Is that the right way to change the dtype?

Edit: I spoke too soon that doesn't work

@ahaldane

@ahaldane - yes, in principle, though be careful that you can get into __array_finalize__ via different routes [1]. You may want to do something like

def __array_finalize__(self, obj):
    if obj is not None and type(obj) is not type(self):
        self.dtype = dtype(record, obj.dtype)

This way nothing gets changed when initializing or slicing a recarray.

Though in practice, since for initialization or slicing the dtype is already correct, you could also just check whether self.dtype is correct and only change it if it is not.

[1] http://docs.scipy.org/doc/numpy/user/basics.subclassing.html#the-role-of-array-finalize

@mhvk

@mhvk I see. I think I got it to work (I was missing ()) with this:

def __array_finalize__(self, obj):
    if obj is not None and type(obj) is not type(self): 
        self.dtype = sb.dtype((record, self.dtype))

then I see it works:

>>> a = np.zeros(2 dtype='i4,f4')
>>> a.view(np.recarray)
rec.array([(0, 0.0), (0, 0.0)], 
      dtype=[('f0', '<i4'), ('f1', '<f4')])
>>> a.view(np.recarray).dtype
dtype((numpy.record, [('f0', '<i4'), ('f1', '<f4')]))

I'll submit a PR. Good thing this came up for discussion again!

@ahaldane

@ahaldane - great! hope this solves @embray's issues too...

@mhvk

@mhvk I was thinking this as well. I guess I figured if you take a view with type=np.recarray but not with dtype=(np.record, arr.dtype) then maybe it would be consistent. But I agree it's actually more consistent that recarray objects always return elements with np.record type, and your suggestion is exactly how that is done.

@abalkin

@abalkin would be good to have a PR for this. Does defining __array_finalize_array solve any of the problems for which there are other PRs?

Navigation Menu

Repository navigation

record array is made up of numpy.void, not numpy.record #3581

10 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

record array is made up of numpy.void, not numpy.record #3581

Description

Activity

potyt commented on Sep 16, 2013

potyt commented on May 1, 2014

nevion commented on Oct 9, 2014

dalexander commented on Jun 2, 2015

10 remaining items

ahaldane commented on Jun 4, 2015

mhvk commented on Jun 4, 2015

ahaldane commented on Jun 4, 2015

mhvk commented on Jun 4, 2015

embray commented on Jun 4, 2015

charris commented on Jun 5, 2015

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions