Repository navigation
Setting masked column on non-masked table should raise error #4675
Description
Somewhat similar to #3505, I find this behavior surprising:
In [4]: t = Table([[1,2,3]], names='a')
In [5]: t
Out[5]:
<Table length=3>
a
int64
-----
1
2
3
In [6]: t['a'] = MaskedColumn(data=[1,2,3], name='a', mask=[True, False,False])
In [7]: t
Out[7]:
<Table length=3>
a
int64
-----
1
2
3
This is due to Numpy's behavior (setting values from a masked array to a normal copies the values and discards the mask), but maybe it should be prevented ?
Activity
astrofrog commentedon Apr 19, 2016
@saimn - I think maybe it should work but raise a warning if any of the mask values are True. I can think of a use case where a user reads a table from a file and it happens to return a maskedcolumn but with no actual masked values.
@taldcroft - what do you think?
pllim commentedon Apr 19, 2016
@astrofrog , I agree with you.
saimn commentedon Apr 19, 2016
Yes, a warning is fine also, I have no strong preference.
taldcroft commentedon Apr 19, 2016
@saimn - I'm -0.5 on raising a warning like this. As you said this is really an upstream issue since the implementation here is completely delegating to numpy:
I think that this is really another instance of being surprised that when you set a column it does an in-place assignment instead of overwriting the entire column. Numpy structured array shares this behavior. Once you have that in your head, then it should not be surprising that when you assign a masked array to an unmasked Column then the result will not have masked values.
Interestingly, Pandas does mutate column types on demand...
taldcroft commentedon Apr 20, 2016
BTW, I was just playing around with pandas and I find it's dtype-changing behavior either confusing or maybe downright buggy:
saimn commentedon May 1, 2018
Closing this old issue, as I agree that the behavior is consistent with Numpy.