Skip to content

Bug with Table sorting? #829

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
taldcroft merged 1 commit into from
Mar 5, 2013
Merged

Bug with Table sorting? #829

taldcroft merged 1 commit into from
Mar 5, 2013

Conversation

@taldcroft
Copy link
Member

@taldcroft taldcroft commented Mar 1, 2013

If I try and read in the following file:

http://www.mpia.de/~robitaille/python/rosat.vot

and sort according to the counts:

In [1]: from astropy.table import Table

In [2]: t = Table.read('rosat.vot')

In [3]: t.keep_columns(['RAJ2000', 'DEJ2000', 'Count'])

In [4]: t.sort('Count')

I get some weird results:

  • print(t) shows an unsorted table:
In [5]: print(t)
 RAJ2000   DEJ2000    Count  
--------- --------- ---------
  0.00000 -39.48403      0.13
  0.02917   8.28153      0.19
  0.04167 -63.59528      0.19
  0.04958   5.38833      0.26
  0.05250   1.77250     0.081
  0.05625  57.94125      0.12
  0.08125 -26.17556      0.12
  0.14792 -28.09819     0.072
  0.16000  79.67694       0.1
  0.17708  62.17611      0.16
  0.18417 -26.08931      0.12
  0.18583 -25.84181      0.05
  • t.more() shows a sorted table:
In [7]: t.more()

 RAJ2000   DEJ2000    Count  
--------- --------- ---------
  2.24333 -23.88486      0.05
  7.83833  30.26611      0.05
 17.45833  19.65736      0.05
 20.64042 -22.90139      0.05
 36.40958  37.54028      0.05
 39.69792  40.18306      0.05
 64.73541  13.23681      0.05
 82.30708  -6.25097      0.05
 93.72875 -60.65736      0.05
 97.19500 -68.13250      0.05
107.63583  -8.70889      0.05
110.80459  58.69194      0.05
125.11417 -22.92417      0.05
  • t['Count'] shows an unsorted column:
In [8]: t['Count']
Out[8]: 
<MaskedColumn name='Count' units='ct / (s)' format='%9.2g' description='? Source countrate (4)'>
masked_array(data = [0.13099999725818634 0.18700000643730164 0.1899999976158142 ...,
 0.06639999896287918 0.05249999836087227 0.11500000208616257],
             mask = [False False False ..., False False False],
       fill_value = 1e+20)

Not sure what's going on! Any ideas?

@ghost ghost assigned taldcroft Feb 26, 2013
@taldcroft
Copy link
Member

taldcroft commented Feb 27, 2013

My initial guess is that the line in sort self._data.sort(order=keys) is not playing well with the Column views into self._data. Maybe sometimes the view is preserved, maybe sometimes not.

Probably sort and reverse should re-make the Columns to ensure integrity of the views into self._data. The code in add_row() is probably a good starting point:

        # Add_row() probably corrupted the Column views of self._data.  Rebuild
        # self.columns.  Col.copy() takes an optional data reference that it
        # uses in the copy.
        cols = [self.ColumnClass(c.name, c).copy(self._data[c.name]) for c in self.columns.values()]
        self.columns = TableColumns(cols)

I can look at this later tomorrow.

@taldcroft
Add new Table method _rebuild_table_column_views
e198b30 
Some table manipulations can corrupt the Column views of self._data.
This function will cleanly rebuild the columns and self.columns.  This
is now used in `add_row()`, `sort()`, and `reverse()`.  It should likely
be used for any method that modifies `self._data`.

This is exactly the code that had been at the end of `Table.add_row()`
but now written out in a more understandable fashion with a bunch of
comments to explain some of the subtleties that I had to rediscover.
@taldcroft
Copy link
Member

taldcroft commented Mar 1, 2013

@astrofrog - I think this is fixed now.

@astrofrog
Copy link
Member Author

astrofrog commented Mar 5, 2013

@taldcroft - looks good to me! Feel free to merge.

taldcroft added a commit that referenced this pull request Mar 5, 2013
@taldcroft
Merge pull request #829 from taldcroft/table/bug-sort
5481c82 
Bug with Table sorting.  This also affected the reverse method.
@taldcroft taldcroft merged commit 5481c82 into astropy:master Mar 5, 2013
@taldcroft taldcroft deleted the table/bug-sort branch March 5, 2013 16:34
@taldcroft taldcroft mentioned this pull request Mar 5, 2013
Merged
embray pushed a commit to embray/astropy that referenced this pull request Mar 29, 2013
@taldcroft @embray
Merge pull request astropy#829 from taldcroft/table/bug-sort
0f59e6e 
Bug with Table sorting.  This also affected the reverse method.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@taldcroft taldcroft

Labels

Bug table

Projects

None yet

Milestone

v0.2.1

Development

Successfully merging this pull request may close these issues.

2 participants

@taldcroft @astrofrog