Skip to content

Bug in table indexing  #4672

Closed
Closed

Description

@joleroi

http://astropy.readthedocs.org/en/latest/table/indexing.html

Continuing the example from the docs page a little bit I get this

In [96]: t = Table([(1, 2, 3, 4), (10, 1, 9, 9)], names=('a', 'b'), dtype=['i8', 'i8'])

In [97]: t.add_index('a')

In [98]: t.loc[2]
Out[98]:

  a     b
int64 int64
----- -----
    2     1

In [99]: test = t[[1, 3]]

In [100]: test.indices
Out[100]:
[ a  rows
 --- ----
   2    0
   4    1]

In [101]: test.loc[2]
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
 in ()
----> 1 test.loc[2]

/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/astropy/table/index.pyc in __getitem__(self, item)
    794             key = self.table.primary_key
    795
--> 796         index = self.indices[key]
    797         if len(index.columns) > 1:
    798             raise ValueError("Cannot use .loc on multi-column indices")

/home/kingj/Software/miniconda3/envs/headsherpa/lib/python2.7/site-packages/astropy/table/index.pyc in __getitem__(self, item)
    756             raise IndexError("No index found for {0}".format(item))
    757
--> 758         return super(TableIndices, self).__getitem__(item)
    759
    760

TypeError: list indices must be integers, not NoneType

So it looks like the indices are updated correctly, but an error is thrown when calling loc. This does not seem like the desired behaviour.

Activity

  1. sas987 commented on Mar 16, 2016

    @sas987

    after compiling the above program i came to know that in astropy library the functions like add_index and loc have not been included into astropy library when installed into python framework,like i was compiling using spyder in anaconda framework,but it was throwing error that
    'Table' object has no attribute 'add_index'
    and so for the loc function
    plzz mentor reply to my above statement
    asap

  2. taldcroft commented on Mar 17, 2016

    @taldcroft
    Member

    @sas987 - I have just marked this issue as requiring expert knowledge of the Table package. It is not a good place for you to start.

  3. rirze commented on Mar 18, 2016

    @rirze
    Contributor

    So it seems that when a new Table is made from a slice of an existing one, it does not carry over its primary_key attribute. This is what is causing the error in index.py:

    key = self.table.primary_key here, primary_key is None
    index = self.indices[key]

    If one inserts a line of code at line 718 on table.py under _new_from_slice(...) :

    self._make_table_from_cols(table, newcols)
    table.primary_key = self.primary_key<-------this is the inserted line
    return table

    Then the error does not show. However, I'm not sure this is the right way to patch the error, or if the statement ought to be placed elsewhere.

  4. taldcroft commented on Mar 18, 2016

    @taldcroft
    Member

    Good catch @rirze! There is one other place to put the line, and for clarity I would move your inserted line up near the self.meta settings:

    diff --git a/astropy/table/table.py b/astropy/table/table.py
    index 9eb65a8..117c762 100644
    --- a/astropy/table/table.py
    +++ b/astropy/table/table.py
    @@ -664,6 +664,7 @@ class Table(object):
             table = data  # data is really a Table, rename for clarity
             self.meta.clear()
             self.meta.update(deepcopy(table.meta))
    +        self.primary_key = table.primary_key
             cols = list(table.columns.values())
    
             self._init_from_list(cols, names, dtype, n_cols, copy)
    @@ -703,6 +704,7 @@ class Table(object):
             table = self.__class__(masked=self.masked)
             table.meta.clear()
             table.meta.update(deepcopy(self.meta))
    +        table.primary_key = self.primary_key
             cols = self.columns.values()
    

    Can you go ahead and make these edits and add tests similar to:

    In [2]: t = simple_table()
    
    In [3]: t.add_index('a')
    
    In [4]: t2 = t[:]
    
    In [5]: t3 = t.copy()
    
    In [6]: t4 = Table(t)
    
    In [7]: t2.primary_key
    Out[7]: ('a',)
    
    In [8]: t3.primary_key
    Out[8]: ('a',)
    
    In [9]: t4.primary_key
    Out[9]: ('a',)
    

    (And of course make sure that .loc works as expected in each case.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions