Skip to content

Table caches column print formats for too long #5800

Closed

Description

@mhvk

If a format for a column is given and a table is printed, the corresponding format function is cached. Unfortuately, this means it will also get used on another table holding a column with the same name:

from astropy.table import Table, QTable, Column
t = Table([Column([1.,2.], unit='m', name='a')])
t['a'].format = '%f'
t
# <Table length=2>
#    a   
#    m   
# float64
# -------
#     1.0
#     2.0
QTable(t)
# ... ---------------------------------------------------------------------------
# UnitConversionError

Not quite sure whether it would be best to clear the cache in various places or to index columns by something else than their name. It is not quite obvious to me speed is an issue here, since printing is so slow anyway.

Activity

  1. taldcroft commented on Feb 13, 2017

    @taldcroft
    Member

    Caching by the object ID seems like it might a fair bit more reliable?

    diff --git a/astropy/table/pprint.py b/astropy/table/pprint.py
    index c523539..846f8a1 100644
    --- a/astropy/table/pprint.py
    +++ b/astropy/table/pprint.py
    @@ -419,7 +419,7 @@ class TableFormatter(object):
             col_format = col.info.format or getattr(col.info, 'default_format', None)
             pssf = (getattr(col.info, 'possible_string_format_functions', None) or
                     _possible_string_format_functions)
    -        auto_format_func = get_auto_format_func(col.info.name, pssf)
    +        auto_format_func = get_auto_format_func(id(col), pssf)
             format_key = (col_format, col.info.name)
             format_func = _format_funcs.get(format_key, auto_format_func)
    

    At least this patch fixes your particular problem. I'm not too inclined to remove the caching unless we have a reliable way to prove that this will not slow things down.

  2. mhvk commented on Feb 13, 2017

    @mhvk
    ContributorAuthor

    Yes, that was my first thought too, and I think we should go for that at least for now (I ran into it in writing the [Q]Table vstack-ing tests, where I run the same tests for both, and then the second one mysteriously failed).

    An alternative (but could be done later), would be to store the format function on info -- this has the advantage that it could be created upon setting info.format, i.e., one would get instant validation.

  3. taldcroft commented on Feb 13, 2017

    @taldcroft
    Member

    See #5803 for the easier solution.

    One issue with instant validation is that you can't always do it because you might not actually have data in the column at the time of defining the format.

  4. mhvk commented on Feb 13, 2017

    @mhvk
    ContributorAuthor

    True -- since you now have a simple fix in the works, which will close this issue, I created a new one as a reminder that perhaps we may want to rethink how this is implemented: #5804.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions