-
-
Notifications
You must be signed in to change notification settings - Fork 2k
WIP: move column/mixin formatting to Info #6720
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Hi there @taldcroft 👋 - thanks for the pull request! I'm just a friendly 🤖 that checks for issues related to the changelog and making sure that this pull request is milestoned and labeled correctly. This is mainly intended for the maintainers, so if you are not a maintainer you can ignore this, and a maintainer will let you know if any action is required on your part 😃. I see this is a work in progress pull request. I'll report back on the checks once the PR is ready for review. If there are any issues with this message, please report them here. |
|
See also #6721 |
mhvk
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the general idea very much - see one question about possible problems with masked columns.
|
|
||
| try: | ||
| # test whether it formats without error exemplarily | ||
| self.pformat(max_lines=1) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this try/except work also for a MaskedColumn in which the first item is masked? I recall having to work around that case in the printing routines... It could either fail on the masked item, or fail to raise on something that would fail on a real number.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, this is a problem (though not one introduced by this PR). But there are mitigating factors:
- The actual smallest allowed value of
max_linesis 7. This allows the internal code to be a bit simpler and not deal with the problems that would ensue from formatting only 1 line. So for a typical table you actually get the first two and last two lines checked. - In the event that all these are masked, the worst case is that later on one gets the same exception you would have seen when setting the
format.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
.. and I found a bug, see #6812. This should be a very quick review.
| lines, outs = _pformat_col(self, max_lines, show_name=show_name, | ||
| show_unit=show_unit, show_dtype=show_dtype, | ||
| html=html) | ||
| # _pformat_col = self._formatter._pformat_col |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just in case: should be removed not commented in a final PR.
|
@mhvk - I've been thinking about going the whole way and putting all the column attributes into So then |
|
@taldcroft - I think moving everything to |
|
Hi humans 👋 - this pull request hasn't had any new commits for approximately 5 months. I plan to close this in a month if the pull request doesn't have any new commits by then. In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock. You may also consider sending a reminder e-mail about it to the astropy-dev mailing list. If you believe I commented on this pull request incorrectly, please report this here. |
|
@taldcroft - I liked this PR a lot. Do you want to revive it? I'm not sure any more what exactly the problems were with masked arrays, but can look into that. |
|
⏰ Time's up! ⏰ I'm going to close this pull request as per my previous message. If you think what is being added/fixed here is still important, please remember to open an issue to keep track of it. Thanks! If this is the first time I am commenting on this issue, or if you believe I closed this issue incorrectly, please report this here. |
|
This would help with #7481, which adds an attribute to |
|
As I wrote above, I thought this was a great idea (and with classes over I have a bit more time to do reviews...) |
94cbac1 to
2f5d6f7
Compare
|
@mhvk - I've rediscovered why I dropped this the first time around, and the problems related to MaskedArray and #6721. Basically I just don't understand what the heck is going on in numpy and MaskedArray. The output of This is beyond my numpy-fu, but maybe you could make sense of this? Definitely all this nonsense is going to make MaskedColumn slicing really slow. |
|
OK, that seems like a nice puzzle.... I'll have a look. Just for information: I presume it is not as bad for regular |
|
For a regular column this behaves like I would expect: |
|
@taldcroft - I think the ultimate problem is like in #7394 (why So, one solution is the hack I proposed in #7394 - with that one only gets 2 p.s. I did check and it seems all table tests still pass with the hack in #7394 |
4901e59 to
19df182
Compare
|
@mhvk - In a5ac8a6 I did the hack from #7394 => #6721 => https://github.com/astropy/astropy/compare/master...mhvk:table-masked-column-array-finalize?expand=1 I might have done something wrong because tests are not passing for me, in particular the test on |
|
I rebased the hack on master, and Now specifically on the problem you noted: looking at the MaskedArray code, I see that in
Now, I just tried on your PR all the steps above, and steps 1-4 are all OK, but step 5 not only does not set |
|
Much thinking... I think the problem is that So, I can solve this problem (and get the format transferred also in step 5), by changing as follows: But: this is not very logical. More logical would be to ensure that p.s. I think the whole loop looking for attrs can be skipped if |
|
Hi humans 👋 - this pull request hasn't had any new commits for approximately 11 months. I plan to close this in a month if the pull request doesn't have any new commits by then. In lieu of a stalled pull request, please consider closing this and open an issue instead if a reminder is needed to revisit in the future. Maintainers may also choose to add If this PR still needs to be reviewed, as an author, you can rebase it to reset the clock. If you believe I commented on this pull request incorrectly, please report this here. |
|
@taldcroft - still keen to move this forward. Hopefully, #8855 will make it easier. |
|
I just don't know what to do about this: Throwing away 8 µs for every single column attribute would really be a bummer. Maybe the |
|
@taldcroft - definitely a good point about the speed, though not all of it counts. For instance, while I confirm your 26-times slower result, One should really imagine the Profiling just doing |
|
See #8968 for a substantial speed-up with a minimal change. |
This is a WIP of moving all the column formatting infrastructure into
Info. In particular this makes Quantity much more like Column. This would also apply to other mixins where it makes sense.But.. this re-introduced some weird regression, possibly related to MaskedArray and
__array_finalize__that was fixed in #3023. Not clear at this point. @mhvk if you have ideas let me know!