Skip to content

Fix access to VLA data after slicing - #16996

Merged
pllim merged 1 commit into
astropy:mainfrom
saimn:fits-16990
Oct 16, 2024
Merged

pllim merged 1 commit into
astropy:mainfrom
saimn:fits-16990

Conversation

@saimn

@saimn saimn commented Sep 11, 2024

Copy link
Copy Markdown
Contributor

Fix #16990

Description

This pull request is to address ...

Fixes #

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@saimn saimn added this to the v6.1.4 milestone Sep 11, 2024
@saimn
saimn requested review from astrofrog and mhvk September 11, 2024 10:37
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

out._heapoffset = self._heapoffset
out._heapsize = self._heapsize
out._tbsize = self._tbsize
out._gap = self._gap

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mhvk - I never really understood Numpy subclassing but, isn't there a better way to propagate those custom attributes ?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Normally, this gets done automatically by __getitem__, which by default would call result.__array_finalize__(self), but here on l.516 we first specifically view as a recarray (for reasons commented on but which I don't know couldn't be worked around another way), and therefore one gets result.__array_finalize(self.view(np.recarray)), i.e., the wrong finalize. So, one could avoid the duplication by explicitly doing out.__array_finalize__(self) here (note that I didn't check whether that does exactly the same thing as the code here).

the stated criteria can be found.
"""
raw_data_bytes = self.nbytes + self._heapsize
raw_data_bytes = self._tbsize + self._heapsize

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Main fix is here, with the propagation of heap size and the original array size.

@saimn

saimn commented Sep 11, 2024

Copy link
Copy Markdown
Contributor Author

Seems to create problems with other tests, was a bit optimistic when running only VLA tests 😬 .

@saimn saimn mentioned this pull request Sep 11, 2024
1 task
@pllim pllim added the Bug label Sep 11, 2024
@pllim pllim modified the milestones: v6.1.4, v6.1.5 Sep 27, 2024
@saimn

saimn commented Oct 10, 2024

Copy link
Copy Markdown
Contributor Author

I fixed the issue so tests are now passing, ready to review :)

assert arr.tolist() == [[45, 56]]
assert arr[0].dtype == np.int16

def test_vla_slice2(self):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

couldn't these test be merged into a single parametrized function ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would make things harder to read and understand. If there were more than 2 maybe but here simple is better imo.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fine by me

@astrofrog astrofrog left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@pllim
pllim merged commit 4588eb8 into astropy:main Oct 16, 2024
@pllim

pllim commented Oct 16, 2024

Copy link
Copy Markdown
Member

Thanks, all!

meeseeksmachine pushed a commit to meeseeksmachine/astropy that referenced this pull request Oct 16, 2024
pllim added a commit that referenced this pull request Oct 16, 2024
…996-on-v6.1.x

Backport PR #16996 on branch v6.1.x (Fix access to VLA data after slicing)
@saimn
saimn deleted the fits-16990 branch October 16, 2024 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Slicing FITS_rec objects produces unexpected dtypes (VLA columns)

5 participants