Skip to content

Add function to flatten N-d columns to multiple 1-d columns - #20138

Merged
taldcroft merged 6 commits into
astropy:mainfrom
taldcroft:table-flatten-to-1d-columns
Jul 31, 2026
Merged

taldcroft merged 6 commits into
astropy:mainfrom
taldcroft:table-flatten-to-1d-columns

Conversation

@taldcroft

@taldcroft taldcroft commented Jul 28, 2026 •

Copy link
Copy Markdown
Member

Description

This pull request is to add a new table function represent_nd_columns_as_1d_columns that flattens any N-D
columns into multiple 1-D ~astropy.table.Column or ~astropy.table.MaskedColumn
columns and returns a new table. This can be useful for writing tables to file formats that do not support N-D
columns or converting to a data frame.

In the original discussion in #11148 the idea was a new Table method Table.flatten() that flattens both mixins and N-d columns. This PR takes a more granular approach and makes a new analog of table.serialize.represent_mixins_as_columns. The new function table.serialize.represent_nd_columns_as_1d_columns is exposed in the table package API.

I'm not sure we need a new Table.flatten() method, but that is for discussion. This method would basically just call the represent N-d function and then the represent mixins function and return the result.

Fixes #11148

AI disclosure

I used GitHub Copilot for these bits:

  • Code to deconflict new column names
  • Much of the function docstring
  • All tests
  • Updating table/__init__.py

I have reviewed all the content carefully.

  • By checking this box, the PR author has requested that maintainers do NOT use the "Squash and Merge" button. Maintainers should respect this when possible; however, the final decision is at the discretion of the maintainer that merges the PR.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Astropy! 🌌 This checklist is meant to remind the package maintainers who will review this pull request of some common things to look for.

  • Do the proposed changes actually accomplish desired goals?
  • Do the proposed changes follow the Astropy coding guidelines?
  • Are tests added/updated as required? If so, do they follow the Astropy testing guidelines?
  • Are docs added/updated as required? If so, do they follow the Astropy documentation guidelines?
  • Is rebase and/or squash necessary? If so, please provide the author with appropriate instructions. Also see instructions for rebase and squash.
  • Did the CI pass? If no, are the failures related? If you need to run daily and weekly cron jobs as part of the PR, please apply the "Extra CI" label. Codestyle issues can be fixed by the bot.
  • Is a change log needed? If yes, did the change log check pass? If no, add the "no-changelog-entry-needed" label. If this is a manual backport, use the "skip-changelog-checks" label unless special changelog handling is necessary.
  • Is this a big PR that makes a "What's new?" entry worthwhile and if so, is (1) a "what's new" entry included in this PR and (2) the "whatsnew-needed" label applied?
  • At the time of adding the milestone, if the milestone set requires a backport to release branch(es), apply the appropriate "backport-X.Y.x" label(s) before merge.

@mhvk mhvk left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good idea, and the implementation is nice. For later, I wonder if we shouldn't expose the opposite direction too?

out = represent_nd_columns_as_1d_columns(tbl)
# Original 1-D column is preserved unchanged
assert "a.0" in out.colnames
np.testing.assert_array_equal(out["a.0"], [10, 20, 30])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this stronger, out["a.0"] is tbl["a.0"]? (fine to keep test as is, though)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it is a new table, the column object is new so that test fails. However, the data are shared since the new table is made with copy=False, so I added a test for that.

@taldcroft
taldcroft force-pushed the table-flatten-to-1d-columns branch from 7e8a4df to 167626f Compare July 31, 2026 10:19
@taldcroft
taldcroft enabled auto-merge (squash) July 31, 2026 10:21
@taldcroft
taldcroft merged commit 1ea8f39 into astropy:main Jul 31, 2026
37 of 38 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ascii.write fails with nondescript error on structured (multi-D) columns

2 participants