Skip to content

WIP: Auto-Import User-Defined Cosmology Realizations - #11517

Closed
nstarman wants to merge 2 commits into
astropy:mainfrom
nstarman:cosmo_read_ecsv
Closed

nstarman wants to merge 2 commits into
astropy:mainfrom
nstarman:cosmo_read_ecsv

Conversation

@nstarman

@nstarman nstarman commented Apr 11, 2021 •

Copy link
Copy Markdown
Member

Description

Allow the user to add an ECSV file of custom realizations to a config directory (default in ~/.astropy/cosmologies/) and have them be created at import. Also, I think using the ECSV format will eventually enable cool things like working through an MCMC chain of cosmologies.

For backward compatibility, the file is read and dicts of each row added into parameters.py.

Part of #10673
Blocked by #11530 #11570 #11542 #11559

@embray, this PR relates to our convo on non-entrypoint custom realizations.


OLD Description

Instead of storing parameters as dictionaries in the file parameters, the parameters for built-in realizations are kept in an ECSV. This enables easy comparison of realizations and, more importantly, lays the foundation for the next PR, which is allow the user to add an ECSV file of custom realizations to the astropy config directory and have them be created at import.
Also, I think using the ECSV format will eventually enable cool things like working through an MCMC chain of cosmologies.
For backward compatibility, the file is read and dicts of each row added into parameters.py.

@github-actions

Copy link
Copy Markdown
Contributor

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

@nstarman nstarman mentioned this pull request Apr 11, 2021
2 of 7 tasks
@nstarman

nstarman commented Apr 11, 2021 •

Copy link
Copy Markdown
Member Author

Uh oh! I forgot PyYAML is not a run-time dependency... how can an ECSV file be read?

Might have to re-think this implementation. Sigh. JSON files, I guess ?

@taldcroft, any suggestions how to avoid JSON and keep ECSV?

@taldcroft

Copy link
Copy Markdown
Member

@nstarman - unfortunately I can't think of a way to use ECSV for this case. It really does require YAML, so the choice is either don't use ECSV or make YAML a requirement for cosmology (and hence astropy).

BTW, if you are wondering about why we used YAML not JSON for ECSV.. I often think about this myself:

  • A key showstopper is that JSON does not allow numeric values for a dict key. So it cannot represent arbitrary pure-Python data structures.
  • Much better system for extensibility.
  • More human-readable.

I spent a little time thinking about ways of reading key bits of the YAML header "by hand" without YAML, but at the end I don't think this will be worth it. Sigh.

@taldcroft

Copy link
Copy Markdown
Member

BTW, 👍 on the idea, even if it ends up not working out!

@nstarman

nstarman commented Apr 12, 2021 •

Copy link
Copy Markdown
Member Author

BTW, 👍 on the idea, even if it ends up not working out!

Thanks!

unfortunately I can't think of a way to use ECSV for this case. It really does require YAML, so the choice is either don't use ECSV or make YAML a requirement for cosmology (and hence astropy).

Well, I don't want to add any requirements... So I guess I'll use JSON for astropy's internal storage 😢 .
I think I'll add methods Cosmology.read/write for JSON I/O.

For user-defined cosmologies, I think the ECSV is more logical and the code I have here will work well. Perhaps I'll just transition this PR into one for user-defined cosmologies.

Thanks @taldcroft for the quick reply.

@pep8speaks

pep8speaks commented Apr 12, 2021 •

Copy link
Copy Markdown

Hello @nstarman 👋! It looks like you've made some changes in your pull request, so I've checked the code again for style.

Line 273:1: W293 blank line contains whitespace

Line 9:1: E302 expected 2 blank lines, found 1

Comment last updated at 2021-04-28 20:07:24 UTC

@nstarman nstarman changed the title Cosmology parameter table Auto-Import User-Defined Cosmology Realizations Apr 12, 2021
@pllim

pllim commented Apr 12, 2021

Copy link
Copy Markdown
Member

I guess I'll use JSON

What about FITS? 😆

@nstarman

Copy link
Copy Markdown
Member Author

I'm also planning on Linear A as soon as it's deciphered.

Comment thread setup.cfg Resolved
@nstarman
nstarman force-pushed the cosmo_read_ecsv branch 3 times, most recently from 8109cf2 to 44f38d0 Compare April 14, 2021 01:31
@nstarman nstarman changed the title Auto-Import User-Defined Cosmology Realizations WIP: Auto-Import User-Defined Cosmology Realizations Apr 14, 2021
@nstarman nstarman mentioned this pull request Apr 14, 2021
@embray

embray commented Apr 22, 2021

Copy link
Copy Markdown
Member

@nstarman At some point I'd like to have a discussion to better wrap my head around your cosmology plans. I like where this is headed though and I think you have the right idea. +1 to using YAML to describe cosomologies, and if we need to add pyyaml as a dependency for astropy I would not be opposed.

@nstarman

Copy link
Copy Markdown
Member Author

@embray Adding PyYAML would be awesome. @taldcroft and I had some discussion in a different PR about this... and it would be very nice to have a clear way of serializing quantities.
I would be more than happy to chat about my cosmology plans... perhaps that would help me understand them as well! 😆. I've defined some broad goals, but as you can see from the various WIP PRs, I haven't yet coordinated a specific line of attack. Would this be best done on Slack? There's the languishing cosmology channel.

@nstarman nstarman mentioned this pull request May 23, 2021
2 of 7 tasks
@nstarman nstarman closed this Sep 11, 2021
@nstarman
nstarman deleted the cosmo_read_ecsv branch September 11, 2021 03:53
@nstarman

Copy link
Copy Markdown
Member Author

Worth doing, but this was way outdated and the wrong approach.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

6 participants